Varnish Statistics One-liners

Varnish Command Line
 
Here are some useful examples of varnishtop and varnishlog at work.

Displays a continuously updated list of the most frequently requested URLs:
varnishtop -i RxURL
varnishlog -c | grep 'RxURL'

Top requests to your backend. This shows only the "TxURL", the URL being retrieved from your backend.
varnishtop -i TxURL
varnishlog -b | grep 'TxURL'

#See what cookies values are the most commonly sent to varnish.
varnishtop -i RxHeader -I Cookie
varnishlog -c | grep 'Cookie: '

#Which host is being requested the most. Only really useful when you're serving multiple hosts in Varnish.
varnishtop -i RxHeader -I '^Host:'
varnishlog -i RxHeader | grep 'Host: '

#Accept-Encoding will show the most popular Accept-Encoding header the client are sending you.
varnishtop -i RxHeader -I

See what useragents are the most common from the clients
varnishtop -i RxHeader -C -I ^User-Agent

See what user agents are commonly accessing the backend servers, compare to the previous one to find clients that are commonly causing misses.
varnishtop -i TxHeader -C -I ^User-Agent

See what accept-charsets are used by clients
varnishtop -i RxHeader -I '^Accept-Charset

Listing all details about requests resulting in a 500/404 status:
varnishlog -b -m "RxStatus:500"
varnishlog -b -m "RxStatus:404"







References

• http://book.varnish-software.com/3.0/Getting_started.html
• https://www.varnish-cache.org/docs/3.0/reference/varnishlog.html
• https://www.varnish-cache.org/docs/3.0/reference/varnishtop.html
• https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html
• https://www.varnish-cache.org/docs/3.0/tutorial/logging.html#tutorial-logging
• https://ma.ttias.be/useful-varnish-3-0-commands-one-liners-with-varnishtop-and-varnishlog/
• http://stackoverflow.com/questions/13247707/how-to-read-output-of-varnishtop
• http://www.eldefors.com/varnish-command-line-tools/
• https://www.varnish-cache.org/trac/wiki/

Varnish and SEO

Varnish is a a great caching solution but it can do more. With Search Engine Optimisation it is always recommended that you have one base URL,often referred to as a Canonical URL,  either www.mysample or mysample domain. Some website owners even have multiple domains. The snippets below show how you can redirect www/non-www to non-www/www and or multiple domains to a Canonical URL.

Varnish 3
in sub vcl_recv add close to the top

    if (req.http.host == "www.mysample.com" || req.http.host == "my-sample.com" || req.http.host == "www.my-sample.com") {
        set req.http.host = "mysample.com";
        error 750 "http://" + req.http.host + req.url;
    }

in sub vcl_error add
   if (obj.status == 750) {

        set obj.http.Location = obj.response;
        set obj.status = 301;
        return(deliver);
    }

Varnish 4
in sub vcl_recv add

    if (req.http.host ~ "^www.mysample.com") {
        return (synth (750, ""));
    }

in sub vcl_synth

    if (resp.status == 750) {
        set resp.status = 301;
        set resp.http.Location = "http://mysampele.com" + req.url;
        return(deliver);
    }

Actually I like the implementation in Varnish 4 better. As you can make all the related changes at one place instead at 2 locations in Varnish 4. This also helps improving your memory used as only a single option is stored in cache instead of one for www.mysample.com/index.html and another for mysample.com/index.html

Hope this helps someone

Source
How to redirect non-www URLs to www in Varnish

Using Varnish to block access to specific folders

If you ever need to block folder or folders using Varnish Cache, here are the simple steps.

Edit  sub vcl_recv and add the following

sub vcl_recv {
  # Ban outside access to #/user, /admin etc
  # works if you : if (req.url ~ "^/user" || req.url ~ "^/admin") {
  if ( (req.url ~ "^/user" || req.url ~ "^/admin" ) && !client.ip ~ yourallowedip) {
      # Have Varnish throw the error directly.
       error 405 "Sorry";
    }
#Other code
#....
}

Create...

acl yourallowedip {
    "1.1.1.1";
}

Restart varnish and you will be good to go.

service restart varnish

It is always good to test your configuration before restarting Varnish. The command to do is below. It there is an error it will let you know otherwise you will get a long display.

varnishd -C -f /etc/varnish/default.vcl

References

• Lockdown Wordpress
• Varnish Command Line tools
• Practical Guide to Varnish