varnishncsa

VARNISHNCSA(1)                                                  VARNISHNCSA(1)



NAME
       varnishncsa - Display Varnish logs in Apache / NCSA combined log format

SYNOPSIS
       varnishncsa [-a] [-b] [-c] [-C] [-d] [-D] [-E] [-F <format>] [-f
       <formatfile>] [-g <request|vxid>] [-h] [-L <limit>] [-n <dir>] [-P
       <file>] [-Q <file>] [-q <query>] [-r <filename>] [-R
       <limit[/duration]>] [-t <seconds|off>] [-V] [-w <filename>]

DESCRIPTION
       The varnishncsa utility reads varnishd(1) shared memory logs and
       presents them in the Apache / NCSA "combined" log format.

       Each log line produced is based on a single Request type transaction
       gathered from the shared memory log. The Request transaction is then
       scanned for the relevant parts in order to output one log line. To
       filter the log lines produced, use the query language to select the
       applicable transactions. Non-request transactions are ignored.

       The following options are available:

       -a     When writing output to a file, append to it rather than
              overwrite it.

       -b     Log backend requests. If -c is not specified, then only backend
              requests will trigger log lines.

       -c     Log client requests. This is the default. If -b is specified,
              then -c is needed to also log client requests

       -C     Do all regular expression and string matching caseless.

       -d     Process log records at the head of the log and exit.

       -D     Daemonize.

       -E     Show ESI request.

       -F <format>
              Set the output log format string.

       -f <formatfile>
              Read output format from a file. Will read a single line from the
              specified file, and use that line as the format.

       -g <request|vxid>
              The grouping of the log records. The default is to group by
              vxid.

       -h     Print program usage and exit

       -L <limit>
              Sets the upper limit of incomplete transactions kept before the
              oldest transaction is force completed. A warning record is
              synthesized when this happens. This setting keeps an upper bound
              on the memory usage of running queries. Defaults to 1000
              transactions.

       -n <dir>
              Specify the varnishd working directory (also known as instance
              name) to get logs from. If -n is not specified, the host name is
              used.

       -P <file>
              Write the process' PID to the specified file.

       -Q <file>
              Specifies the file containing the VSL query to use. When
              multiple -Q or -q options are specified, all queries are
              considered as if the 'or' operator was used to combine them.

       -q <query>
              Specifies the VSL query to use. When multiple -q or -Q options
              are specified, all queries are considered as if the 'or'
              operator was used to combine them.

       -r <filename>
              Read log in binary file format from this file. The file can be
              created with varnishlog -w filename.

       -R <limit[/duration]>
              Restrict the output to the specified limit. Transactions
              exceeding the limit will be suppressed. The limit is specified
              as the maximum number of transactions (with respect to the
              chosen grouping method) and an optional time period. If no
              duration is specified, a default of s is used. The duration
              field can be formatted as in VCL (e.g. -R 10/2m) or as a simple
              time period without the prefix (e.g. -R 5/m). When in -g raw
              grouping mode, this setting can not be used alongside -i, -I, -x
              or -X, and we advise using -q instead.

       -t <seconds|off>
              Timeout before returning error on initial VSM connection. If set
              the VSM connection is retried every 0.5 seconds for this many
              seconds. If zero the connection is attempted only once and will
              fail immediately if unsuccessful. If set to "off", the
              connection will not fail, allowing the utility to start and wait
              indefinetely for the Varnish instance to appear.  Defaults to 5
              seconds.

       -V     Print version information and exit.

       -w <filename>
              Redirect output to file. The file will be overwritten unless the
              -a option was specified. If the application receives a SIGHUP in
              daemon mode the file will be reopened allowing the old one to be
              rotated away. This option is required when running in daemon
              mode.

       --optstring
              Print the optstring parameter to getopt(3) to help writing
              wrapper scripts.

MODES
       The default mode of varnishncsa is "client mode".  In this mode, the
       log will be similar to what a web server would produce in the absence
       of varnish.  Client mode can be explicitly selected by using -c.

       If the -b switch is specified, varnishncsa will operate in "backend
       mode".  In this mode, requests generated by varnish to the backends
       will be logged.  Unless -c is also specified, client requests received
       by varnish will be ignored.

       When running varnishncsa in both backend and client mode, it is
       strongly advised to include the format specifier %{Varnish:side}x to
       distinguish between backend and client requests.

       Client requests that results in a pipe (ie. return(pipe) in vcl), will
       not generate logging in backend mode. This is because varnish is not
       generating requests, but blindly passes on bytes in both directions.
       However, a varnishncsa instance running in normal mode can see this
       case by using the formatter %{Varnish:handling}x, which will be 'pipe'.

       In backend mode, some of the fields in the format string get different
       meanings.  Most notably, the byte counting formatters (%b, %I, %O)
       considers varnish to be the client.

       It is possible to keep two varnishncsa instances running, one in
       backend mode, and one in client mode, logging to different files.

FORMAT
       Specify the log format to use. If no format is specified the default
       log format is used:

          %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"

       Escape sequences \n and \t are supported.

       Supported formatters are:

       %b     In client mode, size of response in bytes, excluding HTTP
              headers.  In backend mode, the number of bytes received from the
              backend, excluding HTTP headers.  In CLF format, i.e. a '-'
              rather than a 0 when no bytes are sent.

       %D     In client mode, time taken to serve the request, in
              microseconds.  In backend mode, time from the request was sent
              to the entire body had been received.

       %H     The request protocol. Defaults to HTTP/1.0 if not known.

       %h     Remote host. Defaults to '-' if not known.  In backend mode this
              is the IP of the backend server.

       %I     In client mode, total bytes received from client.  In backend
              mode, total bytes sent to the backend.

       %{X}i  The contents of request header X. If the header appears multiple
              times in a single transaction, the last occurrence is used.

       %l     Remote logname. Always '-'.

       %m     Request method. Defaults to '-' if not known.

       %{X}o  The contents of response header X. If the header appears
              multiple times in a single transaction, the last occurrence is
              used.

       %O     In client mode, total bytes sent to client.  In backend mode,
              total bytes received from the backend.

       %q     The query string. Defaults to an empty string if not present.

       %r     The first line of the request. Synthesized from other fields, so
              it may not be the request verbatim. See the NOTES section.

       %s     Status sent to the client.  In backend mode, status received
              from the backend.

       %t     In client mode, time when the request was received, in HTTP
              date/time format.  In backend mode, time when the request was
              sent.

       %{X}t  In client mode, time when the request was received, in the
              format specified by X.  In backend mode, time when the request
              was sent.  The time specification format is the same as for
              strftime(3).

       %T     In client mode, time taken to serve the request, in seconds.  In
              backend mode, time from the request was sent to the entire body
              had been received.

       %U     The request URL without the query string. Defaults to '-' if not
              known.

       %u     Remote user from auth.

       %{X}x  Extended variables.  Supported variables are:

              Varnish:time_firstbyte
                     Time from when the request processing starts until the
                     first byte is sent to the client, in seconds.  For
                     backend mode: Time from the request was sent to the
                     backend to the entire header had been received.

              Varnish:hitmiss
                     One of the 'hit' or 'miss' strings, depending on whether
                     the request was a cache hit or miss. Pipe, pass and synth
                     are considered misses.

              Varnish:handling
                     One of the 'hit', 'miss', 'pass', 'pipe' or 'synth'
                     strings indicating how the request was handled.

              Varnish:side
                     Backend or client side. One of two values, 'b' or 'c',
                     depending on where the request was made. In pure backend
                     or client mode, this field will be constant.

              Varnish:vxid
                     The VXID of the varnish transaction.

              VCL_Log:key
                     The value set by std.log("key:value") in VCL.

              VSL:tag:record-prefix[field]
                     The value of the VSL entry for the given tag-record
                     prefix-field combination. Tag is mandatory, the other
                     components are optional.

                     The record prefix will limit the matches to those records
                     that have this prefix as the first part of the record
                     content followed by a colon.

                     The field will, if present, treat the log record as a
                     white space separated list of fields, and only the nth
                     part of the record will be matched against. Fields start
                     counting at 1 and run up to 255.

                     Defaults to '-' when the tag is not seen, the record
                     prefix does not match or the field is out of bounds. If a
                     tag appears multiple times in a single transaction, the
                     first occurrence is used.

SIGNALS
       · SIGHUP

         Rotate the log file (see -w option) in daemon mode, abort the loop
         and die gracefully when running in the foreground.

       · SIGUSR1

         Flush any outstanding transactions.

NOTES
       The %r formatter is equivalent to "%m http://%{Host}i%U%q %H". This
       differs from apache's %r behavior, equivalent to "%m %U%q %H".
       Furthermore, when using the %r formatter, if the Host header appears
       multiple times in a single transaction, the first occurrence is used.

EXAMPLE
       Log the second field of the Begin record, corresponding to the VXID of
       the parent transaction:

          varnishncsa -F "%{VSL:Begin[2]}x"

       Log the entire Timestamp record associated with the processing length:

          varnishncsa -F "%{VSL:Timestamp:Process}x"

SEE ALSO
       varnishd(1) varnishlog(1) varnishstat(1) vsl(7)

HISTORY
       The varnishncsa utility was developed by Poul-Henning Kamp in
       cooperation with Verdens Gang AS and Varnish Software AS. This manual
       page was initially written by Dag-Erling Smørgrav <des@des.no>, and
       later updated by Martin Blix Grydeland and PÃ¥l Hermunn Johansen.

COPYRIGHT
       This document is licensed under the same licence as Varnish itself. See
       LICENCE for details.

       · Copyright (c) 2006 Verdens Gang AS

       · Copyright (c) 2006-2016 Varnish Software AS



                                                                VARNISHNCSA(1)