man(1)                      General Commands Manual                     man(1)

       nfsometer - NFS performance measurement tool

       nfsometer [options] [mode] [[<server:path>] [workloads...]]

       nfsometer is a performance measurement framework for running workloads
       and reporting results across NFS protocol versions, NFS options and
       Linux NFS client implementations

       Basic usage (no mode specified):

        nfsometer <server:path> [workloads...]

         This will fetch needed files, run traces, and generate reports,
         same as running the the 'fetch', 'trace' and 'report' stages.

       Advanced usage (specify modes):

        nfsometer list

           List the contents of the results directory.

        nfsometer workloads

           List available and unavailable workloads.

        nfsometer notes

           Edit the notes file of the results directory. These notes will
           be displayed in report headers.

        nfsometer loadgen <server:path> <workload>

           Run in loadgen mode: don't record any stats, just loop over
           <workload> against <server:path>.  Only one -o option is allowed.
           Use the -n option to run multuple instances of the loadgen
           When running more than one instance, the intial start times are

        nfsometer fetch [workloads...]

           Fetch all needed files for the specified workload(s).  If no
           workloads are specified, all workloads are fetched.
           Fetched files are only downloaded once and are cached for
           future runs.

        nfsometer trace <server:path> [workloads...]

           Run traces against <server:path>.  The traces run will be:
           (options + always options + tags) X (workloads) X (num runs)
           This will only run traces that don't already exist in the results

        nfsometer report

           Generate all reports available from the results directory.

        nfsometer example

           Show examples from man page

        -r <dir>, --resultdir=<dir>
            The directory used to save results.  default:

        -o <mount.nfs options>, --options=<mount.nfs options>
            Mount options to iterate through.  This option may be used
            multiple times.  Each mount option must have a version specified.

        -a <mount.nfs options>, --always-options=<mount.nfs options>
            Options added to every trace.  This option may be used multiple

        -t <tags>, --tag=<tags>
            Tag all new traces with 'tags'.  This option may be used multiple

        -n <num runs>, --num-runs=<num runs>
            Number of runs for each trace of <options> X <tags> X <workloads>
            default: 1

            Generate graphs inline while generating reports.  Useful for
            debugging graphing issues.

            Randomize the order of traces

        -h, --help
            Show the help message

       Example 1: See what workloads are available

         $ nfsometer workloads

         This command lists available workloads and will tell you why
         workloads are unavailable (if any exist).

       Example 2: Compare cthon, averaged over 3 runs,
                  across nfs protocol versions

          nfsometer -n 3 server:/export cthon

         This example uses the default for -o: "-o v3 -o v4 -o v4.1".
         To see the results, open results/index.html in a web browser.

       Example 3: Compare cthon, averaged over 3 runs,
                  between v3 and v4.0 only

         nfsometer -n 3 -o v3 -o v4 server:/export cthon

         This example specifies v3 and v4 only.
         To see the results, open results/index.html in a web browser.

       Example 4: Compare two kernels running iozone workload, averaged
                  over 2 runs, across all nfs protocol versions

         nfsometer can compare two (or more) kernel versions, but
         has no way of building, installing or booting new kernels.
         It's up to the user to install new kernels.
         In order for these kernels to be differentiated, 'uname -a'
         must be different.

          1) boot into kernel #1

          2) nfsometer -n 2 server:/export iozone

          3) boot into kernel #2

          4) nfsometer -n 2 server:/export iozone

          5) open results/index.html in a web browser

         To see the results, open results/index.html in a web browser.

       Example 5: Using tags

         Tags (the -t option) can be used to mark nfsometer runs as
         occurring with some configuration not captured by mount options
         or detectable tags, such as different sysctl settings (client side),
         different server side options, or different network conditions.

         1) set server value foo to 2.3

         2) nfsometer -o v4 -o v4.1 -t foo=2.3

         3) set server value foo to 10

         4) nfsometer -o v4 -o v4.1 -t foo=10

         What is passed to -t is entirely up to the user - it will not be
         interpreted or checked by nfsometer at all, so be careful!

         To see the results, open results/index.html in a web browser.

       Example 6: Always options

         The -o flag specifies distinct option sets to run, but sometimes
         there are options that should be present in each.  Instead of
         writing each one out, you can use the -a option:

         nfsometer -o v3 -o v4 -a sec=krb5 server:/export iozone

         this is equivalent to:

         nfsometer -o v3,sec=krb5 -o v4,sec=krb5 server:/export iozone

       Example 7: Using the "custom" workload

         A main use case of nfsometer is the "custom" workload - it allows
         the user to specify the command that nfsometer is to run.

         NOTE: the command's cwd (current working directory) is the runroot
               created on the server.

         export NFSOMETER_CMD="echo foo > bar"
         export NFSOMETER_NAME="echo"
         export NFSOMETER_DESC="Writes 4 bytes to a file"
         nfsometer server:/export custom

         This will run 3 traces (v3, v4, v4.1) against server:/export of
         the command: echo foo > bar.

       Example 8: Using the loadgen mode

        Loadgen runs several instances of a workload without capturing
        traces. The idea is that you use several clients to generate
        load, then another client to measure performance of a loaded
        server. The "real" run of nfsometer (not loadgen) should mark
        the traces using the -t option.

        1) On client A, run the cthon workload to get a baseline of
           a server without any load.

          nfsometer trace server:/export cthon

        2) When that's done, start loadgen on client B:

          nfsometer -n 10 loadgen server:/export dd_100m_1k

           This runs 10 instances of dd_100m_1k workload on server:/export.
           It can take several minutes to start in an attempt to stagger
           all the workload instances.

        3) once all instances are started, run the "real" nfsometer
           trace on client A.  Use the -t option to mark the traces
           as having run under load conditions:

          nfsometer -t "10_dd" trace server:/export cthon

        4) Explain how the tests were set up in the result notes.
           This should be run on client A (which has the traces:

          nfsometer notes

        5) Now generate the reports:

          nfsometer report

       Example 8: Long running nfsometer trace

         The script currently runs in the foreground.  As
         such, it will be killed if the tty gets a hangup or the connection
         to the client is closed.

         For the time being, nfsometer should be run in a screen
         session, or run with nohup and the output redirected to a file.

          1) screen -RD
          2) nfsometer -n 2 server:/export iozone
          3) close terminal window (or ^A^D)
          4) reattach later with screen -RD
          5) once is done, results will be in results/index.html

       mountstats, nfsstats

       No known bugs.

       Weston Andros Adamson (

nfsometer                             1.9                               man(1)