dnewslink

DNEWSLINK(8)                System Manager's Manual               DNEWSLINK(8)



NAME
       dnewslink - Diablo program to feed news to remote systems

SYNOPSIS
       dnewslink -s xxx...xxxx -n -i -D -f -p -P port -d[#] -b
       batchfile/template -r -S beginSeq -N numSeq -H -L -A[classes] -h
       remote-host -w delay -t timeout -l logFrequency -c reconnectFrequency
       -T txbufsize -R rxbufsize -I rxbufsize -B ip


DESCRIPTION
       DNewslink generally takes queue files from the dqueue directory that
       were generated by the diablo server and uses them to transfer articles
       from the news spool to a remote host.  There are a horrendous number of
       options covering four modes of operation.  DNewslink can transmit
       articles based on a discrete queue file, can transmit using multiple
       queue files based on a template, can transmit based on input from a
       pipe, or can sit on a realtime queue file to transmit a feed in near
       real time.  DNewslink will usually rewrite the queue file with any
       entries that had to be defered.  When DNewslink connects to a remote
       host, it will attempt to negotiate a check/takethis streaming
       connection and, if that fails, will fallback to the non-streaming ihave
       protocol.

       DNewslink locks queue files as it processes them, allowing multiple
       dnewslinks to operate in parallel on different queue files without the
       possibility of creating a conflict.  DNewslink uses a sophisticated
       non-blocking I/O model to keep the TCP pipeline full for streaming
       feeds and is capable of generating serious bandwidth.  The options are
       described below.


       -s xxx...xxxx

       This option reserves space in the argv array, allowing dnewslink to
       store realtime status in it such that it shows up in a ps.  At least 23
       characters must be given to reserve sufficient space.


       -n

       This option is a NOP.  It does nothing, but is often used by other
       programs (aka dspoolout) to placehold optional arguments.


       -i

       This option disables the streaming negotiation, forcing the non-
       streaming ihave protocol to be used.  If not given, DNewslink will
       attempt to negotiate the streaming protocol and fallback to the non-
       streaming protocol if the negotiation fails.


       -D

       This option causes dnewslink to detect when the batch file it is
       operating on is deleted out from under it.  DNewslink will not attempt
       to rewrite the batchfile, or will rewrite and remove the batchfile if
       it detects this case and the -D option is given.


       -f

       This option has been depreciated.


       -p

       This option specifies that the input is coming from a pipe rather then
       the batchfile (see -b option).  A batchfile must still be specified,
       but it will be used solely for the rewriting of defered lines.


       -P port

       This option specifies the port on the destination host dnewslink should
       connect to.  The default is port 119 (nntp).


       -d [level]

       This option turns on debugging.  Normally something like -d2 is
       specified when dnewslink is run manually in order to trace the protocol
       interaction.


       -b batchfile/template

       This option specififes the batchfile.  The batchfile specification
       depends on the mode of operation.  For a pipe input, this is the spill
       file.  With a normal batch, this is simply a batch file such as
       'label.S04522'.  When combined with -S, this is a template (usually
       'label.S%05d').  For a realtime feed, this is the name of the realtime
       queue file... usually just 'label'.


       -r

       This option turns on realtime operation, indicating that the batchfile
       is being actively appended to by another program (aka the diablo
       server).  This option enables a number of optimizations including
       detecting when dspoolout flushes/renames the active file.  When this
       occurs, DNewslink will attempt to reopen/lock the new realtime queue
       file and, when this succeeds, will delete the renamed file.  However,
       if deferals had to be made, DNewslink will not remove the renamed file
       which usually results in dspoolout rerunning the file as a normal spool
       file.


       -S beginSeq

       This option turns on multi-batch sequencing, causing DNewslink to
       attempt to feed several batch files in sequence.  If given, you must
       specify the beginning sequence number in this option and the batchfile
       name (-b option) must contain a template rather then a filename.


       -N numSeq

       This option specifies the number of sequential sequence numbers to
       attempt to feed out.  DSpoolout usually specifies 32 here.  If there
       are more then 32 batch files or if dnewslink gets interrupted before
       completion, dspoolout will simply start another one.  For realtime
       batches dspoolout specifies a higher number which reflects the number
       of new queue files it is allowed to rotate through before exiting.  For
       queue-delayed batches (qNNNN option in dspoolout.ctl), dspoolout will
       specify an appropriate number based on the number of queue files.


       -H

       This option tells dnewslink to generate a header-only feed, used when
       feeding the diablo reader system and often coupled with diablo support
       for Xref: headers to assign article numbers.  The remote end must
       support sent to a normal feed and propogating outward.  Beyond that,
       the header-only feed uses the same ihave/check/takethis protocols that
       a normal feed uses.


       -L

       This option will enable generation of Lines: headers in headfeed mode
       (see above); the reason for this option is to always have a meaningful
       lines count in overview. Specifying the 'genlines' option in
       dnntpspool.ctl will make dspoolout pass -L to dnewslink.


       -h remote-host

       This option specifies the remote system to connect to.  See also: -P
       option.


       -w delay

       This option specifies the connection fail / reconnection delay in
       seconds.  The default is 10 seconds.


       -t timeout

       This option specifies the inactivity timeout.  The default is 10
       minutes.


       -l logFrequency

       This option specifies the log frequency.  The default is to log every
       1000 transactions.


       -c reconnectFrequency

       This option specifies the forced disconnect/reconnect frequency to
       cause the remote end to generate a log line.  The default is to close
       and reopen the connection every 1000 transactions.


       -T txbufsize

       -R rxbufsize

       These options set the TCP receive and transmit buffer sizes.  DNewslink
       will impose a minimum receive buffer size to prevent pipeline lockups.
       This option is normally used to tune the system.  You can usually
       reduce the receive buffer size to 4096 bytes without effecting
       efficiency.  Some people may want to increase the transmit buffer size
       from the system defaults.  These same options can be given to dspoolout
       and will be passed to dnewslink.

       -I

       This option disables the "check" NNTP command.  Rather then check each
       message id, dnewslink will simply push the article out with "takethis",
       assuming a streaming feed was negotiated.  Useful for header-only
       feeds.

       -A [classes]

       This option turns on article logging to <logpath>/feedlog.<hostname>.
       Classes is is either comma-separated list of any combination of
       "accept", "reject", "defer", "refuse" and "error", or "all". Not that
       specifying "refuse" can generate huge logfiles. Default is "all" if
       classes is not specified, and no logging if this option is not given.

       -B

       This option sets the source ip address for outbound connections. It can
       be either an ip address or a hostname which will be resolved by DNS.


TYPICAL USE
       DNewslink is run by dspoolout and usually operated manually only for
       debugging purposes.  Typical usage is something like: dnewslink -b
       fubar.S23343 -h news.fubar.com


SEE ALSO
       diablo(8), dicmd(8), didump(8), diload(8), dnewslink(8), doutq(8),
       dexpire(8), dexpireover(8), diconvhist(8), dilookup(8), dspoolout(8),
       dkp(8), dpath(8), diablo-kp(5), diablo-files(5)

                                                                  DNEWSLINK(8)