Distributed Checksum Clearinghouse (DCC) Installation

1. Fetch the Source and Read the License

   The DCC source is available at dcc-servers.net and Rhyolite Software.

   Please do not try to use the ancient, modified versions of DCC software distributed by some Linux packagers. Those versions do not detect bulk mail as well as more recent versions. Installations using those old versions also have problems using the public DCC servers that often make it necessary to add their IP addresses to the blacklist that protects the public DCC servers. Even worse, all known Linux redistributions of DCC software have been changed in ways that break things, including the /var/dcc/libexec/updatedcc shell script that could otherwise be used to fetch, configure, compile, install, and restart a current version.

   The license on the free source is in the source as well as dcc-servers.net. The free license is intended to cover individuals and organizations including Internet service providers using DCC to filter their own mail. Organizations selling anti-spam appliances or managed mail services are not eligible for the free license.

2. Read the Documentation

   The DCC and other man pages describe the features, operating modes, required data files, and other characteristics of the DCC. Also see the DCC FAQ or list of frequently answered questions.

3. Build Sendmail

   If the DCC-sendmail interface, dccm, is not used, then skip to the next step.

   Sendmail must have the Mail Filter API or Milter enabled. Some systems such a FreeBSD 4.6 and newer are shipped with Milter enabled and the library installed by default. If your system comes with the Milter interface turn on, then skip to the next step. Otherwise, the Milter interface must be explicitly enabled by adding lines like those in misc/site.config.m4 to your sendmail/devtools/Site/site.config.m4 file or equivalent. Then build sendmail as described in the INSTALL file distributed with sendmail. You must build libmilter separately by something like

   	    cd libmilter
   	    sh ./Build
   

   After sendmail has been rebuilt if necessary it will need to be restarted. That should be done after the next step after misc/dcc.m4 has been created by the ./configure script.

4. Configure, Build, and Install the DCC Programs

   See the installation considerations in the DCC man page.

   Most DCC files are in a "home directory" such as /var/dcc. DCC programs such as cdcc and dccproc are run by end users and should be installed in a directory such as /usr/local/bin. They must also be set-UID to the UID that can change the DCC data files. DCC programs that do not need to be run by end users are installed by default in the libexec subdirectory of the DCC home directory. See the table of ./configure script and makefile parameters. If necessary, set CFLAGS, LDFLAGS, LIBS or other environment variables listed in the table. Omit any parameters you don't really need to change.

   End users installing only dccproc can install it in their private ~/bin directories and use private directories for their DCC home directories. In this case, the DCC programs that would otherwise need to be set-UID need not be.

   To build dccproc for an individual user, use something like

   	./configure --disable-sys-inst --disable-dccm --homedir=$HOME/dccdir  --bindir=$HOME/bin
   	make install
   

   If you do not intend to use the sendmail interface, dccm, use something like

   	./configure --disable-dccm
   	make install
   

   If you will use dccm, it must be built with the sendmail source and object tree. By default, the makefiles look for native sendmail libraries (e.g. on FreeBSD), an installed "package" (e.g. on FreeBSD), or a directory named sendmail parallel to the DCC source and object tree. Those who regularly build new versions of sendmail may find it convenient to make a symbolic link there to their current sendmail. Otherwise configure the dccm makefile with

   	./configure --with-sendmail=/some/where/sendmail
   	make install
   

   If dccm does not build because it cannot find libmilter, check that libmilter was compiled with sendmail in the previous step.

   To connect the sendmail Milter interface to dccm, copy or symbolically link misc/dcc.m4 to your sendmail/cf/feature directory and add FEATURE(dcc) lines to your sendmail.mc configuration file. It can be useful to modify sendmail.cf by using misc/hackmc. Then rebuild and reinstall your sendmail.cf file, and restart sendmail.

5. Create Client Configuration Files

   All DCC configuration files are in the DCC home directory, usually /var/dcc. See the dcc, dccm, dccifd, and dccproc man pages for the files each needs. Example files are in the homedir directory in the source.
   • Unless run anonymously, DCC clients need client-ID numbers and passwords assigned by the operators of the chosen DCC servers in the /var/dcc/map file.

   • Even if run anonymously, the /var/dcc/map file must contain the IP addresses of DCC servers. If your mail system handles fewer than 100,000 mail messages per day, the installation process generates a serviceable /var/dcc/map file from the included homedir/map.txt file. That file points to the public DCC servers.

   • If using remote DCC servers such as the public DCC servers, ensure that your firewalls allow outgoing packets to UDP port 6277 on distant systems and incoming responses from UDP port 6277. There is a description one firewall's configuration.

   • Your MX servers and mail submission clients should be listed in the main whiteclnt file with lines like:

         mx		ip  10.2.3.4
         mx		ip  10.5.6.0/28
         mxdcc       ip  10.5.6.0/28
         ok		ip  10.7.8.9
         submit      ip  192.168.1.0/24
     

     If those other systems also run DCC clients, use MXDCC instead of MX so that messages will not be reported twice to the DCC network and so have higher target counts, and appear to be unsolicited bulk mail.

     Use OK for mail systems that you trust to never send or forward unsolicited bulk mail.

     Untrusted SMTP clients such as end users with browsers used as MUAs (mail user agents) should be listed in the whiteclnt file with submit.

   • Sources of legitimate bulk mail must be recorded in whitelists. Example whiteclnt, whitelist, and common whitelists are among the sample configuration files in the homedir directory. The format of DCC whitelists is described in the DCC man page.

   • Put suitable values in the DCC configuration file, /var/dcc/dcc_conf for dccm or dccifd. The default client values are usually good for a start and often only DCCM_REJECT_AT needs to be changed when it is time to reject spam.

   • Optionally configure DNS blacklist (DNSBL) checks in dccm or dccifd by setting DNSBL_ARGS in in the configuration file, /var/dcc/dcc_conf, in the home directory.

   • Optionally create per-user directories for logs and whitelists. See also the CGI scripts in /var/dcc/cgi-bin that allow users to maintain their private whitelists and monitor their individual logs of rejected mail.

   • Install a daily or more frequent cron job like misc/crontab and /var/dcc/libexec/cron-dccd to prune dccm or dccifd log files and the prune dccd database with dbclean.

6. Create Server Files and Start the Server

   Skip this and the next step if only remote DCC servers will be used. You should use your own, local DCC servers only if your mail system handles more than 100,000 mail messages per day. If your mail system handles fewer than 100,000 mail messages per day, you should use the public DCC servers. The DCC client programs are configuerd to use the public DCC servers by default.

   It is best to use remote servers until the DCC client, dccm, dccifd, or dccproc, is stable. Then

   • Put suitable values for dccd in the configuration file, /var/dcc/dcc_conf. Every DCC server requires a unique server-ID. Obtain a server-ID by contacting Vernon Schryver at vjs@rhyolite.com by email or via a web form.

     On Linux or Solaris, consider adding -H to DBCLEAN_ARGS in /var/dcc/conf. See the dbclean man page.

   • Choose a secret password for your server-ID in your /var/dcc/ids file. This password is used to control your server with the cdcc program. Your server-ID must be unique and can be obtained by contacting Vernon Schryver at vjs@rhyolite.com by email or via a web form.

   • If you have more than one DCC server, ensure that there are common client-IDs with the same passwords in the /var/dcc/ids files on all of your servers You should define a DNS name like dcc.example.com with A and AAAA records for all of your DCC servers. Then configure your DCC clients to spread their load among your DCC servers with a command like

     	cdcc "dcc.example.com  RTT-1000 ms  32768 secret"
     

     where 32768 and secret are the common client-ID and password found in /var/dcc/ids on all of your DCC servers.

   • Start the server with the system by installing /var/dcc/libexec/rcDCC or an equivalent. If it is used unchanged, rcDCC is best installed with a symbolic link to automate installing updates. The server can be started manually with

     	rcDCC start
     

   • The script /var/dcc/libexec/cron-dccd must be used to run dbclean about once a day. An entry like misc/crontab can be put into the crontab file for the user that runs dccd. If you have more than one DCC server, stagger the times at which the cron job is run so that not all of your servers are simultaneously busy cleaning databases.

   • Install the shutdown script /var/dcc/libexec/rcDCC to shut down the DCC server as the operating system stops. If the DCC server fails to close the database cleanly, the database must be cleaned by the server with it starts. That takes time.

7. Configure Flooding

   Skip to the next step if only remote DCC servers will be used.

   Flooding requires that every server participating in a network of DCC servers have a unique server-ID. Server-IDs can be obtained by contacting Vernon Schryver at vjs@rhyolite.com by email or via a web form.

   After you have an official server-ID,

   • Obtain the passwd-ID and its password and add them to your /var/dcc/ids file.
   • If necessary adjust your firewalls to allow both incoming TCP connections to port 6277 on your DCC server and outgoing TCP connections to port 6277 on your flooding peer.
   • Add a line for each flooding peer to the /var/dcc/flod file.
   • Wait a few minutes for dccd to notice the change to the file and start flooding. The cdcc stats, cdcc "id X; flood list" and /var/dcc/libexec/dblist -Hv commands can be used to monitor the floods of reports of checksums of bulk mail.

   Flooded reports of bulk email contain timestamps that are used for several things including expiring old reports. To accurately detect stale incoming reports, a DCC server needs a clock that is not too inaccurate. For that reason it is good to run an NTP daemon on systems running DCC servers.

8. Configure Greylisting

   Skip to the next step if greylisting will not be used. Greylist is very effective. See this description.

   Larger sites can use more than one greylist server, with the greylist servers flooding data just like DCC servers.

   To configure greylisting:

   1. Assign greylist client-IDs and server-IDs

      Client-IDs and matching passwords must be used by clients of greylist servers such as dccm and dccifd. The client-IDs must be in the /var/dcc/map file on the client system. Greylist client-IDs and server-IDs must be in the /var/dcc/ids file on the greylist server. When a system hosts both DCC and greylist servers, it is convenient for clients to use the same client-ID and password for both. It is also convenient for a greylist server and a DCC server on a system to share a common server-ID and password.

      The vast majority of installations do not have local DCC servers and can use the greylist server-ID generated automatically in the /var/dcc/ids file.

   2. Add the greylist server to /var/dcc/map

      If the cdcc "info" command does not show the correct greylist server, add it with something like

      	cdcc "add localhost greylist 32768 secret"
      

      The DCC makefile files add a greylist server at localhost or 127.0.0.1 to /var/dcc/map file created for a new DCC installation.

   3. Set /var/dcc/dcc_conf

      In most installations, enable a local greylist server by installing the script /var/dcc/libexec/rcDCC with a symbolic link, setting GREY_ENABLE=on in /var/dcc/dcc_conf and then running

      	/var/dcc/libexec/rcDCC start
      

      If absolutely necessary, override the greylist embargo, wait, and white values in GREY_DCCD_ARGS in /var/dcc/dcc_conf. Usually simply set GREY_CLIENT_ARGS=on

   4. Set /var/dcc/grey_flod

      Sites with more than one greylist server should arrange to flood data among them by adding lines to /var/dcc/grey_flod files in the same format as /var/dcc/flod files. Flooding among greylist servers uses port 6276 by default, and so that port may need to be opened in firewalls.

   5. Set cron job

      Install a daily cron job like misc/crontab and /var/dcc/libexec/cron-dccd to clean the database.

   6. Whitelist Mail Submission Clients

      Greylisting of local mail systems must be turned off because common mail user agents (MUAs) cannot handle temporary rejections. One way to turn off greylisting of local client is with submit lines in the main whiteclnt file as described above.

      An alternative to whitelisting mail submission clients is available with dccm and sendmail by using the misc/hackmc -T script to modify sendmail.cf to trust SMTP clients authenticated with SMTP-TLS or SMTP-AUTH.

9. Start dccm

   If the DCC-sendmail interface, dccm, is not used, skip to the next step.

   The DCC sendmail milter interface dccm should be started before sendmail. That often requires changing an /etc/rc script or configuration file. The script /var/dcc/libexec/rcDCC should be installed, best with a symbolic link. The milter daemon can be started manually with

   	rcDCC start
   

10. Start dccifd

    If the general MTA interface, dccifd, is not used, skip to the next step. If you are using SpamAssassin, then you almost certainly should be using dccifd.

    The general MTA interface dccifd should usually be started before the mail transfer agent or MTA. It should be enabled by setting DCCIFD_ENABLE=on in /var/dcc/dcc_conf. It is also usually necessary to change an /etc/rc script or configuration file to start and stop the daemon with the system. The script /var/dcc/libexec/rcDCC should be installed, best with a symbolic link. The daemon can be started manually with

    	rcDCC start
    

    Dccifd can be used as a Postfix Before-Queue Content filter as described the dccifd man page.

11. Configure Uses of dccproc

    If dccproc is used with procmail, add rules to procmailrc files as described in the dccproc man page.

12. Adjust Rejection Thresholds

    It is best to only mark mail with X-DCC SMTP headers before changing procmail or dccm to reject mail. Configure dccm with DCCM_LOG_AT in /var/dcc/dcc_conf to log bulk mail with somewhat lower counts.

13. Additional Considerations

    Some additional mechanisms are available in the DCC client programs. They are often unnecessary when greylisting is used.

    • DNS blacklists (DNSBL) can reject messages containing "spamvertised" URLs.
    • DCC reputations are available in the commercial version of the DCC source.

    When possible, it is almost always better to use dccifd than dccproc. This is certainly true with SpamAssassin. When using SpamAssassin, ensure that the SpamAssassin plugin DCC.pm is up to date. The DCC source includes a copy in the misc directory. Please consider setting dcc_learn_score to report spam to other SpamAssassin with DCC users.

14. Update As Needed

    New versions released at the usual place can be installed by running the /var/dcc/libexec/updatedcc script. That script is (re)built by the ./configure script and runs ./configure with parameters and environment variables from the previous installation.

15. Remove or Uninstall

    Most of the DCC can be removed by running /var/dcc/libexec/uninstalldcc script. Some logs and configuration files with locally chosen parameters in the home directory are not deleted. Manual changes such as links to /var/dcc/libexec/rcDCC or the installation of the cron job, /var/dcc/libexec/cron-dccd, are not reversed.

Installation Parameters

There are several installation configuration parameters that can set to suit individual preferences and systems.

Note¹

These values are not built into the Makefiles by the ./configure script but their current values in the environment are used by the script and the Makefiles.

Note²

These values are copied by the ./configure script from the environment into the generated Makefiles.

Note³

When --disable-sys-inst is specified, the current UID and GID become the defaults, and the man pages are not installed. If the ./configure script is not run as root, dccproc, cdcc, and dccsight are not installed set-UID. It is usually also necessary to set --bindir to a private directory such as $HOME/bin.

Compatibility

DCC is thought to work on several systems including:

BSDI BSD/OS

DCC works starting with version 3.0 of BSD/OS.

FreeBSD

The works starting with at least version 4.0 of FreeBSD.

NetBSD

DCC works without change starting with at least NetBSD 4.0.

OpenBSD

DCC works starting with at least 2.9 despite the lame mmap() implementation.

Linux

Avoid the several years old, modified versions shipped by some Linux packagers.

On 64-bit PowerPC systems with more than 4 GBytes, use ./configure --with-64-bits to build a DCC server that can benefit from a full sized database. A 64-bit sendmail milter library will be needed if dccm is used

AIX

DCC on 4.1.PPC has been tried but not well tested. Rumor has it that the 4.1.PPC pthreads code does not work with the sendmail milter library and dccm, but the rest of DCC does work.

Solaris

DCC compiles on several versions of Solaris with gcc or native C compiler by setting the environment variable CC appropriately. You must install gmake. Do not use "CFLAGS=-fast" with the native compiler.

While building the sendmail milter library, consider using _FFR_USE_POLL to avoid problems with large file descriptors and select().

On 64-bit systems with more than 4 GBytes, use ./configure --with-64-bits to build a DCC server that can benefit from a full sized database. A 64-bit sendmail milter library will be needed if dccm is used

HP-UX

DCC compiles on versions of HP-UX starting with 11.00. It requires gmake. Dccproc and dccm work. Dccifd does not work with UNIX domain sockets because select() and poll() do not notice the results of shutdown(). Dccifd does work with TCP/IP connections to MTAs or spam filters.
Dccproc should work on version 10.20, since it does not use pthreads.

IRIX

DCC compiles on IRIX 6.5. It requires gmake.

OSF1

DCC compiles on OSF1 V5.0 with gmake.

OpenUNIX

DCC compiles on OpenUNIX 8.0.1.

Mac OS/X

DCC compiles on at least some versions of Apple's OS/X.

Windows

The DCC client dccproc compiles and works on at least some versions of Microsoft Windows 98, XP, and Vista with Borland's free SDK and with Microsoft's SDK. See the main Makefile for Windows.

Those system names include trademarks. Please don't abuse them.

Troubleshooting

Much of the DCC list of frequently asked questions concerns troubleshooting DCC installations. Many of the messages in the archive of the DCC mailing list are also troubleshooting questions and answers.

Spam Traps

Dccm and sendmail can be configured to report the checksums of unsolicited bulk mail so that other DCC clients can reject later copies of the same unsolicited bulk mail sent from other sources. Such mechanisms are commonly called spam traps.

Entries in a sendmail access_db can also be rejected or discarded while they are reported to the DCC server by dccm. The script misc/hackmc modifies the output of sendmail .mc files to tell dccm about some undesirable mail. The script accepts one or more .mc files and generates the corresponding slightly modified .cf files. If the access_db entry starts with the string "DCC:", the message is reported by dccm to the DCC server as extremely bulky. Otherwise the message is rejected as usual. The remainder of the the access_db entry after "DCC:" consists of the optional string "DISCARD" followed by an optional SMTP status message. If the string "DISCARD" is present, the message is discarded instead of rejected. This is important to keep senders of unsolicited bulk mail from discovering and removing "spam trap" addresses from their target lists.

For example, a line like the following in an access_db can discard all mail from example.com while reporting it to the DCC server as extremely bulky. Note the quotes (").

    example.com     DCC: "DISCARD spam"

Spam traps can be configured with option spam-trap-discard and option spam-trap-accept lines in whiteclnt files.

Mail from a spam trap address can be sent to dccproc as described in the dccproc man page

SOCKS

The DCC client and server programs can be built to use the SOCKS protocol. The --with-socks ./configure parameter configures the DCC client library and the DCC server to use common SOCKS network library functions. If the SOCKS library is in a standard place, something like --with-socks=socks should be sufficient. Setting the environment variable DCC_LDFLAGS to something like -L/usr/local/lib is sometimes helpful. Otherwise, using --with-socks without specifying the library name and setting LIBS to the full pathname of the library might work.

DCC client programs including dccproc and dccm that use the DCC client library must be told to use the SOCKS5 protocol with the SOCKS on operation of cdcc. SOCKS5 is required instead of SOCKS4 because DCC clients communicate with DCC servers using UDP.

DCC servers can use SOCKS4 or SOCKS5 when exchanging floods of reports of checksums. Links between individual pairs of peers are configured with the passive and SOCKS flags in the flod file described in the dccd man page. In both cases, the SOCKS library code must be configured, often in the files /etc/socks.conf and /etc/socksd.conf.

When the DCC software is built with SOCKS, IPv6 name resolution is turned off.

The DCC server and client programs have been tested with the DANTE library and server. The DANTE SOCKS implementation is also one of the FreeBSD "ports" or packages.

Note that if a connection fails repeatedly, Dante will disable the rule that failed and will eventually try the underlying connect() call. This fails in almost every SOCKS environment because there is no available route for an ordinary connect(). Dante by default won't re-enable the failing rule. To fix this, change BADROUTE_EXPIRE from the default of 0*60 to 5 in include/config.h in the Dante source and recompile.

This document describes DCC version RHYOLITE_VERSION.