Viewing file:  mod_digest.html (12.92 KB) -rw-r--r--
Select action/file-type:
 (+) | (+) |  (+) | Code (+) | Session (+) |  (+) | SDB (+) |  (+) |  (+) |  (+) |  (+) |  (+) |
ProFTPD module mod_digest
ProFTPD module mod_digest
The mod_digest module offers functionality for calculating the hash
(or digest) value of files. This is particularly useful when verifying
the integrity of files. This functionality is used by the following custom
FTP commands:
XCRC (requests CRC32 digest/checksum)
MD5/XMD5 (requests MD5 digest/checksum)
XSHA/XSHA1 (requests SHA1 digest/checksum)
XSHA256 (requests SHA256 digest/checksum)
XSHA512 (requests SHA512 digest/checksum)
In addition, mod_digest supports the more modern HASH command.
Depending on the file size and the hash function, it takes a fair amount of
CPU and IO resources to calculate the result. Therefore decide wisely where
to enable the features and set the DigestMaxSize
configuration directive appropriately.
This module was compiled and tested against ProFTPD 1.3.3 Installation
instructions are discussed here.
The most current version of mod_digest is distributed with the
ProFTPD source code.
Author
Please contact TJ Saunders <tj at castaglia.org> with any
questions, concerns, or suggestions regarding this module.
Thanks
2016-01-09: Thanks to Mathias Berchtold <mb at
smartftp.com> for his original mod_digest, upon which this
version is based.
Directives
Syntax: DigestAlgorithms ["crc32"|"md5"|"sha1"|"sha256"|"sha512"|"all"]
Default: DigestAlgorithms all
Context: server config, <VirtualHost>, <Global>, <Anonymous>
Module: mod_digest
Compatibility: 1.3.6rc2 or later
The DigestAlgorithms directive configures the enabled digest
algorithms. If no DigestAlgorithms directive is configured, then
all supported digest algorithms are enabled.
Enabled digest algorithms are announced/discovered via the FEAT
response.
The following algorithms are currently supported by mod_digest:
crc32 (e.g. for the XCRC command)
md5 (e.g. for the XMD5 command)
sha1 (e.g. for the XSHA/XSHA1 commands)
sha256 (e.g. for the XSHA256 command)
sha512 (e.g. for the XSHA512 command)
Syntax: DigestCache on|off|"size" count ["maxAge" secs]
Default: DigestCache size 10000 maxAge 30s
Context: server config, <VirtualHost>, <Global>, <Anonymous>
Module: mod_digest
Compatibility: 1.3.6rc2 or later
The mod_digest module will cache the results of any checksum
command, on a per-file basis. This improves performance, and reduces
computational overhead. To disable this caching for any reason, use this
directive:
# Disable checksum caching
DigestCache off
This is not recommended.
The DigestCache directive can also be used to configure/tune the
max-size of the in-memory cache. Note that once the maximum cache
size is reached, any checksum FTP commands will be temporarily refused:
# Use a smaller cache size
DigestCache size 100
Cached digests will be expired/ignored after 30 seconds, by default. To change
the expiration, you would use:
# Retain cached entries longer
DigestCache maxAge 60s
If on is used, mod_digest will use the default
max-size of 10000:
DigestCache on
Syntax: DigestDefaultAlgorithm algo
Default: DigestDefaultAlgorithm sha1
Context: server config, <VirtualHost>, <Global>
Module: mod_digest
Compatibility: 1.3.6rc3 or later
The default digest algorithm that the mod_digest module uses,
for e.g. opportunistic digesting of file transfers, is SHA1. For
selecting a different default algorithm, use the
DigestDefaultAlgorithm directive:
# Use MD5 rather than SHA1 as the default algorithm
DigestDefaultAlgorithm md5
Note that the DigestAlgorithms directive takes precedence;
if the DigestDefaultAlgorithm is not included in the
DigestAlgorithms, the default algorithm setting will be ignored.
Syntax: DigestEnable on|off
Default: Non
Context: <Directory>, .ftpaccess
Module: mod_digest
Compatibility: 1.3.6rc2 or later
The DigestEnable directive can be used to block or prevent
checksumming/digests on files in the configured <Directory>.
This can be very useful for preventing checksumming of files located
on network-mounted filesystems, for example.
Syntax: DigestEngine on|off
Default: DigestEngine on
Context: server config, <VirtualHost>, <Global>, <Anonymous>
Module: mod_digest
Compatibility: 1.3.6rc2 or later
The DigestEngine directive enables or disables the handling of
the checksum-related FTP commands by mod_digest, i.e.:
XCRC
XMD5
XSHA
XSHA1
XSHA256
XSHA512
If the parameter is off, then these commands will be ignored.
Syntax: DigestMaxSize number [units]
Default: None
Context: server config, <VirtualHost>, <Global>, <Anonymous>
Module: mod_digest
Compatibility: 1.3.6rc2 or later
The DigestMaxSize directive configures the maximum number of bytes
a single hash command is allowed to read from a file. If the number of bytes
to be read from the file is greater than the configured number the
server will refuse that command.
If no DigestMaxSize directive is configured, then there is no
limit. It is highly recommended to set an upper limit.
Example:
# Limit hashing to 1GB of data
DigestMaxSize 1 GB
Syntax: DigestOptions opt1 ...
Default: None
Context: server config, <VirtualHost>, <Global>
Module: mod_digest
Compatibility: 1.3.6rc2 and later
The DigestOptions directive is used to configure various optional
behavior of mod_digest.
The currently implemented options are:
NoTransferCache
The mod_digest module will automatically calculate and
cache the results of any transferred file, on a per-file basis. This is
done assuming that many FTP clients will want to verify the integrity of
the file just uploaded/downloaded. This improves performance, and
reduces computational overhead. To disable this caching for any reason,
use this option. Not recommended.
Note: The NoTransferCache option is
automatically enabled when using ProFTPD versions before
1.3.6rc2, due to bugs/missing support in the older versions.
The mod_digest module is distributed with ProFTPD. Follow the
normal steps for using third-party modules in ProFTPD:
$ ./configure --enable-openssl --with-modules=mod_digest
To build mod_digest as a shared/DSO module:
$ ./configure --enable-dso --enable-openssl --with-shared=mod_digest
Then follow the usual steps:
$ make
$ make install
Alternatively, if your proftpd was compiled with DSO support, you can
use the prxs tool to build mod_digest as a shared
module:
$ prxs -c -i -d mod_digest.c
Usage
Example Configuration
<IfModule mod_digest.c>
# Set a limit on file sizes that can be digested
DigestMaxSize 1 GB
</IfModule>
Recording Uploaded/Downloaded File Checksums
One particular use case that comes up is whether the mod_digest
can be used to record the digests ("checksums") of uploaded/downloaded files
in e.g. a SQL database. The answer is "yes", with some caveats.
First, here is a configuration excerpt showing show such functionality might
be implemented, using mod_digest and mod_sql:
<IfModule mod_digest.c>
</IfModule>
<IfModule mod_sql.c>
...
SQLNamedQuery log-file-checksum FREEFORM "INSERT INTO file_checksums (user, file, algo, checksum) VALUES ('%u', '%f', '%{note:mod_digest.algo}', '%{note:mod_digest.digest}')"
SQLLog RETR,STOR log-file-checksum
...
</IfModule>
As you can see, this makes use of the %{note:...} syntax of
the SQLLog directive; the same syntax also works for
LogFormat definitions as well. The mod_digest module
uses the following notes:
- mod_digest.algo
Name of the digest algorithm used, e.g. "SHA1".
- mod_digest.digest
Calculated digest of the file as a hex-encoded lowercase string.
Now, the caveats with this technique:
- Does not work if the
NoTransferCache DigestOption is used.
- Only works for binary, not ASCII, FTP uploads/downloads currently.
- Only works for uploads (
STOR) and downloads (RETR), but not for appends (APPE) or resumed uploads/downloads (REST + RETR/STOR).
- Does not work for FTP downloads if
UseSendfile is in effect.
In addition, the order in which the mod_digest and
mod_sql appear in your build command is important;
mod_digest must come after mod_sql,
otherwise the note values will not be populated properly in the
SQLLog statement. Thus, if you are building static modules,
your --with-modules parameter would look something like:
$ ./configure --with-modules=mod_sql:mod_sql_mysql:mod_digest ...
Or, if you are using shared modules, then your LoadModule
directives must look like:
LoadModule mod_sql.c
LoadModule mod_sql_mysql.c
LoadModule mod_digest.c
© Copyright 2016 TJ Saunders
All Rights Reserved
|