ZAGGREGATE(1)

zaggregate reads relevant message data from the database, using the queries configured as described in zfilter_db(1), and formats the result according to the XML schema defined in ``RFC 7489, Appendix C''.

To comply with DMARC specifications, this command should be run by cron once per day or according to the configured honored_report_interval.

DMARC provides for sending aggregate reports to multiple URIs, subject to size limits. Only "mailto:" URIs are considered here. The size limits refer to the compressed and encoded content of reports, so they can be enacted only after compression and base64 encoding (option -z). For each target address, zaggregate verifies that either its domain part is aligned with the domain reported or the target domain authorizes sending reports about the domain reported.

zaggregate design goal is to ease reporting while letting users devise how to set up actual sending. In addition, it provides for debugging its three queries, which zfilter_db(1) does not.

Unless --pipe is used, reports are written to files in the current directory. Two files per report, with the same base name and different extensions. The base name is composed according to DMARC specification, that is:

: receiver ``!'' domain ``!'' period_start ``!'' period_end [ ``!'' unique-id ]

where "receiver" is the domain part of the ORG_EMAIL environment variable, "domain" is the one where the DMARC record was retrieved from, period boundaries are expressed as seconds since Unix epoch, UTC, and "unique-id" depends on option -u. The first file has an ".env" extension, and contains the environment variables described below. The second file contains the report; its extension is either ".xml" or ".xml.gz.b64" depending on option -z.

OPTIONS

Short options can be stuffed into a single argument.

-%

Double each percent character ("%") that may appear in the environment variables TO_HEADER and URLENCODED_RCPT. That is to be used if those variables are included in a printf template.

Note that URIs in DMARC records are urlencoded. Still, the resulting addresses can contain percent characters in the local part. URLENCODED_RCPT, if set, contains at least %40.

-1

Set fork for each recipient, if --pipe is also given. That may entail more forks if a report unlikely has multiple recipients. In addition, this flag causes the environment variable URLENCODED_RCPT to be set. The latter can be used to set unique bounce addresses.

-f config-filename

Specify a different configuration file.

-l

Log to syslog (LOG_MAIL facility) rather than stderr.

-t

If --pipe is given, write the report to a temporary file before piping. This option is automatically enabled if there are multiple size limits.

-u

Generate a unique identifier for each report. If generated, the unique-id appears in the filename as well as in the body of the report. It should also match the "Message-Id:" header field.

This option is available only if zaggregate is compiled with the UUID library.

-v

Debug queries. If issued multiple times, it increases verbosity. This option is automatically enabled if zdkimfilter.conf has verbose >= 10.

-z

Zip and encode the report. This enables checking size limits. If the limit is exceeded the report is aborted. A line is logged saying no rcpt left. DMARC recommends sending a notice to the relevant domain so that they can increase the report frequency, the size limit, or both.

--pipe

This pipes the output to a child process. The arguments following this option are passed to the child process, starting with the executable path.

A child is forked for each mail, setting the environment variables described below. The report content is piped to child's standard input. If the size limit is exceeded, the child is killed with SIGTERM. However, if there are multiple limits or option -t is used, the report is first written to a temporary file, and, if all size limits are exceeded, no child is forked.

Children are spawned one at a time, and waited for. If a child exit status is not 0, the report is not considered sent, and the domain timestamp is not updated.

--help

Print usage and exit.

--version

Print version and exit.

--dry-run

Don't update domain timestamps after report is (considered) sent.

--fake-dns

For debugging, don't issue DNS queries but look up TXT records in KEYFILE. The latter is a plain text file in the current directory, having a name, a space, and text RDATA in each relevant line.

--config

Print out the configuration file and some debug info, and exit.

--only-domain

Specify a domain name to have the report produced for such domain only. This is useful for debugging unexpected errors occurred for a given domain. This option implies --dry-run.

ENVIRONMENT VARIABLES

Input Environment Variables

These variables are read from the environment. They are also available to any child, of course.

ORG_EMAIL: The email address of the reporting organization. Its domain part is used for the first piece of the filename ("receiver" above).
ORG_NAME: Human readable name of the reporting organization.
EXTRA_CONTACT: If set, goes to the "extra_contact_info" part of the report.

Output Environment Variables

These variables are added to child environment if --pipe is used. Otherwise they are written, one per line, to a *.env file.

FILENAME: The report filename. When -z is used, the real filename gets an additional ".b64" extension, but the environment variable only has ".gz". Otherwise the extension is just ".xml".
CONTENT_TYPE: Either "application/gzip" or "text/xml" depending on -z.
CONTENT_TRANSFER_ENCODING: Either "base64" or "7bit" depending on -z.
DOMAIN: The target domain, which also makes the second piece of the filename.
LIMIT: The lowest limit specified by target addresses. If no address specifies a limit, the default is 18446744073709551615 (UINT64_MAX).
TO_HEADER: The "To:" header field, containing all addresses, or just one if -1 is used. If -z is used, only addresses that accept the report size are included. In any case, addresses which specify a non-default size limit are followed by a comment with the specified limit.
Use of option -% affects this variable.
URLENCODED_RCPT: This variable is empty unless -1 is used. In that case, it contains the urlencoded target address, in particular replacing the at ("@") sign with %40, to be used to compose bounces.
Use of option -% affects this variable.
MESSAGE_ID: This variable is only set if option -u is used. It contains the uniq-id.

head1 QUERIES

These queries must be configured for zaggregate to function. See zfilter_db(1) for details on the available variables.

db_sql_dmarc_agg_domain: DMARC aggregate domain query, to select domains.
db_sql_dmarc_agg_record: DMARC aggregate record query, given a domain, select records.
db_sql_check_dmarc_rcpt: Check recipient address, so as to skip bouncing targets.
db_sql_set_dmarc_agg: Adjust the reported perios after the report is considered sent.

FILES

/local/courier/etc/path/filters/zdkimfilter.conf: Default configuration file.
*.env and *.xml[.gz.b64]: Reports generated in the current directory.

EXAMPLES

A pipeline, where "head" and "tail" have the initial and final parts of the report, expressed in terms of environment variables and printf escape sequences, can be structured like so:

  ORG_EMAIL=foo@example.com zaggregate -zul --pipe /bin/sh -c \
    '(eval printf %b "\"$(<head)\""; cat -; printf %b "\"$(<tail)\"") |\
      sendmail'

Using --pipe instead of a real pipe is necessary because a new /bin/sh has to be spawned for each report. "eval" replaces shell variables in "head", and is not needed for "tail" if there are none. For a fully worked out example using that approach see zaggregate-example.sh in the tarball.

Working with files may allow greater control and flexibility.

To test database queries from the distribution directory:

  source ./odbx_example.sh
  source ./odbx_example.sh
  sleep 5m
  ORG_EMAIL=foo@example.com src/zaggregate -vf odbx_example.conf

ZAGGREGATE(1)

NAME

SYNOPSIS

DESCRIPTION