fix: Remove mkcert.sh usage + _setup_ssl refactor. (#2196)

* chore(refactor): DRY up the `_setup_ssl` method

- `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`.
- The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead.
- This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result.

- Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths.

- Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor.

- `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key.
- That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files.

- I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion).

---

* chore: Remove mkcert logic and dovecot cert

- No longer serving a purpose.
- Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`.
- To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image.
- Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition.

---

* Error handling for SSL_TYPE

- Added a panic utility to exit early when SSL_TYPE conditions are misconfigured.
- Some info text had order of key/cert occurrence swapped to be consistent with key then cert.
- Some existing comments moved and rephrased.
- Additional comments added.
- `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks).
- _notify messages lifted out of conditionals so that they always output when the case is hit.
- ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~

---

* Use sedfile + improve sed expressions + update case style

- Uses sedfile when appropriate (file change intentional, not optional match/check).
- sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)).
- sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement.

- case statements adopt the `(` prefix, adopting recent shell style for consistency.

---

* Refactor SSL_TYPE=disabled

- Postfix is also disabled now.
- Included heavy inline documentation reference for maintainers.
- Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'.

---

* I'm a friggin' sed wizard now

- The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`.

- I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines).

- Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better.

---

* tests(fix): Tests that require SSL/TLS to pass

- SSL_TYPE=snakeoil added as temporary workaround.

- nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests.
- Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past.
- The removed SSL test is already handled in mail_ssl_manual.bats

ldap test:
- Replace `--link` alias option with `--network` and alias assignment.
- Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure.

privacy test:
- Also needed `SSL_TYPE` to pass the starttls test.

`tests.bats` had another starttls test for imap:
- Workaround for now is to give the main test container `SSL_TYPE=snakeoil`.

---

* Remove the expired lets-encrypt cert

This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores.

No longer the case today.

---

* chore: Housekeeping

Not required for this PR branch, little bit of tidying up while working on these two test files.

- privacy test copied over content when extracted from `tests.bats` that isn't relevant.
- ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find.
- Additionally converted the two localhost FQDN to vars.

---

* Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case)

- This is to prevent other tests from failing by hitting the panic catchall case.
- More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value.

---

* Add inline documentation for `dms_panic`

- This could later be better formatted and placed into contributor docs.

Panic with kill (shutdown) not exit (errex):
- `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process).
- `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't.

The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
This commit is contained in:
Brennan Kinney 2021-09-20 00:31:11 +12:00 committed by GitHub
parent 2bf24e4c08
commit c851f5b6aa
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
12 changed files with 452 additions and 427 deletions

View file

@ -14,27 +14,30 @@ setup_file() {
PRIVATE_CONFIG="$(duplicate_config_for_container . mail)"
mv "${PRIVATE_CONFIG}/user-patches/user-patches.sh" "${PRIVATE_CONFIG}/user-patches.sh"
docker run --rm -d --name mail \
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-v "$(pwd)/test/onedir":/var/mail-state \
-e ENABLE_CLAMAV=1 \
-e SPOOF_PROTECTION=1 \
-e ENABLE_SPAMASSASSIN=1 \
-e REPORT_RECIPIENT=user1@localhost.localdomain \
-e REPORT_SENDER=report1@mail.my-domain.com \
-e SA_TAG=-5.0 \
-e SA_TAG2=2.0 \
-e SA_KILL=3.0 \
-e AMAVIS_LOGLEVEL=2 \
-e SA_SPAM_SUBJECT="SPAM: " \
-e VIRUSMAILS_DELETE_DELAY=7 \
-e ENABLE_SRS=1 \
-e SASL_PASSWD="external-domain.com username:password" \
-e ENABLE_MANAGESIEVE=1 \
--cap-add=SYS_PTRACE \
-e PERMIT_DOCKER=host \
-e DMS_DEBUG=0 \
-h mail.my-domain.com -t "${NAME}"
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-v "$(pwd)/test/onedir":/var/mail-state \
-e ENABLE_CLAMAV=1 \
-e SPOOF_PROTECTION=1 \
-e ENABLE_SPAMASSASSIN=1 \
-e REPORT_RECIPIENT=user1@localhost.localdomain \
-e REPORT_SENDER=report1@mail.my-domain.com \
-e SA_TAG=-5.0 \
-e SA_TAG2=2.0 \
-e SA_KILL=3.0 \
-e AMAVIS_LOGLEVEL=2 \
-e SA_SPAM_SUBJECT="SPAM: " \
-e VIRUSMAILS_DELETE_DELAY=7 \
-e ENABLE_SRS=1 \
-e SASL_PASSWD="external-domain.com username:password" \
-e ENABLE_MANAGESIEVE=1 \
-e PERMIT_DOCKER=host \
-e DMS_DEBUG=0 \
-e SSL_TYPE='snakeoil' \
-h mail.my-domain.com \
--cap-add=SYS_PTRACE \
--tty \
"${NAME}"
wait_for_finished_setup_in_container mail
@ -43,34 +46,34 @@ setup_file() {
docker exec mail addmailuser pass@localhost.localdomain 'may be \a `p^a.*ssword'
# setup sieve
docker cp "${PRIVATE_CONFIG}/sieve/dovecot.sieve" mail:/var/mail/localhost.localdomain/user1/.dovecot.sieve
docker cp "${PRIVATE_CONFIG}/sieve/dovecot.sieve" mail:/var/mail/localhost.localdomain/user1/.dovecot.sieve
# this relies on the checksum file beeing updated after all changes have been applied
wait_for_changes_to_be_detected_in_container mail
wait_for_smtp_port_in_container mail
wait_for_smtp_port_in_container mail
# wait for clamav to be fully setup or we will get errors on the log
repeat_in_container_until_success_or_timeout 60 mail test -e /var/run/clamav/clamd.ctl
# sending test mails
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-spam.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-virus.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-recipient-delimiter.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user1.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user2.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user3.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-added.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user-and-cc-local-alias.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-catchall-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-spam-folder.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-pipe.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/non-existing-user.txt"
docker exec mail /bin/sh -c "sendmail root < /tmp/docker-mailserver-test/email-templates/root-email.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-spam.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-virus.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-recipient-delimiter.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user1.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user2.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user3.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-added.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user-and-cc-local-alias.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-catchall-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-spam-folder.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-pipe.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/non-existing-user.txt"
docker exec mail /bin/sh -c "sendmail root < /tmp/docker-mailserver-test/email-templates/root-email.txt"
wait_for_empty_mail_queue_in_container mail
}
@ -432,20 +435,6 @@ EOF
assert_success
}
#
# ssl
#
@test "checking ssl: generated default cert works correctly" {
run docker exec mail /bin/sh -c "timeout 1 openssl s_client -connect 0.0.0.0:587 -starttls smtp -CApath /etc/ssl/certs/ | grep 'Verify return code: 0 (ok)'"
assert_success
}
@test "checking ssl: lets-encrypt-x3-cross-signed.pem is installed" {
run docker exec mail grep 'BEGIN CERTIFICATE' /etc/ssl/certs/lets-encrypt-x3-cross-signed.pem
assert_success
}
#
# postsrsd
#
@ -701,86 +690,86 @@ EOF
@test "checking quota: setquota user must be existing" {
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user 50M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 50M"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user 50M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 50M"
assert_success
run docker exec mail /bin/sh -c "setquota username@fulldomain 50M"
assert_failure
run docker exec mail /bin/sh -c "setquota username@fulldomain 50M"
assert_failure
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: setquota <quota> must be well formatted" {
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 26GIGOTS"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 123"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld -60M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 26GIGOTS"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 123"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld -60M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10B"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10k"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10M"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10G"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10B"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10k"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10M"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10G"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: delquota user must be existing" {
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "delquota uota_user@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "delquota quota_user"
assert_failure
run docker exec mail /bin/sh -c "delquota dontknowyou@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "delquota uota_user@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "delquota quota_user"
assert_failure
run docker exec mail /bin/sh -c "delquota dontknowyou@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: delquota allow when no quota for existing user" {
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "addmailuser quota_user@domain.tld mypassword"
assert_success
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: dovecot quota present in postconf" {
@ -1216,60 +1205,6 @@ EOF
assert_failure
}
#
# PCI compliance
#
# dovecot
@test "checking dovecot: only A grade TLS ciphers are used" {
run docker run --rm -i --link mail:dovecot \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 993 dovecot | grep "least strength: A"'
assert_success
}
@test "checking dovecot: nmap produces no warnings on TLS ciphers verifying" {
run docker run --rm -i --link mail:dovecot \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 993 dovecot | grep "warnings" | wc -l'
assert_success
assert_output 0
}
# postfix submission TLS
@test "checking postfix submission: only A grade TLS ciphers are used" {
run docker run --rm -i --link mail:postfix \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 587 postfix | grep "least strength: A"'
assert_success
}
@test "checking postfix submission: nmap produces no warnings on TLS ciphers verifying" {
run docker run --rm -i --link mail:postfix \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 587 postfix | grep "warnings" | wc -l'
assert_success
assert_output 0
}
# postfix smtps SSL
@test "checking postfix smtps: only A grade TLS ciphers are used" {
run docker run --rm -i --link mail:postfix \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 465 postfix | grep "least strength: A"'
assert_success
}
@test "checking postfix smtps: nmap produces no warnings on TLS ciphers verifying" {
run docker run --rm -i --link mail:postfix \
--entrypoint sh instrumentisto/nmap -c \
'nmap --script ssl-enum-ciphers -p 465 postfix | grep "warnings" | wc -l'
assert_success
assert_output 0
}
#
# supervisor
#