refactor: CLI commands for database management (#2654)

See the associated PR for more detailed commentary on specific changes.

### Commands refactored:
- User (**All:** add / list / update / del + _dovecot-master variants_)
- Quota (**All:** set / del)
- Virtual Alias (**All:** add / list /del)
- Relay (**All:** add-relayhost / add-sasl / exclude-domain)

### Overall changes involve:
- **Fairly common structure:**
  - `_main` method at the top provides an overview of logical steps:
    - After all methods are declared beneath it (_and imported from the new `helpers/database/db.sh`_), the `_main` is called at the bottom of the file.
    - `delmailuser` additionally processes option support for `-y` prior to calling `_main`.
  - `__usage` is now consistent with each of these commands, along with the `help` command.
  - Most logic delegated to new helper scripts. Some duplicate content remains on the basis that it's low-risk to maintenance and avoids less hassle to jump between files to check a single line, usually this is arg validation.
  - Error handling should be more consistent, along with var names (_no more `USER`/`EMAIL`/`FULL_EMAIL` to refer to the same expected value_).
- **Three new management scripts** (in `helpers/database/manage/`) using a common structure for managing changes to their respective "Database" config file.
  - `postfix-accounts.sh` unified not only add and update commands, but also all the dovecot-master versions, a single password call for all 4 of them, with a 5th consumer of the password prompt from the relay command `addsaslpassword`.
  - These scripts delegate actual writes to `helpers/database/db.sh` which provides a common API to support the changes made.
     - This is more verbose/complex vs the current inline operations each command currently has, as it provides generic support instead of slightly different variations being maintained, along with handling some edge cases that existed and would lead to bugs (notably substring matches).
     - Centralizing changes here seems wiser than scattered about. I've tried to make it easy to grok, hopefully it's not worse than the current situation.
     - List operations were kept in their respective commands, `db.sh` is only really managing writes. I didn't see a nice way for removing the code duplication for list commands as the duplication was fairly minimal, especially for `listalias` and `listdovecotmasteruser` which were quite simple in their differences in the loop body.
     - `listmailuser` and `delmailuser` also retain methods exclusive to respective commands, I wasn't sure if there was any benefit to move those, but they were refactored.

target/scripts/helpers/database/db.sh

@@ -0,0 +1,213 @@
+#! /bin/bash
+
+# Matches relative path to this scripts parent directory,
+# Must be defined above any function that would source relative to it:
+# shellcheck source-path=target/scripts/helpers/database
+
+DMS_CONFIG='/tmp/docker-mailserver'
+# Modifications are supported for the following databases:
+#
+# Accounts and Aliases (The 'virtual' kind):
+DATABASE_ACCOUNTS="${DMS_CONFIG}/postfix-accounts.cf"
+DATABASE_DOVECOT_MASTERS="${DMS_CONFIG}/dovecot-masters.cf"
+DATABASE_VIRTUAL="${DMS_CONFIG}/postfix-virtual.cf"
+# Dovecot Quota support:
+DATABASE_QUOTA="${DMS_CONFIG}/dovecot-quotas.cf"
+# Relay-Host support:
+DATABASE_PASSWD="${DMS_CONFIG}/postfix-sasl-password.cf"
+DATABASE_RELAY="${DMS_CONFIG}/postfix-relaymap.cf"
+
+# Individual scripts with convenience methods to manage operations easier:
+function _db_import_scripts
+{
+  # This var is stripped by shellcheck from source paths below,
+  # like the shellcheck source-path above, it shouold match this scripts
+  # parent directory, with the rest of the relative path in the source lines:
+  local PATH_TO_SCRIPTS='/usr/local/bin/helpers/database'
+
+  source "${PATH_TO_SCRIPTS}/manage/dovecot-quotas.sh"
+  source "${PATH_TO_SCRIPTS}/manage/postfix-accounts.sh"
+  source "${PATH_TO_SCRIPTS}/manage/postfix-virtual.sh"
+}
+_db_import_scripts
+
+function _db_entry_add_or_append  { _db_operation 'append'  "${@}" ; } # Only used by addalias
+function _db_entry_add_or_replace { _db_operation 'replace' "${@}" ; }
+function _db_entry_remove         { _db_operation 'remove'  "${@}" ; }
+
+function _db_operation
+{
+  local DB_ACTION=${1}
+  local DATABASE=${2}
+  local KEY=${3}
+  # Optional arg:
+  local VALUE=${4}
+
+  # K_DELIMITER provides a match boundary to avoid accidentally matching substrings:
+  local K_DELIMITER KEY_LOOKUP
+  K_DELIMITER=$(__db_get_delimiter_for "${DATABASE}")
+  # Due to usage in regex pattern, KEY needs to be escaped:
+  KEY_LOOKUP="$(_escape "${KEY}")${K_DELIMITER}"
+
+  # Support for adding or replacing an entire entry (line):
+  # White-space delimiter should be written into DATABASE as 'space' character:
+  local V_DELIMITER="${K_DELIMITER}"
+  [[ ${V_DELIMITER} == '\s' ]] && V_DELIMITER=' '
+  local ENTRY="${KEY}${V_DELIMITER}${VALUE}"
+
+  # Support for 'append' + 'remove' operations on value lists:
+  # NOTE: Presently only required for `postfix-virtual.cf`.
+  local _VALUE_
+  _VALUE_=$(_escape "${VALUE}")
+  # `postfix-virtual.cf` is using `,` for delimiting a list of recipients:
+  [[ ${DATABASE} == "${DATABASE_VIRTUAL}" ]] && V_DELIMITER=','
+
+  # Perform requested operation:
+  if _db_has_entry_with_key "${KEY}" "${DATABASE}"
+  then
+    # Find entry for key and return status code:
+    case "${DB_ACTION}" in
+      ( 'append' )
+        __db_list_already_contains_value && return 1
+
+        sedfile --strict -i "/^${KEY_LOOKUP}/s/$/${V_DELIMITER}${VALUE}/" "${DATABASE}"
+        ;;
+
+      ( 'replace' )
+        ENTRY=$(__escape_sed_replacement "${ENTRY}")
+        sedfile --strict -i "s/^${KEY_LOOKUP}.*/${ENTRY}/" "${DATABASE}"
+        ;;
+
+      ( 'remove' )
+        if [[ -z ${VALUE} ]]
+        then # Remove entry for KEY:
+          sedfile --strict -i "/^${KEY_LOOKUP}/d" "${DATABASE}"
+        else # Remove target VALUE from entry:
+          __db_list_already_contains_value || return 0
+
+          # The delimiter between key and first value may differ from
+          # the delimiter between multiple values (value list):
+          local LEFT_DELIMITER="\(${K_DELIMITER}\|${V_DELIMITER}\)"
+          # If an entry for KEY contains an exact match for VALUE:
+          # - If VALUE is the only value => Remove entry (line)
+          # - If VALUE is the last value => Remove VALUE
+          # - Otherwise => Collapse value to LEFT_DELIMITER (\1)
+          sedfile --strict -i \
+            -e "/^${KEY_LOOKUP}\+${_VALUE_}$/d" \
+            -e "/^${KEY_LOOKUP}/s/${V_DELIMITER}${_VALUE_}$//g" \
+            -e "/^${KEY_LOOKUP}/s/${LEFT_DELIMITER}${_VALUE_}${V_DELIMITER}/\1/g" \
+            "${DATABASE}"
+          fi
+        ;;
+
+      ( * ) # Should only fail for developer using this API:
+        _exit_with_error "Unsupported DB operation: '${DB_ACTION}'"
+        ;;
+
+    esac
+  else
+    # Entry for key does not exist, DATABASE may be empty, or DATABASE does not exist
+    case "${DB_ACTION}" in
+      # Fallback action 'Add new entry':
+      ( 'append' | 'replace' )
+        [[ ! -d ${DMS_CONFIG} ]] && mkdir -p "${DMS_CONFIG}"
+        echo "${ENTRY}" >>"${DATABASE}"
+        ;;
+
+      # Nothing to remove, return success status
+      ( 'remove' )
+        return 0
+        ;;
+
+      ( * ) # This should not happen if using convenience wrapper methods:
+        _exit_with_error "Unsupported DB operation: '${DB_ACTION}'"
+        ;;
+
+    esac
+  fi
+}
+
+# Internal method for: _db_operation
+function __db_list_already_contains_value
+{
+  # Avoids accidentally matching a substring (case-insensitive acceptable):
+  # 1. Extract the current value of the entry (`\1`),
+  # 2. If a value list, split into separate lines (`\n`+`g`) at V_DELIMITER,
+  # 3. Check each line for an exact match of the target VALUE
+  sed -e "s/^${KEY_LOOKUP}\(.*\)/\1/" \
+      -e "s/${V_DELIMITER}/\n/g"      \
+      "${DATABASE}" | grep -qi "^${_VALUE_}$"
+}
+
+
+# Internal method for: _db_operation + _db_has_entry_with_key
+# References global vars `DATABASE_*`:
+function __db_get_delimiter_for
+{
+  local DATABASE=${1}
+
+  case "${DATABASE}" in
+    ( "${DATABASE_ACCOUNTS}" | "${DATABASE_DOVECOT_MASTERS}" )
+      echo "|"
+      ;;
+
+    # NOTE: These files support white-space delimiters, we have not
+    # historically enforced a specific value; as a workaround
+    # `_db_operation` will convert to ` ` (space) for writing.
+    ( "${DATABASE_PASSWD}" | "${DATABASE_RELAY}" | "${DATABASE_VIRTUAL}" )
+      echo "\s"
+      ;;
+
+    ( "${DATABASE_QUOTA}" )
+      echo ":"
+      ;;
+
+    ( * )
+      _exit_with_error "Unsupported DB '${DATABASE}'"
+      ;;
+
+    esac
+}
+
+# sed replacement feature needs to be careful of content containing `/` and `&`,
+# `\` can escape these (`/` exists in postfix-account.cf base64 encoded pw hash),
+# But otherwise care should be taken with `\`, which should be forbidden for input here?
+# NOTE: Presently only `.` is escaped with `\` via `_escape`.
+function __escape_sed_replacement
+{
+  # Matches any `/` or `&`, and escapes them with `\` (`\\\1`):
+  sed 's/\([/&]\)/\\\1/g' <<< "${ENTRY}"
+}
+
+#
+# Validation Methods
+#
+
+function _db_has_entry_with_key
+{
+  local KEY=${1}
+  local DATABASE=${2}
+
+  # Fail early if the database file exists but has no content:
+  [[ -s ${DATABASE} ]] || return 1
+
+  # K_DELIMITER provides a match boundary to avoid accidentally matching substrings:
+  local K_DELIMITER KEY_LOOKUP
+  K_DELIMITER=$(__db_get_delimiter_for "${DATABASE}")
+  # Due to usage in regex pattern, KEY needs to be escaped:
+  KEY_LOOKUP="$(_escape "${KEY}")${K_DELIMITER}"
+
+  # NOTE:
+  # --quiet --no-messages, only return a status code of success/failure.
+  # --ignore-case as we don't want duplicate keys that vary by case.
+  # --extended-regexp not used, most regex escaping should be forbidden.
+  grep --quiet --no-messages --ignore-case "^${KEY_LOOKUP}" "${DATABASE}"
+}
+
+function _db_should_exist_with_content
+{
+  local DATABASE=${1}
+
+  [[ -f ${DATABASE} ]] || _exit_with_error "'${DATABASE}' does not exist"
+  [[ -s ${DATABASE} ]] || _exit_with_error "'${DATABASE}' is empty, nothing to list"
+}

target/scripts/helpers/database/manage/dovecot-quotas.sh

@@ -0,0 +1,32 @@
+#! /bin/bash
+
+# Manage DB writes for: DATABASE_QUOTA
+
+# Logic to perform for requested operations handled here:
+function _manage_dovecot_quota
+{
+  local ACTION=${1}
+  local MAIL_ACCOUNT=${2}
+  # Only for ACTION 'update':
+  local QUOTA=${3}
+
+  local DATABASE_QUOTA='/tmp/docker-mailserver/dovecot-quotas.cf'
+  case "${ACTION}" in
+    ( 'update' )
+      _db_entry_add_or_replace "${DATABASE_QUOTA}" "${MAIL_ACCOUNT}" "${QUOTA}"
+      ;;
+
+    ( 'delete' )
+      _db_entry_remove "${DATABASE_QUOTA}" "${MAIL_ACCOUNT}"
+      ;;
+
+    ( * ) # This should not happen if using convenience wrapper methods:
+      _exit_with_error "Unsupported Action: '${ACTION}'"
+      ;;
+
+  esac
+}
+
+# Convenience wrappers:
+function _manage_dovecot_quota_update { _manage_dovecot_quota 'update' "${@}" ; } # setquota
+function _manage_dovecot_quota_delete { _manage_dovecot_quota 'delete' "${@}" ; } # delquota, delmailuser

target/scripts/helpers/database/manage/postfix-accounts.sh

@@ -0,0 +1,100 @@
+#! /bin/bash
+
+# Manage DB writes for:
+# - DATABASE_ACCOUNTS
+# - DATABASE_DOVECOT_MASTERS
+
+# Logic to perform for requested operations handled here:
+function _manage_accounts
+{
+  local ACTION=${1}
+  local DATABASE=${2}
+  local MAIL_ACCOUNT=${3}
+  # Only for ACTION 'create' or 'update':
+  local PASSWD=${4}
+
+  _arg_expect_mail_account
+
+  case "${ACTION}" in
+    ( 'create' | 'update' )
+      # Fail early before requesting password:
+      [[ ${ACTION} == 'create' ]] && _account_should_not_exist_yet
+      [[ ${ACTION} == 'update' ]] && _account_should_already_exist
+      _password_request_if_missing
+
+      local PASSWD_HASH
+      PASSWD_HASH=$(doveadm pw -s SHA512-CRYPT -u "${MAIL_ACCOUNT}" -p "${PASSWD}")
+      # Early failure above ensures correct operation => Add (create) or Replace (update):
+      _db_entry_add_or_replace "${DATABASE}" "${MAIL_ACCOUNT}" "${PASSWD_HASH}"
+      ;;
+
+    ( 'delete' )
+      _db_entry_remove "${DATABASE}" "${MAIL_ACCOUNT}"
+      ;;
+
+    ( * ) # This should not happen if using convenience wrapper methods:
+      _exit_with_error "Unsupported Action: '${ACTION}'"
+      ;;
+
+  esac
+}
+
+# Convenience wrappers:
+DATABASE_ACCOUNTS='/tmp/docker-mailserver/postfix-accounts.cf'
+function _manage_accounts_create { _manage_accounts 'create' "${DATABASE_ACCOUNTS}" "${@}" ; }
+function _manage_accounts_update { _manage_accounts 'update' "${DATABASE_ACCOUNTS}" "${@}" ; }
+function _manage_accounts_delete { _manage_accounts 'delete' "${DATABASE_ACCOUNTS}" "${@}" ; }
+
+# Dovecot Master account support can leverage the same management logic:
+DATABASE_DOVECOT_MASTERS='/tmp/docker-mailserver/dovecot-masters.cf'
+function _manage_accounts_dovecotmaster_create { _manage_accounts 'create' "${DATABASE_DOVECOT_MASTERS}" "${@}" ; }
+function _manage_accounts_dovecotmaster_update { _manage_accounts 'update' "${DATABASE_DOVECOT_MASTERS}" "${@}" ; }
+function _manage_accounts_dovecotmaster_delete { _manage_accounts 'delete' "${DATABASE_DOVECOT_MASTERS}" "${@}" ; }
+
+#
+# Validation Methods
+#
+
+# These validation helpers rely on:
+# - Exteral vars to be declared prior to calling them (MAIL_ACCOUNT, PASSWD, DATABASE).
+# - Calling external method '__usage' as part of error handling.
+
+# Also used by setquota, delquota
+function _arg_expect_mail_account
+{
+  [[ -z ${MAIL_ACCOUNT} ]] && { __usage ; _exit_with_error 'No account specified' ; }
+
+  # Dovecot Master accounts are validated (they are not email addresses):
+  [[ ${DATABASE} == "${DATABASE_DOVECOT_MASTERS}" ]] && return 0
+
+  # Account has both local and domain parts:
+  [[ ${MAIL_ACCOUNT} =~ .*\@.* ]] || { __usage ; _exit_with_error "'${MAIL_ACCOUNT}' should include the domain (eg: user@example.com)" ; }
+}
+
+function _account_should_not_exist_yet
+{
+  __account_already_exists && _exit_with_error "'${MAIL_ACCOUNT}' already exists"
+}
+
+# Also used by delmailuser, setquota, delquota
+function _account_should_already_exist
+{
+  ! __account_already_exists && _exit_with_error "'${MAIL_ACCOUNT}' does not exist"
+}
+
+function __account_already_exists
+{
+  local DATABASE=${DATABASE:-"${DATABASE_ACCOUNTS}"}
+  _db_has_entry_with_key "${MAIL_ACCOUNT}" "${DATABASE}"
+}
+
+# Also used by addsaslpassword
+function _password_request_if_missing
+{
+  if [[ -z ${PASSWD} ]]
+  then
+    read -r -s -p 'Enter Password: ' PASSWD
+    echo
+    [[ -z ${PASSWD} ]] && _exit_with_error 'Password must not be empty'
+  fi
+}

target/scripts/helpers/database/manage/postfix-virtual.sh

@@ -0,0 +1,47 @@
+#! /bin/bash
+
+# Manage DB writes for: DATABASE_VIRTUAL
+
+# A virtual alias may be any of `user@domain`, `user`, `@domain`.
+# Recipients are local (internal services), hosted (managed accounts), remote (third-party MTA), or aliases themselves,
+# An alias may redirect mail to one or more recipients. If a recipient is an alias Postfix will recursively resolve it.
+#
+# WARNING: Support for multiple and recursive recipients may not be well supported by this projects scripts/features.
+# One of those features is Dovecot Quota support, which uses a naive workaround for supporting quota checks for inbound
+# mail to an alias address.
+
+# Logic to perform for requested operations handled here:
+function _manage_virtual_aliases
+{
+  local ACTION=${1}
+  local MAIL_ALIAS=${2}
+  local RECIPIENT=${3}
+
+  # Validation error handling expects that the caller has defined a '__usage' method:
+  [[ -z ${MAIL_ALIAS} ]] && { __usage ; _exit_with_error 'No alias specified'     ; }
+  [[ -z ${RECIPIENT}  ]] && { __usage ; _exit_with_error 'No recipient specified' ; }
+
+  local DATABASE_VIRTUAL='/tmp/docker-mailserver/postfix-virtual.cf'
+  case "${ACTION}" in
+    # Associate RECIPIENT to MAIL_ALIAS:
+    ( 'update' )
+      _db_entry_add_or_append "${DATABASE_VIRTUAL}" "${MAIL_ALIAS}" "${RECIPIENT}"
+      ;;
+
+    # Removes RECIPIENT from MAIL_ALIAS - or all aliases when MAIL_ALIAS='_':
+    # NOTE: If a matched alias has no additional recipients, it is also removed.
+    ( 'delete' )
+      [[ ${MAIL_ALIAS} == '_' ]] && MAIL_ALIAS='\S\+'
+      _db_entry_remove "${DATABASE_VIRTUAL}" "${MAIL_ALIAS}" "${RECIPIENT}"
+      ;;
+
+    ( * ) # This should not happen if using convenience wrapper methods:
+      _exit_with_error "Unsupported Action: '${ACTION}'"
+      ;;
+
+  esac
+}
+
+# Convenience wrappers:
+function _manage_virtual_aliases_update { _manage_virtual_aliases 'update' "${@}" ; } # addalias
+function _manage_virtual_aliases_delete { _manage_virtual_aliases 'delete' "${@}" ; } # delalias, delmailuser