[BUG] Building Dockerfile locally fails

# Bug Report

When cloning the project and attempting to build a Docker image locally via `make build` it was failing during package install.

<details>
<summary>Earlier report (invalid)</summary>

**EDIT:** These findings below were mistaken. I was under the belief that the errors I found when disabling the `SHELL` directive was the actual build failures, but those were new errors due to running `bash` syntax with `sh`, thus misleading. **Actual cause** is related to `postfix` package post install failing. I will add a new comment with details.

## Context

Appears to be [introduced in early Jan with the migration](https://github.com/docker-mailserver/docker-mailserver/commit/189e5376cc91af4f8a6b1e0db7b8f20e53a932a1#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R38) by @aendeavor . Usage of `&>/dev/null` was the culprit.

Additionally, the actual error context was hidden due to:

https://github.com/docker-mailserver/docker-mailserver/blob/abdf681d0265fce41aa971dd6b8194c902a3c1f6/Dockerfile#L35

### What is affected by this bug?

Building the Dockerfile locally under latest versions of Docker.

The redirection behaviour is illustrated quite well [here](https://unix.stackexchange.com/a/119650), with the `&> /dev/null` syntax described [here](https://askubuntu.com/a/1145025) with a comment stating this is deprecated in favor of `> /dev/null 2>&1` which it is apparently equivalent to.

Interestingly, if that is the desired behaviour (_to hide stdout, and redirect stderr into stdout to be hidden as well_) that does work as intended with `>/dev/null 2>&1`, just not `&>/dev/null`.

</details>

### When does this occur?

Within the Dockerfile.

<details>
<summary>From original report, unrelated to the real bug, but may want to fix the inconsistency</summary>

There are:

- 16 occurrences of `/dev/null` at present, 15 will match `>/dev/null`, while the other is inconsistent with a space `> /dev/null`. 6 use `&>/dev/null`.
- 4 occurrences of `2>&1`, but only 1 paired with `>/dev/null` but with a swapped order of `2>&1 >/dev/null`(_this redirects stderr into stdout, but original stdout output is hidden as it's sent to `/dev/null` instead_). Unclear if that was the intention.

</details>

### How do we replicate the issue?

1. Ubuntu 21.04 (I used Vultr if it makes any difference)
2. [Install Docker-Engine](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository):

   ```bash
   apt-get update && apt-get install apt-transport-https
   curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
   echo \
     "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
     $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
   
   apt-get update && apt-get install docker-ce docker-ce-cli containerd.io
   ```
3. Clone `docker-mailserver` and build image:

   ```bash
   git clone https://github.com/docker-mailserver/docker-mailserver.git
   cd docker-mailserver
   git submodule update --init --recursive
   apt-get install make
   make build
   ```
4. Build fails.

## Behavior

<details>
<summary>Earlier report (invalid)</summary>

With the first occurrence being changed to `>/dev/null` or `>/dev/null 2>&1`, packages install properly and there is no error about being unable to find packages such as `ca-certificates` (_It is unclear why `&>/dev/null` affected the earlier commands, but presumably it prevented them from running properly for some reason_).

<details>
<summary>Output</summary>

With the F2B section, as-is I noticed that some output was in red (stderr?):

```
gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created                                                                                                                                            
```

and changing to `>/dev/null` retained those two lines, and additionally appended a few more (also in red):

```
gpg: /root/.gnupg/trustdb.gpg: trustdb created                                                                                                                                            
gpg: key 683BF1BEBD0A882C: public key "Serg G. Brester (sebres) <info@sebres.de>" imported                                                                                                
gpg: Total number processed: 1                                                                                                                                                            
gpg:               imported: 1  
```

The F2B `FINGERPRINT` script then uses `2>&1` 3 times, with some conditionals. The output as-is is:

```
/bin/sh: 1: [[: not found
/bin/sh: 1: [[: not found
```

When the `SHELL` directive is re-enabled, this is the output:
```
The command '/bin/bash -o pipefail -c dpkg -i fail2ban.deb &>/dev/null' returned a non-zero code: 1
```

---

Reduced Dockerfile used:

```Dockerfile
FROM docker.io/debian:buster-slim

ARG DEBIAN_FRONTEND=noninteractive

ARG FAIL2BAN_DEB_URL=https://github.com/fail2ban/fail2ban/releases/download/0.11.2/fail2ban_0.11.2-1.upstream1_all.deb
ARG FAIL2BAN_DEB_ASC_URL=${FAIL2BAN_DEB_URL}.asc
ARG FAIL2BAN_GPG_PUBLIC_KEY_ID=0x683BF1BEBD0A882C
ARG FAIL2BAN_GPG_PUBLIC_KEY_SERVER=keys.gnupg.net
ARG FAIL2BAN_GPG_FINGERPRINT="8738 559E 26F6 71DF 9E2C  6D9E 683B F1BE BD0A 882C"

#SHELL ["/bin/bash", "-o", "pipefail", "-c"]

RUN \
  apt-get -qq update && \
  apt-get -qq install apt-utils &>/dev/null && \
  apt-get -qq dist-upgrade >/dev/null
  
RUN apt-get -y --no-install-recommends install ca-certificates curl gnupg python3

RUN \
  # Fail2Ban
  gpg --keyserver ${FAIL2BAN_GPG_PUBLIC_KEY_SERVER} \
    --recv-keys ${FAIL2BAN_GPG_PUBLIC_KEY_ID} &>/dev/null && \
  curl -Lkso fail2ban.deb ${FAIL2BAN_DEB_URL} && \
  curl -Lkso fail2ban.deb.asc ${FAIL2BAN_DEB_ASC_URL}

RUN \
  FINGERPRINT="$(LANG=C gpg --verify \
  fail2ban.deb.asc fail2ban.deb 2>&1 \
    | sed -n 's#Primary key fingerprint: $.*$#\1#p')" && \
  if [[ -z ${FINGERPRINT} ]]; then \
    echo "ERROR: Invalid GPG signature!" 2>&1; exit 1; fi && \
  if [[ ${FINGERPRINT} != "${FAIL2BAN_GPG_FINGERPRINT}" ]]; then \
    echo "ERROR: Wrong GPG fingerprint!" 2>&1; exit 1; fi

RUN \
  dpkg -i fail2ban.deb &>/dev/null

RUN \
  rm fail2ban.deb fail2ban.deb.asc

RUN \
  # cleanup
  apt-get -qq autoremove &>/dev/null && \
  apt-get -qq autoclean && \
  apt-get -qq clean

RUN rm -rf /var/lib/apt/lists/*
RUN c_rehash &>/dev/null
```

```shell-session
$ docker build -t test-build -f Dockerfile .

...
Step 8/17 : RUN   apt-get -qq update &&   apt-get -y install apt-utils &>/dev/null &&   apt-get -y dist-upgrade >/dev/null
 ---> Running in f30ee26e1d1f
Removing intermediate container f30ee26e1d1f
 ---> 4fd0bfffc854
Step 9/17 : RUN apt-get -y --no-install-recommends install ca-certificates curl gnupg
 ---> Running in 9a8e0ee1c9da
Reading package lists...
Building dependency tree...
Reading state information...
Package ca-certificates is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package gnupg is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'ca-certificates' has no installation candidate
E: Unable to locate package curl
E: Package 'gnupg' has no installation candidate
E: Unable to locate package python3
The command '/bin/sh -c apt-get -y --no-install-recommends install ca-certificates curl gnupg python3' returned a non-zero code: 100
```

With `SHELL` directive enabled:

```
Step 13/18 : RUN   dpkg -i fail2ban.deb &>/dev/null
 ---> Running in eb24fdfc2111
The command '/bin/bash -o pipefail -c dpkg -i fail2ban.deb &>/dev/null' returned a non-zero code: 1
```

</details>

### Actual Behavior

With the actual Dockerfile, the error output is:

<details>
<summary>Output</summary>

```
Step 30/71 : SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 ---> Using cache
 ---> 7e8f3851c231
Step 31/71 : RUN   apt-get -qq update &&   apt-get -y install apt-utils &>/dev/null &&   apt-get -y dist-upgrade >/dev/null &&   apt-get -y install postfix >/dev/null &&   apt-get -y --no-install-recommends install   altermime amavisd-new apt-transport-https arj binutils bzip2 bsd-mailx   ca-certificates cabextract clamav clamav-daemon cpio curl   dovecot-core dovecot-imapd dovecot-ldap dovecot-lmtpd   dovecot-managesieved dovecot-pop3d dovecot-sieve dovecot-solr   dumb-init   ed fetchmail file gamin gnupg gzip iproute2 iptables   locales logwatch lhasa libdate-manip-perl liblz4-tool   libmail-spf-perl libnet-dns-perl libsasl2-modules lrzip lzop   netcat-openbsd nomarch opendkim opendkim-tools opendmarc   pax pflogsumm postgrey p7zip-full postfix-ldap postfix-pcre   postfix-policyd-spf-python postsrsd pyzor   razor rpm2cpio rsyslog sasl2-bin spamassassin supervisor   unrar-free unzip whois xz-utils &&   gpg --keyserver ${FAIL2BAN_GPG_PUBLIC_KEY_SERVER}     --recv-keys ${FAIL2BAN_GPG_PUBLIC_KEY_ID} &>/dev/null &&   curl -Lkso fail2ban.deb ${FAIL2BAN_DEB_URL} &&   curl -Lkso fail2ban.deb.asc ${FAIL2BAN_DEB_ASC_URL} &&   FINGERPRINT="$(LANG=C gpg --verify   fail2ban.deb.asc fail2ban.deb 2>&1     | sed -n 's#Primary key fingerprint: $.*$#\1#p')" &&   if [[ -z ${FINGERPRINT} ]]; then     echo "ERROR: Invalid GPG signature!" 2>&1; exit 1; fi &&   if [[ ${FINGERPRINT} != "${FAIL2BAN_GPG_FINGERPRINT}" ]]; then     echo "ERROR: Wrong GPG fingerprint!" 2>&1; exit 1; fi &&   dpkg -i fail2ban.deb &>/dev/null &&   rm fail2ban.deb fail2ban.deb.asc &&   apt-get -qq autoremove &>/dev/null &&   apt-get -qq autoclean &&   apt-get -qq clean &&   rm -rf /var/lib/apt/lists/* &&   c_rehash &>/dev/null
 ---> Running in 5479bdfbd569

E: Sub-process /usr/bin/dpkg returned an error code (1)

The command '/bin/bash -o pipefail -c apt-get -qq update &&   apt-get -y install apt-utils &>/dev/null &&   apt-get -y dist-upgrade >/dev/null &&   apt-get -y install postfix >/dev/null &&   apt-get -y --no-install-recommends install   altermime amavisd-new apt-transport-https arj binutils bzip2 bsd-mailx   ca-certificates cabextract clamav clamav-daemon cpio curl   dovecot-core dovecot-imapd dovecot-ldap dovecot-lmtpd   dovecot-managesieved dovecot-pop3d dovecot-sieve dovecot-solr   dumb-init   ed fetchmail file gamin gnupg gzip iproute2 iptables   locales logwatch lhasa libdate-manip-perl liblz4-tool   libmail-spf-perl libnet-dns-perl libsasl2-modules lrzip lzop   netcat-openbsd nomarch opendkim opendkim-tools opendmarc   pax pflogsumm postgrey p7zip-full postfix-ldap postfix-pcre   postfix-policyd-spf-python postsrsd pyzor   razor rpm2cpio rsyslog sasl2-bin spamassassin supervisor   unrar-free unzip whois xz-utils &&   gpg --keyserver ${FAIL2BAN_GPG_PUBLIC_KEY_SERVER}     --recv-keys ${FAIL2BAN_GPG_PUBLIC_KEY_ID} &>/dev/null &&   curl -Lkso fail2ban.deb ${FAIL2BAN_DEB_URL} &&   curl -Lkso fail2ban.deb.asc ${FAIL2BAN_DEB_ASC_URL} &&   FINGERPRINT="$(LANG=C gpg --verify   fail2ban.deb.asc fail2ban.deb 2>&1     | sed -n 's#Primary key fingerprint: $.*$#\1#p')" &&   if [[ -z ${FINGERPRINT} ]]; then     echo "ERROR: Invalid GPG signature!" 2>&1; exit 1; fi &&   if [[ ${FINGERPRINT} != "${FAIL2BAN_GPG_FINGERPRINT}" ]]; then     echo "ERROR: Wrong GPG fingerprint!" 2>&1; exit 1; fi &&   dpkg -i fail2ban.deb &>/dev/null &&   rm fail2ban.deb fail2ban.deb.asc &&   apt-get -qq autoremove &>/dev/null &&   apt-get -qq autoclean &&   apt-get -qq clean &&   rm -rf /var/lib/apt/lists/* &&   c_rehash &>/dev/null' returned a non-zero code: 100
make: *** [Makefile:20: build] Error 100
```

When disabling the `SHELL` directive on the actual Dockerfile, running `make build` results in:

```
Step 30/70 : RUN   apt-get -qq update &&   apt-get -y install apt-utils &>/dev/null &&   apt-get -y dist-upgrade >/dev/null &&   apt-get -y install postfix >/dev/null &&   apt-get -y --no-install-recommends install   altermime amavisd-new apt-transport-https arj binutils bzip2 bsd-mailx   ca-certificates cabextract clamav clamav-daemon cpio curl   dovecot-core dovecot-imapd dovecot-ldap dovecot-lmtpd   dovecot-managesieved dovecot-pop3d dovecot-sieve dovecot-solr   dumb-init   ed fetchmail file gamin gnupg gzip iproute2 iptables   locales logwatch lhasa libdate-manip-perl liblz4-tool   libmail-spf-perl libnet-dns-perl libsasl2-modules lrzip lzop   netcat-openbsd nomarch opendkim opendkim-tools opendmarc   pax pflogsumm postgrey p7zip-full postfix-ldap postfix-pcre   postfix-policyd-spf-python postsrsd pyzor   razor rpm2cpio rsyslog sasl2-bin spamassassin supervisor   unrar-free unzip whois xz-utils &&   gpg --keyserver ${FAIL2BAN_GPG_PUBLIC_KEY_SERVER}     --recv-keys ${FAIL2BAN_GPG_PUBLIC_KEY_ID} &>/dev/null &&   curl -Lkso fail2ban.deb ${FAIL2BAN_DEB_URL} &&   curl -Lkso fail2ban.deb.asc ${FAIL2BAN_DEB_ASC_URL} &&   FINGERPRINT="$(LANG=C gpg --verify   fail2ban.deb.asc fail2ban.deb 2>&1     | sed -n 's#Primary key fingerprint: $.*$#\1#p')" &&   if [[ -z ${FINGERPRINT} ]]; then     echo "ERROR: Invalid GPG signature!" 2>&1; exit 1; fi &&   if [[ ${FINGERPRINT} != "${FAIL2BAN_GPG_FINGERPRINT}" ]]; then     echo "ERROR: Wrong GPG fingerprint!" 2>&1; exit 1; fi &&   dpkg -i fail2ban.deb &>/dev/null &&   rm fail2ban.deb fail2ban.deb.asc &&   apt-get -qq autoremove &>/dev/null &&   apt-get -qq autoclean &&   apt-get -qq clean &&   rm -rf /var/lib/apt/lists/* &&   c_rehash &>/dev/null
 ---> Running in 851a120aca3a
Removing intermediate container 851a120aca3a
 ---> 6d9f68aa9c37
Step 31/70 : RUN   echo '0 */6 * * * clamav /usr/bin/freshclam --quiet' >/etc/cron.d/clamav-freshclam &&   chmod 644 /etc/clamav/freshclam.conf &&   freshclam &&   sed -i 's/Foreground false/Foreground true/g' /etc/clamav/clamd.conf &&   mkdir /var/run/clamav &&   chown -R clamav:root /var/run/clamav &&   rm -rf /var/log/clamav/
 ---> Running in f16d39d5623b

/bin/sh: 1: cannot create /etc/cron.d/clamav-freshclam: Directory nonexistent

The command '/bin/sh -c echo '0 */6 * * * clamav /usr/bin/freshclam --quiet' >/etc/cron.d/clamav-freshclam &&   chmod 644 /etc/clamav/freshclam.conf &&   freshclam &&   sed -i 's/Foreground false/Foreground true/g' /etc/clamav/clamd.conf &&   mkdir /var/run/clamav &&   chown -R clamav:root /var/run/clamav &&   rm -rf /var/log/clamav/' returned a non-zero code: 2
make: *** [Makefile:20: build] Error 2
```

</details>

</details>

Which hints that something went wrong with prior package installation step..

### Expected Behavior

Successfully build the image locally.

## Your Environment

- OS: Vultr VPS, Ubuntu 21.04
- version: `master` branch
- available RAM: `2GB`
- Docker version: `20.10.7, build f0df350`


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[BUG] Building Dockerfile locally fails #2023

Bug Report

Context

What is affected by this bug?

When does this occur?

How do we replicate the issue?

Behavior

Actual Behavior

Expected Behavior

Your Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Uh oh!

[BUG] Building Dockerfile locally fails #2023

Description

Bug Report

Context

What is affected by this bug?

When does this occur?

How do we replicate the issue?

Behavior

Actual Behavior

Expected Behavior

Your Environment

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions