Skip to content

MacOS linting & testing support + docs#2001

Merged
georglauterbach merged 24 commits intodocker-mailserver:masterfrom
NorseGaud:macos-lint-and-test-support
Jun 7, 2021
Merged

MacOS linting & testing support + docs#2001
georglauterbach merged 24 commits intodocker-mailserver:masterfrom
NorseGaud:macos-lint-and-test-support

Conversation

@NorseGaud
Copy link
Copy Markdown
Member

@NorseGaud NorseGaud commented May 24, 2021

Description

macOS users cannot easily contribute as linting and testing doesn't work locally. Some code is linux/GNU only and breaks when running on mac for various reasons.

A key change: Linting no longer requires you to run things on your host.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (non-breaking change that does improve existing functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@NorseGaud NorseGaud mentioned this pull request May 24, 2021
7 tasks
@NorseGaud NorseGaud changed the title MacOS linting + testing support + docs MacOS linting & testing support + docs May 24, 2021
@wernerfred wernerfred added the meta/feature freeze On hold due to upcoming release process label May 24, 2021
@wernerfred wernerfred added this to the v10.0.1 milestone May 24, 2021
@georglauterbach
Copy link
Copy Markdown
Member

Since this builds on #2000, it does not need to incorporate the same changes again. When #2000 is merged, this PR will possibly have unnecessary merge conflicts. Please remove the changes that are already present in #2000.

Not sure about #1980, but it seems to have these very changes too if I‘m not mistaken.

@NorseGaud

This comment has been minimized.

@georglauterbach

This comment has been minimized.

@polarathene

This comment has been minimized.

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My review is mostly regarding the docs, but I have a few concerns with other changes that I'd appreciate some clarity on.

Comment thread docs/content/contributing/tests.md Outdated
Comment thread Makefile Outdated
Comment thread Makefile Outdated
Comment thread docs/content/contributing/issues-and-pull-requests.md
@wernerfred wernerfred removed the meta/feature freeze On hold due to upcoming release process label Jun 1, 2021
@NorseGaud NorseGaud requested a review from polarathene June 3, 2021 02:11
Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we just have the switch from the current docker image approach for linting to docker-compose using official lint images directly and we're good! 🎉

For the other maintainers, note that the improvement to image name collection was reverted. It did reveal that our current filter likewise isn't identifying all test images by name assigned, mostly because there is no established/documented convention enforced which should be addressed at some point.

Comment thread docs/content/contributing/tests.md
Comment thread setup.sh Outdated
Comment thread Makefile Outdated
Comment thread setup.sh Outdated
Copy link
Copy Markdown
Member

@georglauterbach georglauterbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small parts that still need clarifying IMHO. But the majority looks good!

@polarathene
Copy link
Copy Markdown
Member

polarathene commented Jun 4, 2021

Follow up comment to my earlier review comment. Below shows my preferred approach with using docker images for linting, but they need to be further adapted into lint.sh and verified as working the same.


I have slightly improved/modified EC config below, and enabled --check-sources for shellcheck config (which raises a single SC2153 warning that wasn't picked up). I also believe I resolved the readlink -f issue that was being had on macOS, I am not entirely sure why the more complicated workaround was in place, the option for --source-path has been available for about 2 years or so. @aendeavor added it in Oct 2020 for --external-sources, perhaps was just missing the --source-path=SCRIPTDIR config addition?

Rough shell commands to run each official docker image with config like in lint.sh. If they're successful with no issues, they'll be blank in output when exiting (should function just like error handling for existing command usage). Each one seemed to run fine and as expected, although I've only run them individually manually, no lint.sh.

HADOLINT_VERSION=2.4.1
EC_VERSION=2.3.5
SC_VERSION=0.7.2

docker run --rm --tty \
  --volume "${PWD}:/ci:ro" \
  --workdir "/ci" \
  "hadolint/hadolint${HADOLINT_VERSION}-alpine" hadolint --config "test/linting/.hadolint.yaml" Dockerfile

docker run --rm --tty \
  --volume "${PWD}:/ci:ro" \
  --workdir "/ci" \
  "mstruebing/editorconfig-checker:${EC_VERSION}" ec -config "test/linting/.ecrc.json"

# File paths for shellcheck:
local F_SH="$(find . -type f -iname '*.sh' \
  -not -path './test/bats/*' \
  -not -path './test/test_helper/*' \
  -not -path './target/docker-configomat/*' \
)"
local F_BIN="$(find 'target/bin' -type f -executable)"
local F_BATS="$(find 'test/' -maxdepth 1 -type f -iname '*.bats')"

# This command is a bit easier to grok as multi-line. There is a `.shellcheckrc` file, but it's only supports half of the options below, thus kept as CLI:
local CMD_SHELLCHECK=(
  shellcheck --external-sources --check-sources
  --severity=style
  --color=auto
  --wiki-link-count=50
  --enable=all
  --exclude=SC2154
  --source-path=SCRIPTDIR
  "${F_SH} ${F_BIN} ${F_BATS}"
)

docker run --rm --tty \
  --volume "${PWD}:/ci:ro" \
  --workdir "/ci" \
  "koalaman/shellcheck-alpine:v${SC_VERSION}" ${CMD_SHELLCHECK[@]}

test/linting/.ecrc.json: (modified to remove version as it's no longer required and the docker image was not compiled with it correctly, also moves the exclude config into the actual config file)

{
  "Verbose": false,
  "Debug": false,
  "IgnoreDefaults": false,
  "SpacesAftertabs": true,
  "NoColor": false,
  "Exclude": [
    "^test/",
    "\\.git.*",
    "\\.bats$",
    "\\.cf$",
    "\\.conf$",
    "\\.init$",
    "\\.md$"
  ],
  "AllowedContentTypes": [],
  "PassedFiles": [],
  "Disable": {
    "EndOfLine": false,
    "Indentation": false,
    "InsertFinalNewline": false,
    "TrimTrailingWhitespace": false,
    "IndentSize": false,
    "MaxLineLength": false
  }
}

@georglauterbach

This comment has been minimized.

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there, thanks for your patience! 😅

Comment thread Makefile Outdated
Comment thread docs/content/contributing/issues-and-pull-requests.md Outdated
Comment thread test/linting/lint.sh
Co-authored-by: Brennan Kinney <[email protected]>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 7, 2021

Documentation preview for this PR is ready! 🎉

Built with commit: 7707464

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work @NorseGaud ! Thanks for taking the time to contribute 😀


TL;DR for reviewers of PR thread thus far

Summary of anything possibly worth pointing out:


#2001 (comment) notes that this should have been reverted, but has resulted in: "${@:+$@}", apparently due to "unbound variable problems".


#2001 (comment) notes that the change from find option -executable for compatibility reasons in lint.sh may result in potential false positives, but it's probably unlikely for this project:

[[ "$(uname)" == "Darwin" ]] && PERM="+111" || PERM="/111" 
F_BIN="$(find 'target/bin' -perm "${PERM}" -type f -or -type l)"

#2001 (comment) notes another minor find change with test instead of test/ for matching the target directory to scan (this only covers a limited depth mind you, not sure if that's intentional to skip some of the contents further down?). Should otherwise be harmless?


#2001 (comment) notes that logging the linter version has been dropped. If maintainers still desire that, now that we use docker images for linting, the versions in the variables should be sufficient to log with?


#2001 (comment) notes ERR has been dropped in lint.sh, but is still referenced by this line:

trap '__log_err "${FUNCNAME[0]:-?}" "${BASH_COMMAND:-?}" ${LINENO:-?} ${?:-?}' ERR


#2001 (review) notes unrelated to changes in this PR, our cleanup logic in Makefile needs to be addressed, tests aren't conforming to conventions preventing them from being matched.


#2001 (comment) notes unrelated to this PR, the issues and pull requests docs page still has a rendering issue of nested lists.

@georglauterbach georglauterbach added area/ci area/scripts kind/improvement Improve an existing feature, configuration file or the documentation kind/new feature A new feature is requested in this issue or implemeted with this PR priority/medium os/macOS labels Jun 7, 2021
@georglauterbach georglauterbach merged commit 543bd8b into docker-mailserver:master Jun 7, 2021
@NorseGaud NorseGaud deleted the macos-lint-and-test-support branch June 7, 2021 21:58
@polarathene
Copy link
Copy Markdown
Member

@aendeavor is the trap line with ERR at the end still valid?

@georglauterbach
Copy link
Copy Markdown
Member

@aendeavor is the trap line with ERR at the end still valid?

Yes :) In case of an unexpected error, it will print information via the __log_err function. This can help debugging or with issue reports. I'd leave it in there.

Comment thread Makefile
Comment thread setup.sh
then
# reuse existing container specified on command line
${CRI} exec "${USE_TTY}" "${CONTAINER_NAME}" "${@}"
${CRI} exec "${USE_TTY}" "${CONTAINER_NAME}" "${@:+$@}"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the purpose of all these "${@:+$@}".

If $@ is not empty; then use $@?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you encounter a problem or was this added in good faith?

BASH 4.0.0 was released more than 10 years ago. And the bug got fixed in later 4.x releases (just verified it with debian9 which included BASH 4.4).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with the latest bash in macOS it was throwing the error for me. It's a macOS specific fix

Copy link
Copy Markdown
Member

@wernerfred wernerfred Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't Mac using Bash 3 by default? Or did you install bash 4 via brew

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this all presumes the user has /usr/local/bin in their PATH before /bin... I personally think ${@:+$@} is worth it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my mac i needed to hash -r after brew install bash and this only works bc /usr/local/bin ist listed first on default environments. So if it is environment safe as long as bash4+ is installed anywhere on the system we should go with ${@:+$@} i think

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure TBH. This construct just looks weird to me 😆

But I guess we could make an effort to support macOS users:)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't Mac using Bash 3 by default?

Holy shit. YMMD 😄 Couldn't believe that's true.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth leaving a comment with description or link about it for future maintainer reference? Or is the link to this PR with lengthy discussion thread to wade through sufficient?

Comment thread test/linting/lint.sh

SCRIPT="lint.sh"

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCRIPT_DIR=$(dirname "$(readlink -f "$0")")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix in #1980

@polarathene
Copy link
Copy Markdown
Member

polarathene commented Jun 8, 2021

Yes :) In case of an unexpected error, it will print information via the __log_err function. This can help debugging or with issue reports. I'd leave it in there.

@aendeavor I meant the ERR variable part as it no longer exists anywhere else in the file?

@georglauterbach
Copy link
Copy Markdown
Member

Yes :) In case of an unexpected error, it will print information via the __log_err function. This can help debugging or with issue reports. I'd leave it in there.

@aendeavor I meant the ERR variable part as it no longer exists anywhere else in the file?

I understand. Then this may not be needed anymore and can be removed entirely I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci area/scripts kind/improvement Improve an existing feature, configuration file or the documentation kind/new feature A new feature is requested in this issue or implemeted with this PR os/macOS priority/medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants