Summary

Adds --workers N to molecule test, check, and destroy for running collection scenarios concurrently using concurrent.futures.ProcessPoolExecutor. The existing sequential code path is completely untouched when --workers is absent or set to 1.

Design

The parallel path runs only in collection mode with shared_state. The main process handles the default scenario's create and destroy lifecycle serially. All non-default scenarios are submitted to a process pool and execute their prepare → converge → idempotence → verify → cleanup sequence in isolated workers. Results are collected via futures.

Failure modes:

• fail-fast (default): on first failure, executor.shutdown(cancel_futures=True) is called, remaining workers are cancelled, destroy runs, and collected failures are reported.
• --continue-on-failure: all scenarios run to completion, failures are buffered and displayed after destroy.

Ansible output is suppressed in workers by default (Molecule INFO logs still stream freely). On failure, the captured stdout+stderr is displayed in bordered blocks after the default scenario's destroy phase completes. Pass -v to stream all ansible output from workers.

File-by-file changes

Core runtime (minimal changes, no behavioral change for existing users)

src/molecule/command/base.py — 8 lines added. _run_scenarios gains an early return: if workers > 1, it lazy-imports and delegates to run_scenarios_parallel. The entire existing sequential path is untouched; it only runs when workers <= 1 (the default).

src/molecule/app.py — 4 lines changed. run_command checks MOLECULE_QUIET_ANSIBLE env var. When set, tee is disabled (ansible stdout is captured but not streamed) and CommandBorders initialization is skipped. This env var is only set by worker processes; normal runs are unaffected.

src/molecule/exceptions.py — 3 lines added. ScenarioFailureError.__init__ accepts an optional ansible_output string to carry captured ansible stdout+stderr from the failing step. Fully backward-compatible; defaults to "".

src/molecule/provisioner/ansible_playbook.py — 2 lines added. On non-zero ansible exit, the combined result.stdout + result.stderr is passed as ansible_output to ScenarioFailureError. No change to the raise itself or any other behavior.

src/molecule/state.py — 4 lines changed. _get_data now merges loaded state on top of defaults instead of returning loaded state directly. Handles the case where a state file written by an older molecule version is missing keys that newer code expects. Defensive fix; no impact on current sequential runs.

src/molecule/types.py — 6 lines added. CommandArgs TypedDict extended with workers: int and continue_on_failure: bool. Non-breaking since total=False.

New module

src/molecule/worker.py — 278 lines (new). Contains:

• run_one_scenario(): reconstructs a Config from picklable args in a worker process, runs execute_scenario with shared_state=True, returns a 4-tuple of (ScenarioResults, error_msg, ansible_output, failed_step).
• run_scenarios_parallel(): orchestrates the full lifecycle — default create, prerun, pool dispatch, result collection, destroy, failure reporting.
• validate_worker_args(): ensures --workers > 1 is only used in collection mode and not with --destroy=never.
• _print_failed_output(): renders bordered failure blocks to stderr after destroy.

Forces prepare=True on every worker Config to avoid the shared-state "already prepared" flag race condition. Comment in code explains why this is necessary vs sequential mode.

CLI plumbing

src/molecule/click_cfg.py — 56 lines added. New workers and continue_on_failure CLI options. New resolve_workers() function that parses "cpus", "cpus-1", or an integer string.

src/molecule/command/test.py, check.py, destroy.py — Each adds --workers and --continue-on-failure to its option list, calls resolve_workers(), emits a DeprecationWarning if --parallel is used.

Output formatting

src/molecule/ansi_output.py — 3 new module-level functions extracted from CommandBorders: create_border_header(), create_border_footer(), write_bordered_block(). Used by _print_failed_output in worker.py to render failure output. Existing CommandBorders class is untouched.

Documentation

docs/guides/parallel.md — Updated to document --workers, --continue-on-failure, and the deprecation of --parallel.

Configuration

pyproject.toml — 1 line: added R0401 (cyclic-import) to pylint's global disable list. The base.py → worker.py lazy import is intentional and unavoidable.

.config/dictionary.txt — cspell dictionary update.

Tests

tests/unit/test_worker.py — 430 lines (new). Unit tests for validate_worker_args, run_one_scenario, and run_scenarios_parallel. Uses monkeypatch/mocker.patch (no @patch decorators, no test classes). Covers success, failure, fail-fast, continue-on-failure, and quiet-ansible env var behavior.

tests/unit/test_click_cfg.py — 118 lines added. Tests for resolve_workers: integers, cpus, cpus-1, case insensitivity, None fallback, invalid input, zero, negative values.

tests/integration/test_workers.py — 168 lines (new). 4 end-to-end tests against a static fixture collection (tests/fixtures/integration/test_workers/): parallel success with output suppression and report, verbose mode ansible streaming, continue-on-failure with grouped failure output, and fail-fast with cpus-1 resolution.

tests/fixtures/integration/test_workers/ — Static ansible-native collection with a default scenario (create/destroy) and 6 lightweight test scenarios using ansible.builtin.debug tasks.

Test plan

• tox -e lint — all 22 pre-commit checks pass (ruff, pylint, mypy, pydoclint, cspell, etc.)
• tox -e py — unit and integration tests pass
• Sequential execution (--workers 1 or omitted) is completely unaffected
• CI passes on this PR

Made with Cursor