Implement Action Completion Reporting System

DEMO

Overview

This PR introduces a structured action completion reporting system that provides users with clear, consistent feedback about command execution status while maintaining semantic clarity in the codebase. The implementation focuses on improving user experience through better visual feedback and establishing a solid foundation for future reporting enhancements.

Core Data Structure Hierarchy

The new reporting system introduces a hierarchical data structure:

ScenariosResults (list[ScenarioResults])
└── ScenarioResults (dataclass)
    ├── name: str
    └── actions: list[ActionResult]
        └── ActionResult (dataclass)
            ├── action: str | None
            ├── states: list[CompletionStateInfo]
            │   └── CompletionStateInfo (dataclass)
            │       ├── state: str
            │       ├── message: str
            │       ├── note: str | None
            │       └── color: str (computed from ANSICodes)
            └── summary: CompletionStateInfo (computed property)

Result Capture Flow

The reporting system follows a clear capture pattern where results flow from individual actions up to the final scenarios collection:

1. Scenario Initialization

When a scenario starts, a ScenarioResults container is initialized:

# In src/molecule/scenario.py
def __init__(self, config: Config) -> None:
    self.results: ScenarioResults = ScenarioResults(name=config.scenario.name, actions=[])

2. Action Container Creation

Before each action executes, an ActionResult container is created:

# In command execution flow
scenario.results.add_action_result("dependency")  # Creates new ActionResult for this action
scenario.results.add_action_result("create")
scenario.results.add_action_result("converge")

3. Completion State Capture

Individual playbooks and operations report their completion states:

# In src/molecule/provisioner/ansible_playbook.py
def execute(self, action_args: list[str] | None = None) -> None:
    if not self._config.provisioner.playbooks.converge:
        self._config.scenario.results.add_completion(CompletionState.missing)
        return
    
    try:
        result = util.run_command(cmd, cwd=cwd, env=env)
        if result.returncode != 0:
            self._config.scenario.results.add_completion(CompletionState.failed)
        else:
            self._config.scenario.results.add_completion(CompletionState.successful)
    except Exception:
        self._config.scenario.results.add_completion(CompletionState.failed)

4. Scenario Collection

After scenario completion, the ScenarioResults is appended to the global collection:

# In src/molecule/command/base.py
def _run_scenarios(self, scenarios: list[Scenario]) -> ScenariosResults:
    scenarios_results = ScenariosResults()
    for scenario in scenarios:
        self.execute_scenario(scenario)
        scenarios_results.append(scenario.results)  # Add completed scenario results
    return scenarios_results

5. Summary Generation

The ActionResult.summary property intelligently aggregates multiple completion states:

# In src/molecule/reporting.py
@property
def summary(self) -> CompletionStateInfo:
    """Generate summary completion state from all states in this action."""
    if not self.states:
        return CompletionState.partial("No states recorded")
    
    # Priority ranking: failed > partial > missing > disabled > skipped > successful
    state_counts = Counter(state.states for state in self.states)
    
    for rank in ["failed", "partial", "missing", "disabled", "skipped", "successful"]:
        if rank in state_counts:
            return cast(CompletionStateInfo, getattr(CompletionState, rank))(
                message=partial_message, note=note
            )

6. Real-time Log Messages

The section_logger decorator uses completion states for immediate user feedback:

# In src/molecule/logger.py
def section_logger(func: Callable[..., None]) -> Callable[..., None]:
    def wrapper(self: Any, *args: Any, **kwargs: Any) -> None:
        # Log starting message
        logger.info("%s ➜ %s Starting", scenario_name, action_name, extra=extra)
        
        func(self, *args, **kwargs)  # Execute the actual command
        
        # Get completion state and log result
        state_info = config_instance._config.scenario.results.last_action_summary
        logger.log(
            state_info.log_level,
            "%s%s ➜ %s %s%s%s",
            state_info.color, scenario_name, action_name, 
            state_info.message, note_part, ANSICodes.RESET,
            extra=extra
        )

7. Final Report Generation

At execution end, the complete results generate a user-facing summary:

# In src/molecule/reporting.py
def report(results: ScenariosResults) -> None:
    """Generate final execution report from all scenario results."""
    if not results:
        return
        
    ao = AnsiOutput()
    ao.write(ao.process_markup("Summary", "scenario"))
    
    for scenario_result in results:
        for action_result in scenario_result.actions:
            summary = action_result.summary
            ao.write(f"{summary.color}{scenario_result.name} ➜ {action_result.action} {summary.message}")
            if summary.note:
                ao.write(f" ({summary.note})")
            ao.write(f"{ANSICodes.RESET}")

File Changes by Category

Core Reporting Infrastructure

• src/molecule/reporting.py (NEW): Complete reporting system implementation with dataclasses for CompletionStateInfo, ActionResult, ScenarioResults, and ScenariosResults. Includes the main report() function for generating user-facing summaries.

Constants and ANSI Management

• src/molecule/constants.py: Refactored to centralize ANSI codes using StrEnum, introduced CompletionState class with predefined completion states as class attributes, and implemented dynamic MARKUP_MAP with __missing__ method for robust key lookups.

Logger Integration

• src/molecule/logger.py: Enhanced section_logger decorator to integrate with the new completion state system, adding structured logging with molecule_scenario and molecule_step extras for better traceability.

Command Execution Integration

• src/molecule/provisioner/ansible_playbook.py: Updated to report completion states (missing, failed, successful) through the new result tracking system.
• src/molecule/verifier/ansible.py: Integrated completion state reporting for verify operations, handling both disabled and missing playbook scenarios.
• src/molecule/command/base.py: Enhanced base command execution to work with the new reporting hierarchy.

Test Coverage and Quality

• tests/unit/test_constants.py (NEW): Comprehensive test coverage for ANSI codes, markup mapping, and constant validation with proper handling of StrEnum comparison quirks.
• tests/unit/test_reporting.py (NEW): Full test suite for the reporting system using pytest-only approach with monkeypatch for mocking.
• tests/integration/test_command.py: Updated integration tests to validate actual user-facing output formatting, including proper spacing and completion message validation.
• Multiple unit test files: Fixed to work with new structured logging system, replacing caplog.text assertions with caplog.records inspection for better reliability.

User Experience Enhancements

The new system provides:

• Clear visual feedback: Consistent starting and completion messages with appropriate ANSI coloring
• Structured reporting: End-of-execution summaries showing all actions and their outcomes
• Better error context: Enhanced logging with scenario and step information for easier debugging

Breaking Changes

None. The changes are fully backward compatible and maintain existing CLI behavior while enhancing the underlying reporting infrastructure.

This implementation establishes a solid foundation for future reporting enhancements while immediately improving user experience through clearer, more consistent feedback during command execution.