Add testing philosophy documentation#4504
Merged
cidrblock merged 30 commits intoansible:mainfrom Aug 15, 2025
Merged
Conversation
- Add comprehensive philosophy documentation covering testing suite fundamentals - Explain essential functionality, configuration flexibility, and lifecycle management - Document how Molecule addresses testing suite requirements for Ansible automation - Cover sequence configuration, shared state/inventory, and extensibility design - Position philosophy as foundational understanding before installation guide This documentation provides the conceptual foundation for understanding Molecule's design decisions and architectural approach to Ansible testing.
- Broaden scope from infrastructure-only to all Ansible use cases - Include application configuration, deployment, API integration, business processes - Update testing requirements to cover infrastructure, apps, services, and integrations - Expand Molecule capabilities explanation for multi-domain automation - Cover diverse platforms: infrastructure, applications, network devices, cloud services, APIs - Address variable management across all credential and configuration types - Update examples to reflect broader automation scenarios and use cases This ensures the philosophy documentation accurately represents Ansible's full automation capabilities and how Molecule supports testing across all domains, not just infrastructure provisioning.
- Add testing workflow fundamentals section covering all testing phases - Document core phases: provisioning, dependency resolution, convergence, idempotence, verification, side effects, cleanup, and destruction - Map testing phases to Molecule actions with detailed purpose explanations - Provide comprehensive command-line subcommand reference with usage scenarios - Add configurable sequence examples for different testing strategies - Enhance shared state documentation with cost and speed optimization benefits - Contrast shared state vs. isolation scenarios with specific recommendations - Include practical workflow examples for both shared and isolated testing - Document isolation vs. efficiency trade-offs for different use cases This provides complete coverage of testing workflow concepts and their implementation in Molecule, helping teams choose appropriate testing strategies for their specific automation domains and requirements.
- Remove Ansible-specific references from core testing phases - Change 'Ansible roles and collections' to 'automation modules' - Change 'playbooks, roles, or tasks' to 'automation scripts, configurations, or procedures' - Ensure testing workflow fundamentals apply to any robust testing framework - Maintain comprehensive coverage while being technology-neutral The core testing phases now describe universal testing framework capabilities rather than Ansible-specific implementation details, making the philosophy section more broadly applicable while still leading naturally into how Molecule implements these concepts for Ansible automation.
- Remove 'automation' specificity from 'What a testing suite provides' - Change from 'Automation testing frameworks' to 'Testing frameworks' - Update from 'automation domains' to 'testing domains' throughout - Change 'automation workflows' to 'development workflows' - Update 'automation testing' to just 'testing' where appropriate - Change 'automation logic' to 'system logic' and 'system under test' - Update examples from automation-specific to universal testing concepts - Make testing workflow fundamentals apply to all testing domains The philosophy section now describes universal testing framework capabilities that apply to unit testing, integration testing, system testing, and any other testing domain, not just automation testing. This provides a broader foundation while still leading naturally into Molecule's automation-specific implementation.
- Fix shared state explanation to accurately reflect default scenario usage - Document that non-default scenarios skip create/destroy when shared state is enabled - Explain that default scenario manages infrastructure lifecycle for shared state - Update workflow examples to show proper default scenario usage - Correct shared inventory coordination to show default scenario role - Fix state isolation examples to reflect actual implementation behavior Based on code review of src/molecule/command/base.py: - execute_scenario() skips create/destroy for shared_data scenarios (lines 357-359) - execute_subcommand_default() runs create/destroy from default scenario (lines 291-321) - Default scenario must exist and have shared_data=True for shared state to work This ensures documentation accurately reflects how Molecule actually implements shared state functionality rather than how it might conceptually work.
- Correct shared state documentation to reflect automatic behavior - Document that Molecule automatically runs default create before scenarios - Document that Molecule automatically runs default destroy after scenarios - Remove manual default scenario management from examples - Show proper single scenario and multi-scenario automatic workflows - Document molecule test --all --shared-state automatic behavior - Fix shared inventory examples to show automatic coordination Based on code review of src/molecule/command/base.py _run_scenarios(): - Line 227: execute_subcommand_default(default_config, 'create') runs automatically - Line 286: execute_subcommand_default(default_config, 'destroy') runs automatically - Lines 162-165: Default config auto-detected from MOLECULE_DEFAULT_SCENARIO_NAME Users never need to manually run default scenario - Molecule handles it automatically when --shared-state is enabled. This provides the key benefit of running create/destroy only once even with molecule test --all.
…aring - Clarify that shared inventory centralizes inventory location for all scenarios - Document that default scenario writes connection details to shared inventory - Document that other scenarios read from shared inventory to access same resources - Add detailed explanation of how shared inventory enables resource sharing - Emphasize write-once-read-many pattern for resource connection details - Show that variables, groups, and connection options are shared through inventory - Update workflow examples to highlight inventory-based resource sharing - Connect shared-state and shared-inventory as complementary mechanisms Technical details: - Default scenario provisions infrastructure and writes to shared inventory - All other scenarios read from same shared inventory location - This enables true resource sharing without duplicate provisioning - Each scenario sees identical infrastructure through shared inventory
- Remove complex driver architecture discussion - Focus on Ansible as the default driver and its benefits - Emphasize consistency between test and production patterns - Highlight reduced learning curves through familiar constructs - Document reliability benefits of using same Ansible knowledge - Show how Ansible's declarative syntax improves test scenarios - Connect test environment management to production automation Benefits highlighted: - Same Ansible knowledge for test and production - Familiar playbooks and patterns across domains - Declarative syntax for all environment management - Test scenarios mirror real-world automation usage - Reduced complexity and learning requirements
- Remove complex plugin architecture discussion - Focus on Ansible as both provisioner and verifier - Emphasize unified automation execution and validation logic - Highlight same skills, modules, and patterns for test and production - Document benefits of Ansible's extensive module ecosystem - Show consistent interfaces across all automation domains - Emphasize reduced complexity through unified approach - Connect test validation to existing team Ansible expertise Benefits highlighted: - Unified approach using Ansible for both execution and validation - Leverages existing team Ansible skills and knowledge - Consistent module ecosystem across all domains - Easily understood and maintainable by Ansible practitioners - Reduces learning curve and operational complexity
- Document Molecule's 9-year evolution since 2015 inception - Highlight that defaults are optimized from real-world usage at scale - Explain enterprise workflow customization capabilities - Emphasize sequence configurations as committed, shareable code - Connect team consistency through version-controlled test sequences - Show how defaults satisfy most users while enabling enterprise flexibility Key points added: - Real-world optimization of defaults over 9 years - Enterprise-specific workflow requirements recognition - Configuration as code for team consistency - Committed sequences shared between team members - Balance between sensible defaults and customization flexibility
- Document Molecule's strategic focus on deeper Ansible integration - Emphasize specialization over broader tool support - Detail current and planned integration enhancements - Cover enhanced collection testing with shared state/inventory - Describe unified UX with Ansible output alignment - Explain deeper inventory integration and validation - Document ansible-navigator and execution environment support - Detail automatic collection detection and dependency resolution - Establish strategic direction as definitive Ansible testing solution - Emphasize backward compatibility while advancing Ansible-native experience - Position Molecule as specialized framework rather than generic testing tool Key themes: - Ansible as de facto automation DSL drives development priorities - Molecule + Ansible experience optimization over tool breadth - Seamless integration while maintaining compatibility - Enterprise collection testing workflow improvements - Natural transition between development, testing, and deployment
648ebb9 to
57b870f
Compare
dd9444e to
849c740
Compare
4abfb73 to
ff45252
Compare
ffc265c to
2a6850f
Compare
- Replace all 'provisioner:' sections with modern 'ansible:' configuration - Convert 'ansible_args' to 'ansible.executor.args.ansible_playbook' - Add ansible.cfg configuration example for enhanced integration - Update descriptive text to reference modern configuration patterns - Maintain all existing functionality and testing approaches - Ensure schema compliance and consistent formatting All configuration examples now use the supported ansible-native approach while preserving the comprehensive testing philosophy and strategies.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #4505
Summary
This pull request adds comprehensive testing philosophy documentation to help users understand Molecule's approach to Ansible automation testing and its strategic direction.
Changes Made
New Documentation: docs/philosophy.md
Key Technical Improvements
Documentation Quality
Target Audience
This documentation serves automation practitioners, enterprise teams, and decision makers who need to understand Molecule's testing philosophy, capabilities, and strategic direction within the Ansible ecosystem.
Testing
ROOT: provision> .tox/.tox/bin/python -m tox -e docs
.pkg: _optional_hooks> python /home/bthornto/github/molecule311/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_editable> python /home/bthornto/github/molecule311/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_editable> python /home/bthornto/github/molecule311/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
docs: install_package> .tox/.tox/bin/uv pip install --reinstall --no-deps molecule@/home/bthornto/github/molecule311/.tox/.tmp/package/49/molecule-25.7.1.dev20-0.editable-py3-none-any.whl
docs: commands_pre[0]> sh -c 'rm -f /home/bthornto/github/molecule311/.tox/docs/.coverage* 2>/dev/null || true'
docs: commands[0]> mkdocs build --strict --site-dir=_readthedocs/html/
docs: OK (8.61=setup[1.41]+cmd[0.01,7.19] seconds)
congratulations :) (8.71 seconds)