Skip to content

feat(powerpoint): add Hypothesis property tests for priority modules #1013

@WilliamBerryiii

Description

@WilliamBerryiii

Summary

Add Hypothesis property-based testing to the PowerPoint skill's priority modules. The codebase currently has ~300+ deterministic unit tests with well-formed inputs but zero property-based or fuzz tests. Hypothesis will generate structured adversarial inputs (malformed YAML dicts, edge-case values, type-confused inputs) that exercise business logic with invalid data — the primary gap in the current test suite.

Context

This is Phase 1 of the Python Security Testing & Fuzzing Initiative. Hypothesis was selected over ClusterFuzzLite/Atheris as the primary testing approach because:

  • The codebase's primary input is structured YAML dictionaries — Hypothesis generates valid structured inputs natively while Atheris wastes cycles on syntactically invalid raw bytes
  • Zero infrastructure overhead (no Docker, storage repos, or PAT tokens)
  • Native pytest integration with the existing test suite
  • Automatic test case shrinking for minimal reproducing examples

Implementation

Dependencies

Add to .github/skills/experimental/powerpoint/pyproject.toml:

[dependency-groups]
dev = [
    "hypothesis>=6.100",
    # ... existing dev deps
]

Priority Test Targets

1. validate_slides.py / validate_deck.py — Input validation robustness

from hypothesis import given, settings
import hypothesis.strategies as st

yaml_values = st.recursive(
    st.none() | st.booleans() | st.integers() | st.floats(allow_nan=False) | st.text(),
    lambda children: st.lists(children) | st.dictionaries(st.text(), children),
    max_leaves=50,
)

@given(data=yaml_values)
@settings(max_examples=500, deadline=None)
def test_validate_slides_handles_arbitrary_yaml(data):
    """Property: validate functions never raise unhandled exceptions on arbitrary input."""
    try:
        validate_slides(data)
    except (ValueError, TypeError, KeyError):
        pass  # Expected rejections

2. build_deck.py — Element builder dispatch robustness

slide_element = st.fixed_dictionaries({
    "type": st.sampled_from(["text", "image", "table", "chart", "shape", "group", "connector"]),
}, optional={
    "x": st.floats(min_value=-100, max_value=100),
    "y": st.floats(min_value=-100, max_value=100),
    "width": st.floats(min_value=0, max_value=100),
    "height": st.floats(min_value=0, max_value=100),
    "text": st.text(max_size=1000),
    "rows": st.integers(min_value=0, max_value=50),
    "columns": st.integers(min_value=0, max_value=50),
})

@given(elements=st.lists(slide_element, max_size=20))
@settings(max_examples=300, deadline=None)
def test_build_slide_handles_arbitrary_elements(elements):
    """Property: build_slide handles arbitrary element definitions without crashing."""
    ...

3. pptx_colors.py — Hex parsing edge cases

@given(hex_str=st.text(alphabet="0123456789abcdefABCDEF", min_size=0, max_size=20))
def test_resolve_color_hex_robustness(hex_str):
    """Property: resolve_color handles any hex-like string."""
    try:
        resolve_color(hex_str)
    except (ValueError, TypeError):
        pass

4. pptx_tables.py — Merge bounds validation

@given(
    rows=st.integers(min_value=1, max_value=20),
    cols=st.integers(min_value=1, max_value=20),
    merge_right=st.integers(min_value=-5, max_value=25),
    merge_down=st.integers(min_value=-5, max_value=25),
)
def test_table_merge_bounds(rows, cols, merge_right, merge_down):
    """Property: table merge operations handle out-of-bounds gracefully."""
    ...

File Organization

New test files follow the existing pattern under .github/skills/experimental/powerpoint/tests/:

  • test_fuzz_validate_slides.py
  • test_fuzz_build_deck.py
  • test_fuzz_pptx_colors.py
  • test_fuzz_pptx_tables.py
  • test_fuzz_extract_content.py (for XML parsing and blob extraction)

RPI Framework

task-researcher

  • Review each priority module's public API and input types
  • Identify which functions accept external/untrusted input
  • Catalog existing edge cases already covered by deterministic tests
  • Determine appropriate Hypothesis strategy configurations for each input type

task-planner

  • Define property invariants for each target function (what should always be true)
  • Design custom Hypothesis strategies matching the YAML slide definition schema
  • Plan test file organization and naming conventions
  • Set max_examples and deadline settings appropriate for CI runtime

task-implementor

  • Add hypothesis>=6.100 to pyproject.toml dev dependencies
  • Implement property tests for all 4 priority targets (minimum)
  • Extend to extract_content.py and pptx_text.py as stretch goals
  • Verify all tests pass in CI and Hypothesis database is persisted
  • Document property test conventions for future contributors

Acceptance Criteria

  • hypothesis>=6.100 is added to pyproject.toml dev dependencies
  • Property tests exist for validate_slides.py, build_deck.py, pptx_colors.py, and pptx_tables.py (minimum 4 modules)
  • Each property test defines clear invariants (not just "doesn't crash")
  • Custom Hypothesis strategies generate inputs matching the YAML slide definition schema
  • All property tests pass in CI with max_examples >= 200
  • Hypothesis database directory is configured for persistence across runs
  • No unhandled exceptions are discovered (or discovered exceptions are fixed or documented)
  • Test execution completes within reasonable CI time limits (< 5 minutes for property tests)

Metadata

Metadata

Assignees

No one assigned

    Labels

    skillsCopilot skill packages (SKILL.md)testingTest infrastructure and test files

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions