Skip to content

chore: refactor integrations#1165

Merged
VaibhavUpreti merged 6 commits intomainfrom
feat/refactor_integrations
May 1, 2026
Merged

chore: refactor integrations#1165
VaibhavUpreti merged 6 commits intomainfrom
feat/refactor_integrations

Conversation

@VaibhavUpreti
Copy link
Copy Markdown
Member

refactor the integrations catalogue and module

Signed-off-by: vaibhav upreti <[email protected]>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 30, 2026

Greptile Summary

This refactoring splits catalog.py (~1800 lines) into _catalog_impl.py, extracts integration config models into config_models.py, introduces registry.py with an INTEGRATION_SPECS tuple as the single source of truth for integration metadata, adds a probes.py ProbeResult dataclass, and creates _verification_adapters.py with factory-based verifiers (build_probe_verifier/build_validation_verifier). Each service client also gains a probe_access() method. The structural split is clean, but the migration of the verification layer appears incomplete: verify.py retains its own VERIFIER_REGISTRY (with inline _verify_* implementations) that verify_integrations() still uses in production, while the new registry.VERIFIER_REGISTRY (backed by _verification_adapters.py) is validated by tests but never consumed by any production code path.

Confidence Score: 4/5

Safe to merge with awareness that the verification layer migration is incomplete — verify_integrations() still uses its own parallel registry, not the new registry.VERIFIER_REGISTRY.

All existing functionality is preserved; the split of catalog.py, extraction of config models, and addition of probe_access() methods are clean. The structural concern is that registry.VERIFIER_REGISTRY (validated by tests) is never consumed by the actual verify_integrations() production path, creating a gap between tested and running code. The env_int isdigit issue is obscure but real. No data loss or security issues were found.

app/integrations/registry.py and app/integrations/verify.py — two parallel VERIFIER_REGISTRY definitions where only verify.py's is used in production, but tests/integrations/test_registry.py validates only registry.py's. app/integrations/_relational.py env_int() isdigit bug.

Important Files Changed

Filename Overview
app/integrations/registry.py New central registry of all integration specs; builds VERIFIER_REGISTRY from INTEGRATION_SPECS via _verification_adapters.py, but this registry is never consumed by verify_integrations() in production.
app/integrations/_verification_adapters.py New factory-based verifier implementations using build_probe_verifier/build_validation_verifier; currently only reached via registry.VERIFIER_REGISTRY which no production code imports — verify.py still uses its own parallel inline verifiers.
app/integrations/verify.py Still owns its own VERIFIER_REGISTRY and inline verify* implementations; imports SUPPORTED_VERIFY_SERVICES and CORE_VERIFY_SERVICES from registry.py but continues to use its own local registry for verify_integrations().
app/integrations/_catalog_impl.py Extracted from the old catalog.py; contains all integration resolution/normalization logic; clean extraction with no apparent issues.
app/integrations/config_models.py New canonical home for all integration Pydantic config models; models.py now re-exports from here for backward compatibility.
app/integrations/probes.py New ProbeResult frozen dataclass with ok, passed(), failed(), missing() helpers; used by all the new probe_access() methods in service clients.
app/integrations/_relational.py New shared helpers for relational integrations; env_int() uses str.isdigit() which accepts Unicode superscript digits that int() cannot parse.
app/integrations/effective_models.py New strict models for resolved effective integrations (IntegrationInstance, EffectiveIntegrationEntry, EffectiveIntegrations); clean extraction.
app/services/argocd/client.py Added probe_access() using with self: for proper cleanup; imports config from config_models instead of models; clean addition.
app/services/splunk/client.py Added probe_access() delegating to validate_access(); SplunkConfig aliased to SplunkIntegrationConfig from config_models.
tests/integrations/test_registry.py Tests validate registry.VERIFIER_REGISTRY structural integrity, but this registry is not used by verify_integrations() — the test gives false confidence about production coverage.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    CLI["cli.py\ncmd_verify()"]
    VFY["verify.py\nverify_integrations()"]
    VFY_REG["verify.py\nVERIFIER_REGISTRY\n(inline _verify_* fns)"]
    REG["registry.py\nINTEGRATION_SPECS\n+ VERIFIER_REGISTRY"]
    ADAPTERS["_verification_adapters.py\nbuild_probe_verifier()\nbuild_validation_verifier()"]
    CLIENTS["Service Clients\n(probe_access())"]
    CATALOG["catalog.py → _catalog_impl.py\nclassify / resolve integrations"]
    CFGMODELS["config_models.py\nPydantic config models"]
    MODELS["models.py\n(re-exports → config_models.py)"]
    TEST["test_registry.py\nasserts VERIFIER_REGISTRY\n== SUPPORTED_VERIFY_SERVICES"]

    CLI -->|"imports SUPPORTED_SETUP_SERVICES"| REG
    CLI -->|"calls"| VFY
    VFY -->|"uses"| VFY_REG
    VFY -->|"imports SUPPORTED_VERIFY_SERVICES\nCORE_VERIFY_SERVICES"| REG
    REG -->|"builds VERIFIER_REGISTRY from"| ADAPTERS
    ADAPTERS -->|"delegates to"| CLIENTS
    CATALOG -->|"uses family_key, service_key, etc."| REG
    CFGMODELS -->|"re-exported by"| MODELS
    TEST -->|"checks registry.VERIFIER_REGISTRY"| REG

    style VFY_REG fill:#ffcccc,stroke:#cc0000
    style ADAPTERS fill:#fff3cd,stroke:#cc8800
    style REG fill:#fff3cd,stroke:#cc8800
    style TEST fill:#ffcccc,stroke:#cc0000

    subgraph gap["Gap: registry.VERIFIER_REGISTRY built but never used in production"]
        REG
        ADAPTERS
    end
Loading

Reviews (1): Last reviewed commit: "chore: refactor integrations" | Re-trigger Greptile

Comment thread app/integrations/registry.py Outdated
Comment thread app/integrations/_relational.py Outdated
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
Comment thread app/integrations/verify.py Fixed
@VaibhavUpreti VaibhavUpreti merged commit 5b338a8 into main May 1, 2026
12 checks passed
@VaibhavUpreti VaibhavUpreti deleted the feat/refactor_integrations branch May 1, 2026 00:27
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

🌊 Merged. @VaibhavUpreti is now permanently woven into git history. No take-backs. 😄


👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome.

Sarah-Salah added a commit to Sarah-Salah/opensre that referenced this pull request May 5, 2026
Resolves conflicts in auto-generated bot files: docs/daily-updates/overview.mdx, docs/daily-updates/2026-04-30.mdx, docs/daily-updates/2026-05-01.mdx, and README.md (contributors section). All four are produced by scheduled bot workflows in main and have no relationship to this PR; resolved by accepting the upstream main version verbatim.

Pulls in 35+ commits from main since this PR was opened, including: refactor of integrations module (Tracer-Cloud#1165), Splunk integration (Tracer-Cloud#791), interactive shell improvements (Tracer-Cloud#1159, Tracer-Cloud#1167), Claude Code CLI provider (Tracer-Cloud#1168), and CI quality gate restoration. None of these touch the OpenSearch wizard, detect_sources, or the validation modules this PR modifies.

Refs: Tracer-Cloud#1143
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants