Skip to content

feat(integrations): add VictoriaLogs integration#1144

Merged
VaibhavUpreti merged 8 commits intoTracer-Cloud:mainfrom
yashksaini-coder:feat/126-victoria-logs
May 2, 2026
Merged

feat(integrations): add VictoriaLogs integration#1144
VaibhavUpreti merged 8 commits intoTracer-Cloud:mainfrom
yashksaini-coder:feat/126-victoria-logs

Conversation

@yashksaini-coder
Copy link
Copy Markdown
Collaborator

Closes #126.

Adds a first-class VictoriaLogs integration: config model, catalog wiring (DB-store + env-var paths), verifier, sync HTTP client, LangGraph tool, docs, and tests.

Scope

Strictly limited to the integration. No edits to app/nodes/extract_alert/ or any prompt files — this PR scopes to integration plumbing only.

What's in

Component File
Config model app/integrations/models.py (VictoriaLogsIntegrationConfig, EffectiveIntegrations.victoria_logs)
Catalog wiring app/integrations/catalog.py (service key map, classifier, env loader, direct_services)
Verifier app/integrations/verify.py (_verify_victoria_logs, SUPPORTED_VERIFY_SERVICES, dispatch)
HTTP client app/services/victoria_logs/{__init__.py, client.py}
Tool app/tools/VictoriaLogsTool/__init__.py
Evidence type app/types/evidence.py
Docs docs/victoria_logs.mdx
Tests tests/integrations/test_victoria_logs.py, tests/tools/test_victoria_logs_tool.py

How prior P1/P2 review feedback is addressed

This PR was rebuilt from main after PRs #663 and #1060 were closed. Each issue raised on those PRs is addressed below.

extract_params / run executor-contract mismatch (P1, #663 + #1060)

Prior PRs read credentials from kwargs["sources"] inside run(), but the executor calls tools as tool.run(**tool.extract_params(sources)) with no sources kwarg, so base_url was always empty. Fixed by mirroring AlertmanagerAlertsTool: extract_params() returns every kwarg run() declares, and run() accepts them as explicit parameters. A unit test TestExecutorPathContract::test_run_via_extract_params_succeeds invokes the tool exactly as the executor does, so any future drift fails at unit-test time.

Verifier returned status=\"verified\" instead of \"passed\" (P1, #1060)

verification_exit_code() checks status == \"passed\". _verify_victoria_logs now returns \"passed\" / \"missing\" / \"failed\" consistent with every other verifier in verify.py.

victoria_logs missing from EffectiveIntegrations (P1, #663 + #1060)

EffectiveIntegrations uses extra=\"forbid\". victoria_logs: EffectiveIntegrationEntry | None = None declared. A test (test_effective_integrations_field_exists) re-validates the dump shape to guard against regression.

victoria_logs absent from direct_services and load_env_integrations() (P1, #663)

Both added in catalog.py. _classify_service_instance handles the DB-store path; load_env_integrations reads VICTORIA_LOGS_URL and optional VICTORIA_LOGS_TENANT_ID.

tenant_id defaulted to \"0\" and always sent AccountID (P2, #1060)

tenant_id: str | None = None with a validator collapsing None, \"\", and whitespace to None. The AccountID header is sent only when tenant_id is explicitly configured. Docs reflect this — no VICTORIA_LOGS_TENANT_ID=\"0\" example.

import contextlib inside the loop (P2, #1060)

Moved to module level in client.py.

\"victoria_logs\" not in EvidenceSource Literal (P1, #1060)

Added. BaseTool.__init_subclass__() validates this at import time.

Unrelated extraction-prompt edits (P1, #1060)

Not touched in this PR.

Configuration

VICTORIA_LOGS_URL=http://vmlogs:9428
# Optional, multi-tenant only:
# VICTORIA_LOGS_TENANT_ID=acme

Quality gates

  • ruff check — clean on all 9 changed files
  • ruff format --check — clean
  • mypy — no issues on changed files
  • pytest tests/integrations/test_victoria_logs.py tests/tools/test_victoria_logs_tool.py — 38/38 passed
  • pytest tests/integrations/ tests/tools/ tests/types/ — 1972 passed, 1 xfailed (no regressions)

Copilot AI review requested due to automatic review settings April 30, 2026 12:47
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 30, 2026

Greptile Summary

This PR adds a complete VictoriaLogs integration: Pydantic config model, catalog wiring (DB-store + env-var paths), verifier, synchronous HTTP client, VictoriaLogsTool, evidence type, docs, and 38 unit tests. The config, verification, and catalog layers are all correctly implemented and address every issue raised in the prior PR reviews.

  • P1 — detect_sources.py not updated: Every other tool-backed integration (alertmanager, opsgenie, etc.) has an explicit block in app/nodes/plan_actions/detect_sources.py that copies connection credentials into available_sources. No such block exists for victoria_logs, so VictoriaLogsTool.is_available() always returns False and the tool is never scheduled during an investigation — the integration is silently inert end-to-end.

Confidence Score: 4/5

Safe to review but not ready to merge — one P1 wiring gap makes the tool silently non-functional in production.

All the integration plumbing layers (catalog, models, verify, client, evidence type, docs, tests) are well-implemented and address every issue from prior reviews. The single blocker is the missing detect_sources.py block that prevents victoria_logs from ever appearing in available_sources, making is_available() always return False and the tool perpetually unscheduled. Once that one-function addition is made, the PR is ready.

app/nodes/plan_actions/detect_sources.py (missing victoria_logs wiring) and app/tools/VictoriaLogsTool/init.py (extract_params query defaulting)

Important Files Changed

Filename Overview
app/integrations/catalog.py Adds victoria_logs to service key map, _classify_service_instance, load_env_integrations, and direct_services — all four catalog touch points are correctly wired.
app/integrations/models.py Adds VictoriaLogsIntegrationConfig (with base_url/tenant_id normalization) and victoria_logs field to EffectiveIntegrations — correct and safe.
app/integrations/verify.py Adds _verify_victoria_logs returning passed/missing/failed consistent with the contract; dispatched correctly in the main verify_integrations switch.
app/services/victoria_logs/client.py Clean synchronous client with context-manager lifecycle and NDJSON parser; VictoriaLogsConfig is a near-duplicate of VictoriaLogsIntegrationConfig in models.py — minor DRY concern.
app/tools/VictoriaLogsTool/init.py is_available and extract_params depend on sources["victoria_logs"] being populated by detect_sources.py, which is never updated; the tool will never be selected during an investigation.
app/types/evidence.py Adds "victoria_logs" to the EvidenceSource literal — correct and required for BaseTool.__init_subclass__ validation.
tests/integrations/test_victoria_logs.py Thorough unit coverage of config normalization, env loading, DB classification, and verifier status contract; all 38 assertions focus on the right invariants.
tests/tools/test_victoria_logs_tool.py Covers executor-path contract, availability, extract_params, and run; the contract test injects query into sources artificially — production sources never include that key, so the wildcard default is what actually runs.
docs/victoria_logs.mdx Accurate env-var and JSON store setup docs; troubleshooting table is useful; no issues found.
app/services/victoria_logs/init.py Thin __init__.py that re-exports the public client surface; correct and complete.

Sequence Diagram

sequenceDiagram
    participant ENV as Env / DB Store
    participant Catalog as catalog.py
    participant DetectSrc as detect_sources.py
    participant PlanActions as plan_actions.py
    participant Executor as execute_actions.py
    participant Tool as VictoriaLogsTool
    participant Client as VictoriaLogsClient

    ENV->>Catalog: VICTORIA_LOGS_URL / DB record
    Catalog->>Catalog: load_env_integrations() / _classify_service_instance()
    Catalog->>Catalog: resolve_effective_integrations() → EffectiveIntegrations.victoria_logs ✓

    Note over DetectSrc: ⚠️ Missing block for victoria_logs
    DetectSrc-->>PlanActions: available_sources (no victoria_logs key)

    PlanActions->>Tool: is_available(sources) → False ✗
    Note over PlanActions: Tool not scheduled

    alt If detect_sources.py is fixed
        DetectSrc->>PlanActions: available_sources[victoria_logs] = {base_url, tenant_id}
        PlanActions->>Tool: is_available(sources) → True ✓
        Executor->>Tool: run(**extract_params(sources))
        Tool->>Client: make_victoria_logs_client(base_url, tenant_id)
        Client->>Client: GET /select/logsql/query?query=*&limit=50&start=-1h
        Client-->>Tool: {success, rows, total}
        Tool-->>Executor: {source, available, rows, total}
    end
Loading

Comments Outside Diff (3)

  1. app/nodes/plan_actions/detect_sources.py, line 1315 (link)

    P1 victoria_logs never wired into available_sources

    detect_sources.py builds the available_sources dict that the investigation executor and is_available() checks read. Every other integration that exposes a tool — including alertmanager (lines 1219–1236), opsgenie, jira, etc. — has an explicit block here that populates sources["<service>"]. victoria_logs has no such block, so available_sources never contains a "victoria_logs" key.

    Consequences:

    • VictoriaLogsTool.is_available(sources) always returns False → the tool is never included in the investigation plan.
    • Even if it somehow reached extract_params(sources), sources.get("victoria_logs", {}) returns {}, so base_url is always "" and the client returns an error immediately.

    The fix mirrors the alertmanager block that already exists in this file:

    victoria_logs_int = (resolved_integrations or {}).get("victoria_logs")
    if victoria_logs_int and str(victoria_logs_int.get("base_url", "")).strip():
        sources["victoria_logs"] = {
            "base_url": str(victoria_logs_int.get("base_url", "")).strip().rstrip("/"),
            "tenant_id": victoria_logs_int.get("tenant_id"),
            "connection_verified": True,
        }

    Without this change the entire integration is silently inert in production.

  2. app/tools/VictoriaLogsTool/__init__.py, line 461-468 (link)

    P2 extract_params reads query/limit/start from integration config that never has those keys

    In production, sources["victoria_logs"] contains only base_url, tenant_id, and connection_verified (once the detect_sources.py gap is fixed). It never has query, limit, or start, so config.get("query") is always None and the query will always be the wildcard "*" regardless of what context the investigation has.

    The input_schema marks query as required, which implies the LLM is expected to supply it, but through the executor path (run(**extract_params(sources))), the LLM's chosen query is never forwarded.

    The AlertmanagerAlertsTool pattern for operational params is to hard-code defaults (e.g., active=True), not to read them from sources. If a targeted query is important, consider either accepting that the tool always runs a wildcard (and removing query from required) or documenting that query customisation is not available through the executor path.

  3. app/services/victoria_logs/client.py, line 238-257 (link)

    P2 Duplicate config model — VictoriaLogsConfig vs VictoriaLogsIntegrationConfig

    VictoriaLogsConfig (here) and VictoriaLogsIntegrationConfig in models.py are structurally identical — same fields (base_url, tenant_id, integration_id), same validators, same normalization logic. The only difference is the headers property on this class. Any future change to the normalization rules must be applied in both places.

    Other integrations in the codebase (e.g., Alertmanager) follow the same two-class pattern, so this is consistent, but it's worth noting for maintainability. Consider whether VictoriaLogsIntegrationConfig could simply extend VictoriaLogsConfig or re-use it directly to stay DRY.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "feat(integrations): add VictoriaLogs int..." | Re-trigger Greptile

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a first-class VictoriaLogs integration to the integrations/catalog/tooling stack so investigations can query LogsQL and the CLI can verify connectivity.

Changes:

  • Introduces VictoriaLogs integration config + catalog wiring (DB-store classification and env var loading).
  • Adds VictoriaLogs verifier and a synchronous httpx client for /select/logsql/query.
  • Registers a new victoria_logs_query investigation tool, updates evidence source typing, and adds docs + tests.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
app/integrations/models.py Adds VictoriaLogsIntegrationConfig and exposes victoria_logs on EffectiveIntegrations.
app/integrations/catalog.py Adds service key mapping/classification + env loading + direct service resolution for victoria_logs.
app/integrations/verify.py Adds _verify_victoria_logs and wires it into the verify dispatch.
app/services/victoria_logs/client.py Implements VictoriaLogsClient + config normalization + NDJSON parsing.
app/services/victoria_logs/__init__.py Exports the VictoriaLogs client/config factory.
app/tools/VictoriaLogsTool/__init__.py Adds VictoriaLogsTool with extract_params/run executor contract support.
app/types/evidence.py Extends EvidenceSource with "victoria_logs".
docs/victoria_logs.mdx Adds end-user documentation for setup/verify/troubleshooting.
tests/integrations/test_victoria_logs.py Adds unit tests for config normalization, env loading, classification, and verifier status contract.
tests/tools/test_victoria_logs_tool.py Adds tool contract + executor-path regression tests for extract_params/run.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/victoria_logs.mdx
Comment on lines +77 to +81
| Symptom | Fix |
| --- | --- |
| **Status: missing** | Set `VICTORIA_LOGS_URL` or add a `victoria_logs` entry to your integrations store |
| **Connection refused** | Verify the URL is reachable from this host; check firewall rules |
| **404 on `/select/logsql/query`** | Confirm you are pointing at VictoriaLogs (not VictoriaMetrics for time-series); they expose different endpoints |
Comment on lines +45 to +66
"tenant_id": {
"type": "string",
"default": "",
"description": "Optional VictoriaLogs tenant ID; sent as the AccountID header.",
},
"query": {
"type": "string",
"description": "LogsQL query string (e.g. `_stream_id:* AND error`).",
},
"limit": {
"type": "integer",
"default": 50,
"description": "Maximum number of log entries to return.",
},
"start": {
"type": "string",
"default": "-1h",
"description": "Time range expression accepted by VictoriaLogs (e.g. -1h, -24h).",
},
},
"required": ["base_url", "query"],
}
Comment on lines +112 to +120
def test_alias_victorialogs_normalizes_to_victoria_logs(self) -> None:
# The service key map registers both spellings — confirm classification
# under the canonical key still works when callers feed in a record
# whose service field reaches this function pre-normalized.
config, key = _classify_service_instance(
key="victoria_logs",
credentials={"base_url": "http://vmlogs:9428", "tenant_id": "team-a"},
record_id="vl-team-a",
)
Comment thread docs/victoria_logs.mdx
Comment on lines +28 to +32
| Variable | Default | Description |
| --- | --- | --- |
| `VICTORIA_LOGS_URL` | — | **Required.** Base URL of your VictoriaLogs instance |
| `VICTORIA_LOGS_TENANT_ID` | _(unset)_ | Optional. Sent as the `AccountID` header on multi-tenant deployments |

@yashksaini-coder
Copy link
Copy Markdown
Collaborator Author

Addressed bot review feedback in commit dacd6786.

Fixed:

  • Greptile P1 — detect_sources.py not wired (silently inert tool). Added a victoria_logs block in app/nodes/plan_actions/detect_sources.py mirroring the alertmanager pattern. Without this, is_available() always returned False and the tool was never scheduled. New regression test in tests/nodes/plan_actions/test_detect_sources_victoria_logs.py (4 cases) guards against future drift.
  • Copilot — query schema/runtime mismatch. Removed query from input_schema.required and added "default": "*" to match what extract_params actually produces. Updated test_input_schema_declares_query_with_default to assert the new contract, plus an inline comment explaining why customised queries through the executor path are a known follow-up (Greptile P2).
  • Copilot — misleading test name. Renamed test_alias_victorialogs_normalizes_to_victoria_logs to test_classifies_with_tenant_id since the test exercises the tenant-id classification path, not alias normalization. Records reaching _classify_service_instance are already normalized through the service-key map.

Not changed (false positives):

  • Copilot's two || markdown table comments — the file already uses single-pipe rows (grep -n "^|" docs/victoria_logs.mdx confirmed).
  • Greptile P2 — duplicate config VictoriaLogsConfig vs VictoriaLogsIntegrationConfig. This dual-class pattern is the existing convention for every service in the repo (see AlertmanagerConfig vs AlertmanagerIntegrationConfig); collapsing them here would be inconsistent.

Quality gates:

  • ruff check / ruff format clean
  • mypy clean on changed files
  • pytest — 42 VictoriaLogs tests pass (38 prior + 4 new detect_sources); broader 2151-test suite green, no regressions

@yashksaini-coder yashksaini-coder force-pushed the feat/126-victoria-logs branch from dacd678 to 124dc30 Compare May 1, 2026 11:09
@muddlebee
Copy link
Copy Markdown
Collaborator

@yashksaini-coder can you pls share an e2e video demo of the full integration? will be helpful as its the acceptance criteria for most such integration..

Comment thread app/tools/VictoriaLogsTool/__init__.py
@yashksaini-coder
Copy link
Copy Markdown
Collaborator Author

yashksaini-coder commented May 2, 2026

@muddlebee Here's the demo video, sorry it took some time, was shifting OS

demo-tests.mp4

Copy link
Copy Markdown
Member

@VaibhavUpreti VaibhavUpreti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @yashksaini-coder !

@VaibhavUpreti VaibhavUpreti merged commit 311be2e into Tracer-Cloud:main May 2, 2026
10 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 2, 2026

🎊 Achievement unlocked: PR Merged. @yashksaini-coder passed code review, survived CI, and shipped. Respect. 🤝


👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome.

@yashksaini-coder yashksaini-coder deleted the feat/126-victoria-logs branch May 3, 2026 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Victoria logs integration

4 participants