Skip to content

Conversation

@Subhra264
Copy link
Contributor

pytest to directly access db, and validate test cases using db queries.

@github-actions
Copy link
Contributor

Failed to generate code suggestions for PR

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Nov 27, 2025

Greptile Overview

Greptile Summary

adds new database validation test suite that directly queries postgres metadata database to validate data ingestion, schema creation, and tantivy indexing

Key changes:

  • new github actions workflow db-testing.yml for CI integration with postgres service
  • pytest-based test suite with fixtures for database connections and API interactions
  • tests validate schema creation, data ingestion counts, organization metadata, and tantivy index generation
  • comprehensive documentation in README for local and CI testing

Issues found:

  • logic bug at line 102: assertion len(results) >= 0 will always pass (should be > 0)
  • multiple print statements throughout tests instead of logging framework (violates custom rule)

Confidence Score: 3/5

  • PR has one critical logic bug that makes a test assertion useless, plus style violations
  • score reflects the logic bug at line 102 where len(results) >= 0 will always be true, making that test ineffective at catching failures. the test suite is otherwise well-structured with proper fixtures and good coverage, but the always-passing assertion is a significant issue that must be fixed. multiple print statements violate the custom rule about using logging frameworks.
  • pay close attention to tests/db-testing/tests/test_db_validation.py - fix the assertion logic bug at line 102

Important Files Changed

File Analysis

Filename Score Overview
.github/workflows/db-testing.yml 5/5 adds new CI workflow for database validation tests with postgres service, builds debug binary and runs pytest
tests/db-testing/tests/conftest.py 5/5 pytest fixtures for database connections, API interactions, and test data ingestion
tests/db-testing/tests/test_db_validation.py 3/5 db validation tests for schema, ingestion, metadata and tantivy indexing; contains logic bug in assertion at line 102 and uses print statements instead of logging

Sequence Diagram

sequenceDiagram
    participant CI as GitHub Actions
    participant PG as PostgreSQL Service
    participant O2 as OpenObserve Server
    participant PT as Pytest Tests
    participant DB as Postgres DB

    CI->>PG: start postgres service (port 5432)
    CI->>CI: build openobserve debug binary
    CI->>O2: start with ZO_META_STORE=postgres
    O2->>DB: initialize schema & tables
    CI->>O2: wait for healthz endpoint
    O2-->>CI: ready
    
    CI->>PT: run pytest tests
    
    Note over PT,DB: Test: Schema Validation
    PT->>O2: POST /api/{org}/{stream}/_json (ingest data)
    O2->>DB: create schema entries in meta table
    PT->>DB: SELECT schema FROM meta WHERE module='schema'
    DB-->>PT: return schema records
    PT->>PT: assert schema exists & contains fields
    
    Note over PT,DB: Test: Data Ingestion Count
    PT->>O2: POST /api/{org}/{stream}/_json (10 records)
    O2->>DB: persist data & metadata
    PT->>O2: POST /api/{org}/_search (query count)
    O2->>DB: query data
    DB-->>O2: return results
    O2-->>PT: search response
    PT->>PT: assert count >= 10
    
    Note over PT,DB: Test: Organization Metadata
    PT->>DB: SELECT FROM meta WHERE module='organization'
    DB-->>PT: return org metadata
    PT->>PT: assert org exists (BUG: >= 0 always true)
    
    Note over PT,DB: Test: Tantivy Indexing
    PT->>O2: POST /api/{org}/{stream}/_json (50 records with 'log' field)
    O2->>DB: write to file_list & create tantivy indexes
    PT->>PT: sleep 20 seconds (wait for persistence)
    PT->>DB: SELECT FROM file_list WHERE stream='{org}/logs/{stream}'
    DB-->>PT: return file records with index_size
    PT->>PT: assert all files have index_size > 0
    PT->>PT: assert total records == 50
    
    PT-->>CI: test results
    CI->>O2: stop server
    CI->>CI: upload logs if failure
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 4 comments

Edit Code Review Agent Settings | Greptile

@Subhra264 Subhra264 merged commit 0d956eb into main Nov 28, 2025
37 checks passed
@Subhra264 Subhra264 deleted the test/db_testing branch November 28, 2025 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants