-
Notifications
You must be signed in to change notification settings - Fork 715
test: Add tests to directly validate db records #9356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Failed to generate code suggestions for PR |
Contributor
Greptile OverviewGreptile Summaryadds new database validation test suite that directly queries postgres metadata database to validate data ingestion, schema creation, and tantivy indexing Key changes:
Issues found:
Confidence Score: 3/5
Important Files ChangedFile Analysis
Sequence DiagramsequenceDiagram
participant CI as GitHub Actions
participant PG as PostgreSQL Service
participant O2 as OpenObserve Server
participant PT as Pytest Tests
participant DB as Postgres DB
CI->>PG: start postgres service (port 5432)
CI->>CI: build openobserve debug binary
CI->>O2: start with ZO_META_STORE=postgres
O2->>DB: initialize schema & tables
CI->>O2: wait for healthz endpoint
O2-->>CI: ready
CI->>PT: run pytest tests
Note over PT,DB: Test: Schema Validation
PT->>O2: POST /api/{org}/{stream}/_json (ingest data)
O2->>DB: create schema entries in meta table
PT->>DB: SELECT schema FROM meta WHERE module='schema'
DB-->>PT: return schema records
PT->>PT: assert schema exists & contains fields
Note over PT,DB: Test: Data Ingestion Count
PT->>O2: POST /api/{org}/{stream}/_json (10 records)
O2->>DB: persist data & metadata
PT->>O2: POST /api/{org}/_search (query count)
O2->>DB: query data
DB-->>O2: return results
O2-->>PT: search response
PT->>PT: assert count >= 10
Note over PT,DB: Test: Organization Metadata
PT->>DB: SELECT FROM meta WHERE module='organization'
DB-->>PT: return org metadata
PT->>PT: assert org exists (BUG: >= 0 always true)
Note over PT,DB: Test: Tantivy Indexing
PT->>O2: POST /api/{org}/{stream}/_json (50 records with 'log' field)
O2->>DB: write to file_list & create tantivy indexes
PT->>PT: sleep 20 seconds (wait for persistence)
PT->>DB: SELECT FROM file_list WHERE stream='{org}/logs/{stream}'
DB-->>PT: return file records with index_size
PT->>PT: assert all files have index_size > 0
PT->>PT: assert total records == 50
PT-->>CI: test results
CI->>O2: stop server
CI->>CI: upload logs if failure
|
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
5 files reviewed, 4 comments
haohuaijin
approved these changes
Nov 28, 2025
neha00290
approved these changes
Nov 28, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pytest to directly access db, and validate test cases using db queries.