feat: remove syslog #8227

hengfeiyang · 2025-09-01T05:41:30Z

User description

PR Type

Enhancement, Documentation

Description

Remove syslog feature across codebase
Clean config and routing references
Update metadata prefixes and SBOM files
Refresh i18n and OpenAPI assets

Diagram Walkthrough

flowchart LR
  SYS["Syslog feature"] -- "deprecated & removed" --> CODE["Code paths"]
  CODE -- "delete flags/routes" --> CFG["Configuration"]
  CODE -- "remove storage keys" --> META["Metadata prefixes"]
  CODE -- "update specs" --> SBOM["SBOM/XML assets"]
  CODE -- "update docs/i18n" --> WEB["Web locales & OpenAPI"]

File Walkthrough

Relevant files

Enhancement

13 files

mod.rs `Remove syslog config import from job module`	+3/-26
mod.rs `Stop glob importing config; drop syslog usage`	+1/-3
meta.rs Drop `/syslog` from metadata item prefixes	+1/-2
config.rs `Remove syslog-related configuration options`	+0/-34
config.rs `Purge syslog feature flag and defaults`	+0/-25
mod.rs `Clean references tied to syslog ingestion`	+0/-60
mod.rs `Delete syslog router endpoints`	+0/-5
mod.rs `Detach syslog handler wiring`	+0/-1
mod.rs `Remove syslog-specific log ingestion paths`	+0/-2
mod.rs `Clean DB interactions for syslog keys`	+0/-2
mod.rs `Adjust request handling without syslog`	+0/-2
mod.rs `Unexport syslog services`	+0/-2
mod.rs `Remove syslog metadata definitions`	+0/-2

Tests

2 files

integration_test.rs `Update tests to exclude syslog paths`	+0/-57
useIngestionRoutes.spec.ts `Update UI tests to remove syslog routes`	+0/-1

Documentation

20 files

openapi.rs `Remove syslog routes from OpenAPI`	+0/-5
config.cdx.xml `Update config component index (remove syslog)`	+15011/-4611
wal.cdx.xml `Update WAL component index references`	+19476/-358
openobserve.cdx.xml `Refresh SBOM root; remove syslog entries`	+14015/-7932
infra.cdx.xml `Update infra component index, no syslog`	+14127/-6086
proto.cdx.xml `Update proto index; purge syslog mentions`	+8000/-1513
ingester.cdx.xml `Adjust ingester index excluding syslog`	+15845/-4615
flight.cdx.xml `Add/refresh flight component index`	+19900/-0
hi.json `Remove syslog strings from Hindi locale`	+0/-13
fr.json `Remove syslog strings from French locale`	+0/-13
tr.json `Remove syslog strings from Turkish locale`	+0/-13
de.json `Remove syslog strings from German locale`	+0/-13
ko.json `Remove syslog strings from Korean locale`	+0/-13
it.json `Remove syslog strings from Italian locale`	+0/-13
nl.json `Remove syslog strings from Dutch locale`	+0/-13
es.json `Remove syslog strings from Spanish locale`	+0/-13
pt.json `Remove syslog strings from Portuguese locale`	+0/-13
zh.json `Remove syslog strings from Chinese locale`	+0/-13
ja.json `Remove syslog strings from Japanese locale`	+0/-13
en.json `Remove syslog strings from English locale`	+0/-13

Dependencies

1 files

Cargo.toml `Update dependencies/features after syslog removal`	+0/-3

Additional files

10 files

syslog.rs	+0/-43
websocket.rs	+0/-90
mod.rs	+0/-143
mod.rs	+0/-135
syslog_server.rs	+0/-137
report_server.cdx.xml	+20472/-0
syslog.rs	+0/-180
syslog.rs	+0/-559
syslogs_route.rs	+0/-191
syslog.ts	+0/-37

greptile-apps

Greptile Summary

This PR implements the complete removal of the syslog feature from OpenObserve v0.16.0, following its deprecation in v0.15.0. The changes systematically eliminate all syslog-related functionality across the entire codebase, including:

Backend Infrastructure Removal:

Eliminates syslog HTTP handlers (src/handler/http/request/syslog/mod.rs)
Removes TCP/UDP syslog server implementation (src/handler/tcp_udp/mod.rs, src/job/syslog_server.rs)
Deletes syslog database services and route management (src/service/db/syslog.rs, src/service/syslogs_route.rs)
Removes syslog-related configuration options and global state variables
Eliminates syslog TLS configuration functions from the TLS service layer

API and Routing Changes:

Removes 5 syslog API endpoints from OpenAPI specification and HTTP routing
Eliminates syslog route management operations (create, update, list, delete, toggle)
Removes syslog server state management endpoints

Frontend and Localization Cleanup:

Removes syslog service client (web/src/services/syslog.ts)
Eliminates syslog UI translations across 12 language files (English, Spanish, French, German, Italian, Portuguese, Dutch, Turkish, Chinese, Japanese, Korean, Hindi)
Updates test specifications to remove syslog component mocks

Configuration and Migration Updates:

Removes TCP configuration struct and validation functions
Eliminates /syslog prefix from database migration utilities
Updates status endpoint to remove syslog_enabled configuration field

Dependency Management:

Removes ipnetwork and syslog_loose dependencies from Cargo.toml
Clears CycloneDX SBOM files that contained syslog component metadata

The architectural rationale is to simplify OpenObserve by removing specialized syslog ingestion capabilities and directing users to external tools like Vector.dev and syslog-ng for syslog collection, which can then forward logs to OpenObserve via standard HTTP APIs.

Confidence score: 3/5

This PR requires careful review due to the extensive scope of changes across 35+ files and potential for breaking existing deployments
Score reflects concerns about incomplete cleanup, particularly the removal of websocket.rs which appears unrelated to syslog functionality
Multiple critical system components are being removed simultaneously, increasing the risk of runtime errors or missing dependencies

_{40 files reviewed, no comments}

_{Edit Code Review Bot Settings | Greptile}

github-actions · 2025-09-01T05:44:35Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Fix invalid package URLs The `purl` values reference local `file://` URLs which are not resolvable in CI/CD and may break consumers of the SBOM. Replace them with canonical package URLs or omit the `download_url` qualifier. src/flight/flight.cdx.xml [19-24] -<purl>pkg:cargo/[email protected]?download_url=file://.</purl> +<purl>pkg:cargo/[email protected]</purl> ... -<purl>pkg:cargo/[email protected]?download_url=file://.#src/lib.rs</purl> +<purl>pkg:cargo/[email protected]</purl> Suggestion importance[1-10]: 8 __ Why: Removing local file:// download_url qualifiers from purl improves portability and downstream consumption; the improved code accurately reflects a valid canonical purl for the shown entries.	Medium
	Fix array length mismatch The array length literal was reduced to 12 while only one entry (`"/syslog"`) was removed from the prior 13; ensure the array now has exactly 12 items. If other entries remain unchanged, verify one more item was removed or adjust the length to match the actual count to prevent a compile-time mismatch. src/migration/meta.rs [19-25] const ITEM_PREFIXES: [&str; 12] = [ "/user", "/schema", "/function", "/dashboard", // dashboard "/folders", // dashboard + // ...ensure total entries equal 12 +]; Suggestion importance[1-10]: 6 __ Why: The array size changed from 13 to 12 while removing `"/syslog"`, so ensuring the count matches is valid; impact is moderate as mismatch would be a compile error if counts don’t align.	Low
	Ensure remaining config imports Removing the glob import of `config::` may break code below if it referenced items like `SYSLOG_ENABLED` or other config constants/types. Verify all such references are removed or explicitly qualified to avoid unresolved names at compile time. If any config values are still needed in this module, import them explicitly from `infra::config`. src/handler/http/request/status/mod.rs [69-74] use crate::{ common::{ - infra::cluster, + infra::{cluster}, meta::{ http::HttpResponse as MetaHttpResponse, user::{AuthTokens, AuthTokensExt}, Suggestion importance[1-10]: 5 __ Why: The suggestion correctly notes that removing `infra::config::` could break references like `SYSLOG_ENABLED`; it’s contextually relevant but asks for verification rather than proposing a concrete fix.	Low
Security	Remove environment-specific paths Avoid embedding absolute local filesystem paths and host-specific properties in the SBOM as they leak environment details and hurt reproducibility. Replace `bom-ref` and `purl` download URLs that include `/Users/...` with repository-relative or generic identifiers, and drop host CPU triple if it’s not intended for distribution. This prevents unintended disclosure and makes the SBOM portable. src/wal/wal.cdx.xml [2-31] <bom xmlns="http://cyclonedx.org/schema/bom/1.3" serialNumber="urn:uuid:2ae312bd-f864-4254-a96b-117f13a911b1" version="1"> <metadata> <timestamp>2025-09-01T05:40:24.268857000Z</timestamp> <tools> <tool> <vendor>CycloneDX</vendor> <name>cargo-cyclonedx</name> <version>0.5.7</version> </tool> </tools> - <component type="library" bom-ref="path+file:///Users/hengfeiyang/code/rust/github.com/openobserve/openobserve/src/wal#0.1.0"> + <component type="library" bom-ref="pkg:cargo/[email protected]"> <name>wal</name> <version>0.1.0</version> <scope>required</scope> <licenses> <expression>AGPL-3.0</expression> </licenses> - <purl>pkg:cargo/[email protected]?download_url=file://.</purl> + <purl>pkg:cargo/[email protected]</purl> <components> - <component type="library" bom-ref="path+file:///Users/hengfeiyang/code/rust/github.com/openobserve/openobserve/src/wal#0.1.0 bin-target-0"> + <component type="library" bom-ref="pkg:cargo/[email protected]#bin-target-0"> <name>wal</name> <version>0.1.0</version> - <purl>pkg:cargo/[email protected]?download_url=file://.#src/lib.rs</purl> + <purl>pkg:cargo/[email protected]#src/lib.rs</purl> </component> </components> </component> - <properties> - <property name="cdx:rustc:sbom:target:triple">aarch64-apple-darwin</property> - </properties> + <!-- Removed host-specific property to avoid leaking build environment --> </metadata> Suggestion importance[1-10]: 7 __ Why: Correctly identifies hardcoded local paths and host-specific metadata in the new SBOM that harm portability and leak environment details; proposed rewrite is sensible though may depend on SBOM generation tooling and policy.	Medium
	Remove sensitive local paths The SBOM embeds absolute local file paths and a highly precise build timestamp, which can leak developer environment details and cause non-reproducible diffs. Replace absolute paths with repository-relative or opaque identifiers and round or omit the timestamp to a stable value. src/flight/flight.cdx.xml [2-26] <bom xmlns="http://cyclonedx.org/schema/bom/1.3" serialNumber="urn:uuid:2945ac2d-6aab-4267-9c02-43e70b19f253" version="1"> <metadata> - <timestamp>2025-09-01T05:40:24.293049000Z</timestamp> + <timestamp>2025-09-01T00:00:00Z</timestamp> ... - <component type="library" bom-ref="path+file:///Users/hengfeiyang/code/rust/github.com/openobserve/openobserve/src/flight#0.1.0"> + <component type="library" bom-ref="pkg:generic/[email protected]"> <name>flight</name> <version>0.1.0</version> ... <components> - <component type="library" bom-ref="path+file:///Users/hengfeiyang/code/rust/github.com/openobserve/openobserve/src/flight#0.1.0 bin-target-0"> + <component type="library" bom-ref="pkg:generic/[email protected]#bin-target-0"> <name>flight</name> <version>0.1.0</version> - <purl>pkg:cargo/[email protected]?download_url=file://.#src/lib.rs</purl> + <purl>pkg:cargo/[email protected]</purl> </component> </components> </component> ... </metadata> ... </bom> Suggestion importance[1-10]: 7 __ Why: Replacing absolute local paths and precise timestamps reduces environment leakage and improves reproducibility, but the proposed bom-ref/purl transformations may not conform to the generator’s expected identifiers and could break SBOM tooling.	Medium
	Pin VCS references immutably The `bom-ref` and `purl` include a floating `rev` query that may not guarantee immutability across regenerations. Pin to a commit SHA in both `bom-ref` and `purl` without additional query parameters to ensure SBOM determinism and accurate provenance. src/wal/wal.cdx.xml [33-51] -<component type="library" bom-ref="git+https://github.com/mattsse/chromiumoxide?rev=6f2392f78ae851e2acf33df8e9764cc299d837db#0.7.0"> +<component type="library" bom-ref="pkg:cargo/[email protected]#6f2392f78ae851e2acf33df8e9764cc299d837db"> <author>Matthias Seitz <[email protected]></author> <name>chromiumoxide</name> <version>0.7.0</version> <description>Library for interacting with a chrome instance with the chrome devtools protocol</description> <scope>required</scope> <licenses> <expression>MIT OR Apache-2.0</expression> </licenses> - <purl>pkg:cargo/[email protected]?vcs_url=git%2Bhttps://github.com/mattsse/chromiumoxide%3Frev=6f2392f78ae851e2acf33df8e9764cc299d837db%406f2392f78ae851e2acf33df8e9764cc299d837db</purl> + <purl>pkg:git/github.com/mattsse/chromiumoxide@6f2392f78ae851e2acf33df8e9764cc299d837db</purl> <externalReferences> <reference type="website"> <url>https://github.com/mattsse/chromiumoxide</url> </reference> <reference type="vcs"> <url>https://github.com/mattsse/chromiumoxide</url> </reference> </externalReferences> </component> Suggestion importance[1-10]: 5 __ Why: The concern about reproducibility is valid, but the improved code changes identifier schemes (bom-ref/purl formats) beyond merely pinning and may not conform to CycloneDX cargo conventions; partially correct with limited impact.	Low
General	Drop excluded components entries Components marked `scope="excluded"` should generally not appear under the top-level in a distributed SBOM. Either remove excluded components or ensure they are placed under appropriate metadata so consumers don’t misinterpret them as shipped dependencies. src/wal/wal.cdx.xml [1305-1320] -<component type="library" bom-ref="registry+https://github.com/rust-lang/crates.io-index#[email protected]"> - <name>autocfg</name> - <version>1.4.0</version> - <description>Automatic cfg for Rust compiler features</description> - <scope>excluded</scope> - <hashes> - <hash alg="SHA-256">ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26</hash> - </hashes> - <licenses> - <expression>Apache-2.0 OR MIT</expression> - </licenses> - <purl>pkg:cargo/[email protected]</purl> - <externalReferences> - <reference type="documentation"> - <url>https://docs.rs/autocfg/</url> - </reference> - <reference type="website"> - <url>https://github.com/cuviper/autocfg</url> - </reference> - <reference type="vcs"> - <url>https://github.com/cuviper/autocfg</url> - </reference> - </externalReferences> -</component> +<!-- Removed excluded component from the shipped components list to avoid confusion --> Suggestion importance[1-10]: 6 __ Why: Targets the `autocfg` component marked `scope="excluded"` present in the new hunk and suggests removing it to avoid confusion; impact is moderate and policy-dependent, but the observation is accurate.	Low
	Remove stale syslog references After dropping the import of `infra::config::SYSLOG_ENABLED`, ensure no remaining code in this module references `SYSLOG_ENABLED` or other syslog-related config. If such references exist, remove or guard them to avoid unresolved names and feature drift. src/job/mod.rs [25-30] use crate::{ common::meta::{ organization::DEFAULT_ORG, user::{UserOrgRole, UserRequest}, }, service::{db, self_reporting, users}, }; +// Ensure no references to removed syslog configuration remain in this module. Suggestion importance[1-10]: 5 __ Why: It’s reasonable to verify no remaining references to `SYSLOG_ENABLED` after its removal; however, it’s a general check without specific broken lines, so the impact is moderate.	Low
	Update CycloneDX schema version The BOM uses CycloneDX 1.3 namespace; newer tooling and policies often require 1.5+ with tightened schemas and fields. Consider updating the namespace to a supported schema version used across the repo to prevent validation failures. src/flight/flight.cdx.xml [2] -<bom xmlns="http://cyclonedx.org/schema/bom/1.3" serialNumber="urn:uuid:2945ac2d-6aab-4267-9c02-43e70b19f253" version="1"> +<bom xmlns="http://cyclonedx.org/schema/bom/1.5" serialNumber="urn:uuid:2945ac2d-6aab-4267-9c02-43e70b19f253" version="1"> Suggestion importance[1-10]: 4 __ Why: Using a newer schema can improve validation, but changing the namespace without verifying tool support is speculative and not required by the diff; it risks incompatibility with the producing tool.	Low

hengfeiyang added 2 commits September 1, 2025 13:40

feat: remove syslog

3c3adf2

feat: update SBOM

3294d67

hengfeiyang requested review from bjp232004 and oasisk September 1, 2025 05:41

github-actions bot added ✏️ Feature 🧹 Updates labels Sep 1, 2025

greptile-apps bot reviewed Sep 1, 2025

View reviewed changes

bjp232004 approved these changes Sep 1, 2025

View reviewed changes

hengfeiyang added the breakingChange breaks contract label Sep 1, 2025

oasisk approved these changes Sep 1, 2025

View reviewed changes

hengfeiyang merged commit 1989432 into main Sep 1, 2025
30 checks passed

hengfeiyang deleted the feat/remove-syslog branch September 1, 2025 06:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: remove syslog #8227

feat: remove syslog #8227

Uh oh!

hengfeiyang commented Sep 1, 2025 •

edited by github-actions bot

Loading

Uh oh!

greptile-apps bot left a comment

Uh oh!

github-actions bot commented Sep 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: remove syslog #8227

feat: remove syslog #8227

Uh oh!

Conversation

hengfeiyang commented Sep 1, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Summary

Confidence score: 3/5

Uh oh!

github-actions bot commented Sep 1, 2025

PR Code Suggestions ✨

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hengfeiyang commented Sep 1, 2025 •

edited by github-actions bot

Loading