Repository navigation
fix(telemetry): the first-data wait also finishes on the picked key's first data, for logs that name no service - #7884
Merged
Conversation
… host, port, TLS and paths Logs > Send your first logs (and the Logs Setup Guide) handed out FluentBit and Fluentd files that could not deliver. The FluentBit file put the server's host WITH its port in Host, fixed Port 443 and Tls On, and posted to /v1/logs, which no OneUptime server serves. The 14.0.110 newcomer walk (#7733) ran fluent/fluent-bit:3.2.10 with it against a self-hosted server at http://localhost:18811: "Misformatted domain name", then, with Host, Port and Tls corrected by hand, HTTP 404. The Fluentd file had the same mistakes: https fixed and /v1/logs. The Fluent Bit Docker Compose also set FLB_ES_HOST, an Elasticsearch output's variable nothing in the file reads. The files are now built from the address the Dashboard is served on (HOST and HTTP_PROTOCOL), as the "already using another tool?" guides' files are: - LogShipperSnippets.ts (new, React-free like CompatibleIngestGuide.ts) builds fluent-bit.conf, fluentd.conf and both Docker Compose files. Fluent Bit gets the host alone, the port from the address (80 or 443 when it names none), Tls On only for https, and Logs_uri /otlp/v1/logs: the App's OTLP logs route under the /otlp of the OTLP endpoint the guide's first step shows, which both of the ingress's server blocks proxy as it is. Fluentd posts to <scheme>://<host>/fluentd/logs, the address the ingress rewrites to the App's /fluentd/v1/logs and the one the Fluentd docs page gives, in every language. - CompatibleIngestGuide.ts: getFluentBitHost() is the host as Fluent Bit takes it. Fluent Bit resolves Host as written, so a bracketed IPv6 host ("[::1]") fails as "Misformatted domain name" where "::1" connects (checked with 3.2.10); the Loki guide's Fluent Bit output uses it too. Datadog's host:port keeps its brackets. - Documentation.tsx: the FluentBit and Fluentd steps render the built files (still through withServiceName); the four placeholder snippets are gone. A path prefix in the address cannot happen: the Dashboard parses HOST with Hostname.fromString at load, which refuses a "/". Tests: LogShipperSnippets.test.ts pins both files for a self-hosted http://host:port server and an https server with no port, and the address shapes in between (a bare name, a custom https port, IPv4, IPv6 with and without a port), and checks the paths against the App's routes and the bundled ingress template. TelemetryLogShipperAddress.test.tsx renders the logs guide on both kinds of server and checks the FluentBit and Fluentd steps show those files with the picked key. CompatibleIngestGuide.test.ts covers the IPv6 host; ResourceOnboarding.test.ts counts the builder sites that go through withServiceName; the nginx address list now holds both paths, so TelemetryIngestRouting checks them against the template. Checked for real, with the bundled nginx template in front of the App: fluent/fluent-bit 3.2.10 and latest (5.2.0), and fluent/fluentd 1.20.1, ran the generated files against http://localhost:17849 (billing off) and https://oneuptime.test on 443 (billing on, TLS): every run answered 200 and its lines were stored. main's files failed there as the walk saw. (Fluentd was told to read the file: the Compose mounts it at a name the image does not read, which the PR lists as a follow-up.) Closes #7753 Co-Authored-By: Claude Opus 5.5 <[email protected]>
… first data, for logs that name no service Logs > Send your first logs (and the Logs Setup Guide) ended with "Wait for the first data", which only watched for a new service. FluentBit's logs, as the page configures them, carry no service.name, and neither does OTLP sent without one: ingest files them under no service, so the step said "Waiting for the first data" for as long as the newcomer waited, though the logs were stored. The 14.0.110 newcomer walk (#7733) was stuck there until it sent one OTLP log with service.name set. The step now also watches the ingestion key the guide's snippets carry. Every HTTP ingest route stamps the key's lastUsedAt when it admits a batch (TelemetryIngest -> TelemetryIngestionKeyService.markUsed), with or without a service, so the key's first data is the one signal every way the page offers shares: FluentBit, Fluentd, OTLP without service.name, and the Loki, Splunk HEC, Datadog, Prometheus and Sentry guides. - ResourceOnboarding.ts: hasKeySentFirstData() - the key had sent nothing when the step first looked at it, and now has. Only a key's first data counts: ingest stamps lastUsedAt at most once every five minutes per process, so a key something already sends with moves on its own and cannot say whether the new shipper sent anything. Such a key leaves the step to the service check, as before. Server times only, as the rest of the watch. The new sentence: "The first data arrived with the key {{name}}. It shows up within a minute or two.", in all 18 locales. - ResourceConnectionWatch.tsx: an optional ingestionKey (plus dataPage, onDataArrived, onView). Each key's lastUsedAt is noted when the step first looks at it; the key's first data finishes the step with a link to the product's explorer. The services are still checked, and one that registers afterwards takes over with its own link. Asks about the key no more once its data is in. - Documentation.tsx: the Setup Guide tab's wait watches the picked key and links to the explorer (SETUP_GUIDE_FIRST_DATA_VIEWS.dataPage); onSelectedKeyChange reports the picked key to a dialog that draws its own wait. - TelemetryFirstRunShowcase.tsx: the Send your first ... dialog's wait watches that key too (TelemetryFirstRun.dataPage). Its link is to the product's explorer (for Logs, the page the dialog was opened over), and following it closes the dialog; once the first data arrived, closing the dialog draws the explorer afresh, so the logs are there without the reload the walk needed. - TelemetryGuidedSetup.tsx: unchanged in behaviour, with a note why. Its step 2 sets up alerts on the service that reported, and every way its guide offers names one (the OpenTelemetry snippets set service.name, the Sentry snippet's release names my-service). Tests: App's ResourceOnboarding.test.ts pins hasKeySentFirstData. Common's ResourceOnboarding.test.tsx renders the Logs Setup Guide with FluentBit picked: its first batch stamps the key and registers no service, and the step says the data arrived and links to the Logs explorer; a key that was sending already waits for a service; a service registering afterwards takes over; the key stops being asked about; each key's first look is its own. ProductOnboarding.test.tsx runs the Send your first logs dialog through the key's first data: the explanation gives way, and View logs closes the dialog over the explorer, which mounts again (and a dialog closed before any data leaves it alone). SetupGuideFirstData.test.tsx checks every type's tab passes the picked key and its explorer; TelemetryGuidedSetup.test.tsx that the guided setup waits for a service. Checked for real on a local stack (the App from this branch behind the bundled nginx template, http://localhost:17849): a new owner's new "Logs key" read as the watch reads it had no lastUsedAt; fluent/fluent-bit:3.2.10 ran the page's fluent-bit.conf, its first batch answered 200 and stamped the key, and the watch's own request read the stamp on its next poll, three seconds later, while the project had no service at all. The two lines were stored under no service (primaryEntityType Unknown). Builds on #7849 (#7753's FluentBit and Fluentd files). Closes #7754 Co-Authored-By: Claude Opus 5.5 <[email protected]>
Contributor
Author
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #7849 (#7753's FluentBit and Fluentd files). This branch starts at that PR's head, so its diff shows #7849's commit too until #7849 is merged. The commit for #7754 is the last one.
What was wrong
Logs > Send your first logs (and the Logs Setup Guide tab) ends with "Wait for the first data". The wait was
<ResourceConnectionWatch modelType={Service} />, which watches for a new service. FluentBit's logs, as the page configures them, carry noservice.name, and neither does OTLP sent without one. Ingest files such data under no service (primaryEntityTypeUnknown), so no service registers. The step kept saying "Waiting for the first data. This step updates by itself." for as long as the newcomer waited, though the logs were stored.The 14.0.110 newcomer walk (#7733) was stuck there. It got past only by sending one OTLP log with
service.nameset.Which fix, and why
The issue offered two fixes. This PR takes the first: the step finishes on the first data the picked key sends, with or without a service.
Every HTTP ingest route stamps the key's
lastUsedAtwhen it admits a batch (TelemetryIngest→TelemetryIngestionKeyService.markUsed), whatever the batch names. That makes the key's first data the one signal that all of a newcomer's ways to send on that page share:service.nameFluentd(the server's default,FluentLogsIngestService.DEFAULT_SERVICE_NAME)FluentdserviceFluentdservice once it registersservice.nameThe second fix, configs that set a service name, would only cover the files the page writes (FluentBit; Fluentd is already named by the server). It would not cover OTLP without
service.name, a compatible tool's data that names none, or a config whose name was edited out.Only a key's first data counts. Ingest stamps
lastUsedAtat most once every five minutes per process. So the stamp of a key that something already sends with moves on by itself, and cannot tell whether the shipper just set up sent anything. For such a key the step waits for a service, as before.A newcomer's key has never been used (the guide offers to create one, named after the guide, such as "Logs key"), so their first data always counts. The rule reads only the server's stamp, as the rest of the watch does; the browser's clock is never used.
The guided telemetry setup (
/service/setup) is unchanged, deliberately. Its step 2 sets up alerts on the service that reported, so it needs a service. Every way its guide offers names one: the OpenTelemetry snippets setservice.name, and the Sentry snippet'srelease: "[email protected]"namesmy-service. A code note there says so, and a test pins it.The change
ResourceOnboarding/ResourceOnboarding.tshasKeySentFirstData()is the rule above. Adds the new sentenceRESOURCE_CONNECTION_DATA_ARRIVED: "The first data arrived with the key {{name}}. It shows up within a minute or two."ResourceOnboarding/ResourceConnectionWatch.tsxingestionKey,dataPage,onDataArrivedandonView. The first look at each key notes itslastUsedAt(kept per key, so going back to a key keeps its first look). The key's first data finishes the step with a link to the product's explorer. Services are still checked, and one that registers afterwards takes over with its own link. Once its data is in, the key is not asked about again.Telemetry/Documentation.tsxSETUP_GUIDE_FIRST_DATA_VIEWS[...].dataPage). A newonSelectedKeyChangeprop reports the picked key to a dialog that draws its own wait.ProductOnboarding/TelemetryFirstRunShowcase.tsxTelemetryFirstRun.dataPage). Its link goes to the product's explorer, which for Logs is the page the dialog was opened over, so following it closes the dialog. Once the first data has arrived, closing the dialog draws the explorer afresh, so the logs are there without the reload the walk needed.GuidedSetup/Telemetry/TelemetryGuidedSetup.tsxLocales/*.json(18)i18n:checkreports every locale file sound and no translation lost.Tests
Tests/Dashboard/ResourceOnboarding.test.tspinshasKeySentFirstData. A never-used key's first stamp counts. No stamp, or an unreadable one, does not. A key that had a stamp when the step looked does not count, however far the stamp moves.Tests/App/Dashboard/ResourceOnboarding.test.tsxgets a new describe, a telemetry guide's last step, for data that names no service:/dashboard/<project>/logs.Tests/App/Dashboard/ProductOnboarding.test.tsxruns the Send your first logs dialog through the key's first data. The explanation gives way. View logs closes the dialog over the explorer, which mounts again. A dialog closed before any data arrives leaves the explorer as it was.Tests/App/Dashboard/SetupGuideFirstData.test.tsxchecks that every type's Setup Guide tab passes the picked key and its explorer page.Tests/App/Dashboard/TelemetryGuidedSetup.test.tsxchecks that the guided setup's wait is not handed the key.Mutation checks. I broke the fix one way at a time, ran the suites that should notice, and restored the file. All 15 were caught (in two full runs, with the same results):
Checked for real
I ran this on a local stack: the App from this branch, behind the bundled nginx template, at
http://localhost:17849.telemetry-ingestion-key/get-list,_id,name,lastUsedAt), the key had nolastUsedAt, and the project had no services.fluent/fluent-bit:3.2.10ran the page'sfluent-bit.conffor that key (Host localhost,Port 17849,Tls Off,Logs_uri /otlp/v1/logs). Its first batch gotHTTP status=200at 22:34:49.lastUsedAtbecame22:34:49.379Z. The watch's request read it on its next poll, at 22:34:52.primaryEntityTypeUnknown.On main, that step waits for a service and would never have finished. With this change it finishes on that poll. The stack checked the server side the step relies on, with the same request the watch makes; the page itself is what the RTL tests above render.
Follow-ups (not in this PR)
lastUsedAt.GrpcServer.tsresolves and checks the key but never callsmarkUsed, unlike the HTTP middleware. So a key used only over gRPC (port 4317, or the ingress's gRPC route) reads as never used on its own page, and this wait would not see its first data. The Dashboard's guides do not offer gRPC, so this PR does not depend on it.lastUsedAtis throttled to five minutes per process.Closes #7754
🤖 Generated with Claude Code