Skip to content

prime001/primebus-spec

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 

Repository files navigation

PrimeBus: A Reference Architecture for Telemetry-Driven Change Intelligence

Version: 1.0
Date: April 2026
Author: Erik Anderson, Prime Automation Solutions
Status: Production-validated


1. Abstract

PrimeBus is an event-driven intelligence bus that collects operational telemetry from software engineering systems -- git repositories, CI/CD pipelines, dependency ecosystems, application lifecycle events, and infrastructure metrics -- correlates signals across those systems in real time, and triggers autonomous remediation when confidence thresholds are met. Built on NATS JetStream with CloudEvents-aligned envelopes and a hierarchical subject taxonomy, PrimeBus implements a three-stage pipeline (Signal, Detect, Resolve) that transforms fragmented telemetry into classified change events and, where appropriate, into verified automated actions. This document specifies the architecture pattern, event envelope format, agent lifecycle, reactor execution model, and observability contract. For the operational philosophy and deployment context, see the companion whitepaper: Anderson, E. (2026), Autonomous NOC Operations: A Framework for Network Self-Healing Systems.


2. Problem Statement

Modern engineering operations generate telemetry across dozens of systems that were never designed to communicate with each other. A single code change may produce a git commit event, trigger a CI pipeline, update a container image tag, cause a dependency resolution change, alter application metrics, and eventually surface as a monitoring alert. Each of these signals lives in a separate tool with its own data model, its own alerting rules, and its own notion of "what happened." No single system correlates the commit to the metric regression to the alert.

Martin Fowler identifies four distinct patterns commonly conflated under "event-driven architecture": Event Notification, Event-Carried State Transfer, Event Sourcing, and CQRS [1]. Most operational tooling implements only Event Notification -- a lightweight signal that something occurred -- without carrying enough context for downstream consumers to classify and act without callbacks to the originating system. The result is a monitoring landscape where tools observe in isolation and engineers serve as the correlation engine, manually connecting signals across dashboards, log aggregators, and alerting channels.

The Cloud Native Computing Foundation's serverless working group recognized this gap in their 2018 whitepaper [6] and subsequent work on CloudEvents [5], establishing common event envelopes and identifying event-driven automation as a first-class operational pattern. The CNCF blog "DevOps + Serverless = Event Driven Automation" (2020) [9] explicitly calls for systems that bridge the gap between observation and action. Yet the dominant operational tools -- Datadog, Grafana Cloud, New Relic, PagerDuty -- remain firmly on the observation side. Workflow engines like Netflix Conductor [4] and Temporal [3] provide durable execution but require pre-defined workflow DAGs, making them workflow-first rather than telemetry-first. They answer "execute this sequence reliably" but not "what just happened across my entire stack and what should I do about it."

PrimeBus addresses this gap by implementing a telemetry-first intelligence layer: collect signals broadly, classify changes with carried state, and act selectively through a confidence-gated reactor pattern that enforces verification before execution and escalates to human operators when confidence is insufficient. The architecture treats the event bus not as a message transport but as the central nervous system of the engineering operation.


3. Architecture Overview

PrimeBus implements a three-stage pipeline: Signal (collect and normalize telemetry), Detect (classify, correlate, and score changes), and Resolve (execute verified remediation or escalate).

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Signal    │    │   Detect    │    │   Resolve   │
│ Watchers    │    │ Agents      │    │ Executors   │
│ Bridges     │    │ Correlators │    │ Validators  │
│ Collectors  │    │ Classifiers │    │ Escalation  │
└──────┬──────┘    └──────┬──────┘    └──────┬──────┘
       │                  │                  │
═══════╧══════════════════╧══════════════════╧═══════
              Event Bus (NATS JetStream)
═════════════════════════════════════════════════════

All three stages communicate through the event bus. Signal producers publish normalized events. Detect agents subscribe to those events and publish classification results. Resolve executors subscribe to classification outputs and publish outcome telemetry. The bus itself serves as both transport and persistence layer, with JetStream providing durable replay from any point in the stream.

Components

Event Bus (NATS JetStream). The transport and persistence backbone, providing subject-based routing, durable consumers, at-least-once delivery, and message replay by sequence or timestamp [2].

Telemetry Collectors. Watcher agents that observe external systems (git repositories, dependency registries, application metrics endpoints, infrastructure monitors) and publish normalized ChangeEvent envelopes to the bus.

Intelligence Agents. Subscribers that consume telemetry events, apply classification logic (rule-based or AI-assisted), and publish enriched events with confidence scores and recommended actions.

Scoring Engine. Evaluates proposed actions against configurable thresholds, determining whether an action proceeds to execution, requires additional validation, or escalates to a human operator.

Event Store (SQLite). Materializes the event stream into a queryable relational store for trend analysis, replay, and audit. Functions as the read-optimized projection of the JetStream event log.

Digest Engine. Aggregates events over configurable time windows and produces summary notifications, surfacing what the system observed and what actions it took.

Performance Analyzer. Monitors application telemetry for anomalous patterns, generating alerts when metrics deviate from established baselines.


4. Event Envelope Specification

Subject Taxonomy

PrimeBus uses a hierarchical dot-delimited subject namespace that maps directly to NATS wildcard subscriptions. Subjects follow the pattern {domain}.{scope}.{identifier}.{action}[.{result}]:

# Code telemetry
change.repo.{project}.push              # Git push detected
change.repo.{project}.test.fail         # Test suite failed
change.repo.{project}.test.pass         # Test suite passed
change.dep.{ecosystem}.{package}        # Dependency update available
change.api.{service}.breaking           # Breaking API change detected

# Infrastructure telemetry
change.infra.{host}.{metric}            # Infrastructure anomaly

# Agent telemetry
agent.{name}.{action}                   # Agent lifecycle and work events

# Fix pipeline telemetry (sub-namespace, not an agent name)
# agent.fix.* tracks fix-pipeline lifecycle events, not a specific agent instance.
# "fix" is a functional sub-namespace representing the fix generation, scoring,
# and merge pipeline -- distinct from agent.{name}.{action} instance events.
agent.fix.proposed                      # AI-generated fix ready for validation
agent.fix.scored                        # Scoring complete, winner selected
agent.fix.merged                        # Fix auto-merged
agent.fix.escalated                     # Escalated to human review

# Validation telemetry
result.validation.{project}.{run_id}    # Test results for a variant
result.score.{project}.{run_id}         # Final score for a variant
result.feedback.{project}.{run_id}      # Outcome feedback

# Operational telemetry
digest.daily                            # Daily summary event
digest.alert                            # High-priority alert

# Application telemetry (external publishers)
app.{application}.{event}               # External application lifecycle events

This taxonomy enables pattern-based routing via NATS wildcards. An agent subscribing to change.repo.> receives all repository events across all projects. An agent subscribing to change.repo.ProjectX.* receives only events for a specific project. An agent subscribing to app.> receives all external application events without coupling to their internal structure.

Namespace Ownership

Namespace Owner Purpose
change.> PrimeBus collectors Git, dependency, API, and infrastructure telemetry
agent.> PrimeBus intelligence Fix generation, scoring, escalation
result.> PrimeBus harness Validation scores, feedback loops
digest.> PrimeBus digest engine Summaries and alerts
app.> External applications Application lifecycle telemetry from any system

Envelope Format

Every event on the bus conforms to the following envelope:

{
  "event_id":   "unique identifier (UUID hex)",
  "event_type": "hierarchical subject taxonomy string",
  "source":     "originating system identifier",
  "project":    "project scope identifier",
  "timestamp":  "ISO 8601 UTC timestamp",
  "payload":    {},
  "metadata":   {}
}

Required fields:

Field Type Description
event_id string Globally unique event identifier (UUID)
event_type string The subject taxonomy string, also used as the NATS publish subject
source string Identifier of the system that generated the event
project string Project scope for the event
timestamp string ISO 8601 timestamp in UTC
payload object Event-specific data, varies by event type
metadata object Contextual data (JetStream sequence, correlation IDs); field is required but contents are optional -- an empty {} is valid

Alignment with CloudEvents

The envelope aligns with the CloudEvents specification v1.0.2 [5] on the core required attributes: source (URI-reference identifying the event origin), type (event kind descriptor), and time (RFC 3339 timestamp). PrimeBus extends CloudEvents in two ways:

  1. JetStream sequence numbers. CloudEvents does not define a durable ordering primitive. PrimeBus leverages JetStream's stream sequence numbers to provide total ordering within a stream and enables replay from any sequence position.

  2. Hierarchical dot-delimited subjects. CloudEvents defines type as a flat string. PrimeBus structures event_type as a dot-delimited hierarchy (e.g., change.repo.ProjectX.push) that doubles as the NATS routing subject, enabling wildcard subscriptions for pattern-based filtering that CloudEvents' flat type string does not support.


5. Agent Lifecycle

Agents are long-running processes that subscribe to event streams via JetStream durable consumers. The lifecycle consists of four phases: registration, consumption, processing, and shutdown.

Registration

On startup, an agent:

  1. Establishes a connection to the NATS server with automatic reconnection (infinite retry, configurable backoff).
  2. Obtains a JetStream context and ensures the target stream exists (creates or updates as needed).
  3. Creates a durable subscription on its target subject pattern. The durable name uniquely identifies this consumer -- JetStream tracks delivery position per durable name, so an agent that disconnects and reconnects resumes from its last acknowledged message.
  4. Records its initial heartbeat.

Two-Stage Consumption Architecture

To prevent slow handlers from causing NATS slow-consumer drops, agents implement a two-stage consumption pattern:

┌──────────────────┐      ┌──────────────┐      ┌───────────────┐
│  NATS JetStream  │─────▶│  Stage 1:    │─────▶│  Stage 2:     │
│  (durable sub)   │      │  Fast Ack +  │      │  Worker Pool  │
│                  │      │  Queue Push  │      │  (N workers)  │
└──────────────────┘      └──────────────┘      └───────────────┘

Stage 1 (Consume): Reads messages from NATS as fast as possible, deserializes and validates the envelope, acknowledges immediately (the agent now owns the message), and pushes the event onto an internal bounded queue.

Stage 2 (Workers): A configurable pool of async workers pulls events from the internal queue and runs the handler function. Slow operations (AI inference, external API calls, notification delivery) execute in this stage without blocking NATS consumption.

This architecture decouples NATS delivery guarantees from handler execution speed. If the internal queue fills, the agent drops the oldest unprocessed event and increments a drop counter -- preferring lossy processing over NATS back-pressure that could affect other consumers on the same stream. When a consumer's pending queue reaches capacity, the oldest unacknowledged event is dropped. This policy prioritizes recent signals over stale ones -- in operational contexts, a 10-minute-old alert that hasn't been processed is less actionable than a current one.

Heartbeat and Health

Agents report liveness through two Prometheus gauges:

  • primebus_agent_up{agent="name"} -- binary liveness indicator (1 = alive, 0 = dead)
  • primebus_agent_heartbeat_timestamp{agent="name"} -- Unix timestamp of last heartbeat

External monitoring (Grafana alerting or equivalent) watches for stale heartbeats and absent agent_up signals to detect agent failures.

Cooldown and Deduplication

Agents that trigger actions implement cooldown windows per rule or per subject pattern. When an event matches a rule that has fired within its cooldown period, the event is skipped and primebus_reactor_cooldown_blocked_total{rule_id="id"} is incremented. This prevents event storms from cascading into action storms.

Prometheus Metrics Contract

Every agent exposes the following metrics:

Metric Type Labels Description
primebus_agent_up Gauge agent Agent liveness (1/0)
primebus_agent_heartbeat_timestamp Gauge agent Last heartbeat (Unix epoch)
primebus_events_consumed_total Counter consumer, project Events successfully processed
primebus_events_published_total Counter event_type, project Events published to the bus
primebus_reactor_triggered_total Counter rule_id, risk Events that matched a reactor rule
primebus_reactor_completed_total Counter rule_id, status Reactor sessions completed
primebus_reactor_cooldown_blocked_total Counter rule_id Events skipped due to cooldown

6. Reactor Pattern

The reactor is PrimeBus' execution engine: the component that translates classified events into verified actions. It implements a six-step pipeline with mandatory verification and rollback gates.

Execution Flow

Event Received
    │
    ▼
Rule Match? ──No──▶ Skip
    │ Yes
    ▼
Cooldown Active? ──Yes──▶ Skip (increment cooldown counter)
    │ No
    ▼
Verify Sender ──Fail──▶ Escalate to Human
    │ Pass
    ▼
Execute Action
    │
    ▼
Run Verification Tests
    ├──Pass──▶ Log Outcome ──▶ Publish result.* event ──▶ Done
    └──Fail──▶ Rollback ──▶ Escalate to Human

Step Definitions

Rule Match. Each incoming event is evaluated against a set of rules. A rule defines a subject pattern (supporting NATS wildcards), optional payload conditions, a risk classification, and a maximum concurrency limit. Rules are declarative -- they define when to act, not how to act.

Cooldown Check. If the matched rule has fired within its cooldown window, the event is discarded. The primebus_reactor_cooldown_blocked_total counter records the skip. This prevents cascading reactions to event bursts (e.g., rapid successive commits to the same repository).

Sender Verification. Before executing, the reactor validates that the event source is authorized for the matched rule. Sender verification may be implemented via allowlists, role-based access control, or cryptographic signature validation. The mechanism is deployment-specific; the requirement is that untrusted sources cannot trigger remediation actions. Events from unrecognized sources are escalated rather than executed.

Execute Action. The reactor spawns an isolated execution session with a scoped working directory, a constrained tool set, and a timeout. Active sessions are tracked by primebus_reactor_active_sessions and bounded by a configurable concurrency limit.

Verification Tests. After execution, the reactor runs the project's test suite or validation checks. This is the critical gate: no action is considered successful until independently verified.

Rollback and Escalation. If verification fails, the reactor reverts the action (git reset, config restore, or equivalent) and escalates to a human operator with full context: what event triggered the action, what action was taken, what verification failed, and what was rolled back.

The Say-Do Loop

The reactor implements a closed-loop verification pattern: every action (the "do") is correlated with its stated intent (the "say") and measured outcome (the "result"). This three-part correlation -- intent, action, outcome -- is recorded in the event store and published as result.feedback.* events. Over time, the historical record of say-do-result triples enables trend analysis: which types of automated actions succeed reliably, which require frequent rollback, and which should be removed from automation entirely.

Escalation Boundary

PrimeBus enforces a hard boundary between autonomous action and human review. Events escalate to human operators when:

  • Confidence scores fall below threshold
  • Verification tests fail after execution
  • The event involves security-sensitive changes
  • The matched rule is classified as high-risk
  • The reactor reaches its concurrent session limit

Escalation is not a failure mode -- it is a design constraint. The system is explicitly built to know what it does not know and to route uncertainty to human judgment rather than act on insufficient confidence.


7. Observability

PrimeBus follows the principle: monitor the automation as rigorously as the infrastructure it manages. Every component exposes Prometheus metrics via an HTTP /metrics endpoint, and a /health endpoint for liveness probes.

Full Metrics Table

Metric Type Labels Description
primebus_events_published_total Counter event_type, project Total events published to the bus
primebus_events_consumed_total Counter consumer, project Total events consumed by subscribers
primebus_nats_connected Gauge -- NATS connection status (1/0)
primebus_agent_up Gauge agent Agent liveness (1/0)
primebus_agent_heartbeat_timestamp Gauge agent Last heartbeat Unix timestamp
primebus_fixes_generated_total Counter project, variant Fix variants generated
primebus_fixes_scored_total Counter project, action Scoring decisions (create_pr / escalate)
primebus_fix_score Histogram project, variant Distribution of fix scores (buckets: 0-100)
primebus_test_runs_total Counter project, status Test suite runs (pass / fail / skip)
primebus_test_duration_seconds Histogram project Test suite execution time
primebus_dep_updates_found_total Counter project, ecosystem Outdated dependencies detected
primebus_reactor_triggered_total Counter rule_id, risk Events that matched a reactor rule
primebus_reactor_completed_total Counter rule_id, status Reactor sessions completed
primebus_reactor_duration_seconds Histogram rule_id Reactor session duration
primebus_reactor_active_sessions Gauge -- Currently running reactor sessions
primebus_reactor_cooldown_blocked_total Counter rule_id Events skipped due to cooldown
primebus_claude_sessions_total Counter agent, project, status AI sessions spawned
primebus_claude_session_duration_seconds Histogram agent, project AI session duration
primebus_claude_sessions_active Gauge agent Currently active AI sessions
primebus_claude_token_estimate_total Counter agent, project Estimated tokens consumed
primebus_sqlite_events_total Gauge -- Total events in persistent store
primebus_sqlite_outcomes_total Gauge -- Total outcomes in persistent store

Grafana Dashboard Patterns

The metrics above support four standard dashboard views:

Bus Health. primebus_nats_connected, primebus_events_published_total (rate), primebus_events_consumed_total (rate). Alerts on connection loss, publish rate anomalies, or consumer lag (published rate significantly exceeding consumed rate).

Agent Health. primebus_agent_up (panel per agent), primebus_agent_heartbeat_timestamp (staleness alert at configurable threshold). A single pane showing which agents are alive, when they last reported, and whether any are in a degraded state.

Reactor Activity. primebus_reactor_triggered_total (rate by rule), primebus_reactor_completed_total (rate by status), primebus_reactor_active_sessions, primebus_reactor_duration_seconds (p50/p95/p99), primebus_reactor_cooldown_blocked_total (rate). This dashboard answers: how often is the system acting, how long do actions take, how often do they succeed, and how often is the cooldown preventing cascades.

AI Resource Consumption. primebus_claude_sessions_total (rate by agent and status), primebus_claude_session_duration_seconds (histograms), primebus_claude_sessions_active, primebus_claude_token_estimate_total (rate). Tracks AI compute cost and identifies agents that are consuming disproportionate resources.


8. Reference Implementation Notes

This specification is validated by a production deployment operational since March 2026, managing telemetry collection, change classification, and automated remediation across 32+ projects operated by a single-person engineering team, demonstrating that the architecture scales through agent addition rather than headcount. The system processes git push events, dependency updates, application lifecycle telemetry, infrastructure metrics, and AI-generated content pipeline events through the three-stage Signal-Detect-Resolve pipeline described in this document, processing thousands of events daily across application lifecycle, code changes, infrastructure telemetry, and operational workflows.

The deployment demonstrates that the architecture pattern scales not by adding engineers but by adding agents: each new telemetry source or remediation capability is an additional subscriber on the bus, with no changes required to existing components. The event envelope and subject taxonomy have remained stable through six development sprints and the addition of multiple external publishers.

Early deployment revealed that reactor cooldown periods require per-rule tuning -- a 5-minute cooldown appropriate for deployment events caused missed alerts when applied to infrastructure telemetry with sub-minute signal frequency. This led to the per-rule cooldown configuration described in Section 6, where each rule defines its own cooldown window matched to the expected signal cadence of its subject pattern.

No source code, deployment configuration, or implementation details are included in this specification. For the operational philosophy and organizational context, see the companion whitepaper: Anderson, E. (2026), Autonomous NOC Operations: A Framework for Network Self-Healing Systems, Prime Automation Solutions.


9. Prior Art and References

[1] Fowler, M. (2017). "What do you mean by 'Event-Driven'?" martinfowler.com. https://martinfowler.com/articles/201701-event-driven.html

[2] Synadia Communications / CNCF. (2021). "NATS JetStream -- Persistence, Streaming, and Higher Message Guarantees." https://docs.nats.io/jetstream

[3] Temporal Technologies Inc. (2019). "Temporal -- Durable Execution Platform." https://docs.temporal.io/

[4] Netflix Technology Blog. (2016). "Netflix Conductor: A microservices orchestrator." https://netflixtechblog.com/netflix-conductor-a-microservices-orchestrator-2e8d4771bf40

[5] Cloud Native Computing Foundation. (2019). "CloudEvents -- A specification for describing event data in a common way." Version 1.0.2. https://github.com/cloudevents/spec

[6] CNCF Serverless Working Group. (2018). "CNCF Serverless Whitepaper v1.0." https://github.com/cncf/wg-serverless/blob/master/whitepapers/serverless-overview/README.md

[7] Anderson, E. (2025). The Autonomous Engineer. Prime Assets Inc.

[8] Anderson, E. (2026). Autonomous NOC Operations: A Framework for Network Self-Healing Systems. Prime Automation Solutions. (Companion whitepaper)

[9] CNCF. (2020). "DevOps + Serverless = Event Driven Automation." Cloud Native Computing Foundation Blog. https://www.cncf.io/blog/2020/02/14/devops-serverless-event-driven-automation/

Further Reading

Arafat, J., Tasmin, F., & Poudel, S. (2025). "Next-Generation Event-Driven Architectures: Performance, Scalability, and Intelligent Orchestration Across Messaging Frameworks." arXiv preprint. https://arxiv.org/html/2510.04404


This document describes an architecture pattern. It does not contain connection strings, deployment configuration, agent source code, scoring algorithms, or reactor rule definitions. For implementation inquiries, contact Prime Automation Solutions.

About

PrimeBus: An Event-Driven Intelligence Architecture for Autonomous Operations

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors