Flow Metrics & Analytics

LumenFlow tracks flow metrics to help teams identify bottlenecks and improve delivery performance.

Overview

The @lumenflow/metrics package provides:

• DORA Metrics — The dora.dev 2025 canonical 5-metric model
• Flow Analysis — Bottleneck detection and critical path calculation
• Telemetry — Event emission for local NDJSON logs and cloud sync

Concept reference

For formal definitions, formulas, tier thresholds, and the MTTR → FDRT / Rework Rate rationale, see the DORA Metrics concept page.

DORA Metrics (2025 canonical 5-metric model)

LumenFlow tracks the five metrics defined by dora.dev and refreshed by the CDF Oct 2025 announcement. Aggregation follows DORA canonical guidance: mean for deployment frequency, median for lead time and FDRT.

Metric	Group	Formula	Unit	Aggregation	Target
Deployment Frequency	Throughput	commits_in_window / days_in_window * 7	/week	mean	Daily → weekly
Lead Time for Changes	Throughput	WU cycle time = completed_at − claimed_at	hours	median	< 24h
Failed Deployment Recovery (FDRT)	Throughput	median(time between paired EMERGENCY commits)	hours	median	< 1h
Change Failure Rate (CFR)	Instability	failures / total_deployments * 100	%	ratio	< 15%
Deployment Rework Rate	Instability	(revert + hotfix commits) / total_deployments * 100	%	ratio	< 5%

MTTR retired in favour of FDRT

DORA replaced MTTR with Failed Deployment Recovery Time (FDRT) in 2023 because MTTR conflated broad production incidents (infra outages, third-party failures) with deploy-caused failures — the only kind DORA research claims predictive power over. FDRT scopes the metric to deploy failures and standardises on median aggregation (mean skews to long-tail outliers).

Local proxies in LumenFlow OSS

• CFR uses the skip-gates audit log as a failure proxy. True production incident signals require CI-side event integration (future work).
• FDRT pairs consecutive EMERGENCY / fix(EMERGENCY) commits as {break, fix} events.
• Deployment Rework Rate counts commits matching Revert , revert:, hotfix, or fix(EMERGENCY); commits matching multiple patterns are deduplicated by hash.

Viewing DORA Metrics

pnpm metrics:snapshot                 # All metrics, JSON output
pnpm metrics:snapshot --type dora     # DORA metrics only
pnpm metrics:snapshot --days 30       # 30-day window (normalised to per-week)
pnpm metrics:snapshot --dry-run       # Preview; no NDJSON written, no cloud sync

Example output:

DORA METRICS (2025 canonical 5-metric model)
Deployment Frequency: 6/week (elite)
Lead Time: 12h median (elite)
Failed Deployment Recovery Time: 0.5h median (elite)
Change Failure Rate: 8% (elite)
Deployment Rework Rate: 3% (elite)

Cloud Sync

When a workspace has a control_plane endpoint configured, DORA records are shipped to POST <endpoint>/api/v1/telemetry in batches of up to 1000 records. A typical metrics:snapshot run emits 5 records and fits in a single batch.

NDJSON staging

Records are first appended to .lumenflow/telemetry/dora.ndjson, then the cloud sync worker reads from the persisted cursor offset and posts batched payloads. This gives offline resilience: retries resume from the last acknowledged offset.

Tag taxonomy

Every record carries a tags bag the control plane can slice dashboards by. Values are primitive (string | number | boolean); missing values are omitted rather than emitted as empty strings.

Tag	Source	Example
source_type	Hard-coded	"dora"
calculated_by	Hard-coded	"metrics:snapshot"
tier	Per-metric classification	"elite"
repo	git config --get remote.origin.url → parsed owner/repo	"hellmai/lumenflow"
branch	git rev-parse --abbrev-ref HEAD	"lane/framework-metrics/wu-2635"
commit_sha	git rev-parse HEAD	"deadbeef…"
service	workspace.yaml → service (or software_delivery.service)	"control-plane"
environment	workspace.yaml → environment, fallback LUMENFLOW_ENV	"prod"
snapshot_window	--days flag	"7d", "30d"
pipeline	CI_PIPELINE_NAME, fallback GITHUB_WORKFLOW	"main-ci"
deploy_target	DEPLOY_TARGET	"prod-eu"
workflow_run_id	GITHUB_RUN_ID, fallback CI_PIPELINE_ID	"987654"

Lead time and FDRT records additionally carry aggregation: "median", mean_hours, and p90_hours so trend dashboards can plot all three aggregations without re-running the CLI. CFR records carry failures + total_deployments; Deployment Rework Rate carries rework_commits + total_deployments.

Enabling cloud sync

pnpm cloud:connect                        # Interactive OAuth + workspace.yaml scaffolding
pnpm config:get --key control_plane       # Verify endpoint + sync_interval
pnpm metrics:snapshot                     # Emits NDJSON + triggers cloud sync when configured

See Workspace spec for the full control_plane schema.

Full-stream telemetry routing (INIT-078)

A connected control plane no longer sees only the original four NDJSON sources (gates, flow, dora, costs). INIT-078 routes every audited local stream to an existing transport — the telemetry registry, the kernel-event envelope, or a memory/signal port — or records why it is excluded. The routing contract is ratified once, vendor-neutrally, in ADR-023 in the internal architecture decision record set. No new packages and no new sync endpoints are introduced.

Six new telemetry source ids join the registry, bringing it to ten append-only NDJSON sources synced with offset tracking:

Source id	Local file	Default payload mode
llm-classification	.lumenflow/telemetry/llm-classification.ndjson	full (PII-free at emit)
lane-signals	.lumenflow/telemetry/lane-signals.ndjson	full
tools	.lumenflow/telemetry/tools.ndjson	metadata-only (hashes + sizes)
methodology	.lumenflow/telemetry/methodology.ndjson	full
prompt-lint	.lumenflow/telemetry/prompt-lint.ndjson	full
incidents	.lumenflow/incidents/*.ndjson (directory)	metadata-only (stack → hash)

The sidecar tick and session-end flush keep these sources current for a connected session, not just at gate-run time.

Per-stream payload modes

Each cloud-synced stream is configured under software_delivery.telemetry.cloud_sync.streams.<source-id> with enabled and an optional payload_mode:

software_delivery:
  telemetry:
    cloud_sync:
      streams:
        tools:
          enabled: true
          payload_mode: metadata-only # default for tools/memory/incidents
        incidents:
          enabled: true
          payload_mode: metadata-only
        llm-classification:
          enabled: true
          payload_mode: full # PII-free streams default to full

payload_mode has two values:

• metadata-only (default for the free-text-bearing streams — tools, memory, incidents): free-text fields are replaced by a content hash plus a byte size (e.g. arg_sha256 + arg_bytes, stack_sha256 + stack_bytes). Structured low-risk fields (tool name, exit code, duration, lane, WU id, memory type, tags) pass through unredacted.
• full: the complete payload is sent. Streams that are already free of user content (llm-classification, lane-signals, methodology, prompt-lint) default to full.

Redaction is applied in mappers, not emitters: the local file always keeps the complete record for local debugging (cost:summary, flow:report, forensic inspection). Only the wire projection is redacted, and flipping a stream to full changes only the mapper output, never the on-disk file.

Governance evidence as kernel events

Force-bypass and skip-gates audit logs are now cloud-visible as schema-validated kernel events on POST /api/v1/events, forming a governance event domain:

Kernel event	Source log
governance:force_bypass_recorded	force-bypasses.log
governance:gates_skipped	skip-gates-audit.log

Both carry full governance evidence (the audit trail is the point). A wu:done --skip-telemetry-check bypass is itself audited and surfaces as a governance:force_bypass_recorded event, so bypasses cannot hide.

Memory and signal routing

Shared memory (.lumenflow/memory/memory.jsonl) and A2A signals (.lumenflow/memory/signals.jsonl) route through dedicated memory/signal sync ports rather than the telemetry registry. Memory rows default to metadata-only (body → hash + size); signals send their structured keys verbatim and apply the redaction policy only to free-text bodies.

Sync-health enforcement

Sync health is loud and, when configured, enforceable at wu:done via software_delivery.telemetry.cloud_sync.enforcement:

software_delivery:
  telemetry:
    cloud_sync:
      enforcement: warn # off | warn | block (default warn)

Level	wu:done behaviour
off	No gate. Health still surfaces in the doctors.
warn	Default. Prints per-stream pending counts + remediation; never blocks.
block	Blocks wu:done only on a deterministic failure class (HTTP 4xx auth/config rejection, missing token).

Transient failures (timeouts, 5xx, DNS, connection resets) never block, even under block — they degrade to a warning so a flaky network cannot brick wu:done. Bypass a single run with the audited pnpm wu:done --id WU-XXX --skip-telemetry-check --reason "...".

Sync health is also a first-class line in both doctors:

pnpm lumenflow:doctor      # main doctor — telemetry sync-health section
pnpm compute:doctor        # compute doctor — same per-source health

Each reports per-source last_success_at / last_error, so a silent skip is no longer the only observable state.

Flow Analysis

Identifying Bottlenecks

pnpm flow:bottlenecks

This analyzes your WU flow to identify:

• Lane Congestion — Lanes exceeding WIP limits
• Blocked WUs — Work units waiting on dependencies
• Stale WUs — WUs in progress for too long
• Critical Path — WUs blocking the most downstream work

Metrics Snapshots

Capture point-in-time metrics for dashboards or CI:

pnpm metrics:snapshot                    # Full snapshot, writes .lumenflow/snapshots/metrics-latest.json
pnpm metrics:snapshot --type dora        # DORA only
pnpm metrics:snapshot --days 30          # 30-day window; value still reported per-week

Telemetry Events

LumenFlow emits structured NDJSON telemetry under .lumenflow/telemetry/:

File	Purpose
.lumenflow/telemetry/gates.ndjson	Gate execution events (duration, pass/fail, WU, lane)
.lumenflow/flow.log	WU lifecycle events (wu:claim, wu:prep, wu:done)
.lumenflow/telemetry/dora.ndjson	DORA metric records with canonical tag bag
.lumenflow/telemetry/costs.ndjson	LLM cost events (model, tokens, USD)
.lumenflow/telemetry/brief-metrics.ndjson	wu:brief prompt token reports by WU, client, section
.lumenflow/telemetry/llm-classification.ndjson	LLM classification lifecycle events
.lumenflow/telemetry/lane-signals.ndjson	Lane signal events
.lumenflow/telemetry/tools.ndjson	Tool invocation events (metadata-only by default)
.lumenflow/telemetry/methodology.ndjson	Methodology metrics
.lumenflow/telemetry/prompt-lint.ndjson	Prompt-lint metrics
.lumenflow/incidents/*.ndjson	Incident records (metadata-only by default)

See Full-stream telemetry routing above for which of these stream to the control plane and at what payload mode.

pnpm wu:brief --id WU-XXX --client <client> --report-tokens prints the same total and section-level token accounting it appends to brief-metrics.ndjson. The file uses schema-versioned control-plane event records and stays local by default; hosted, self-hosted, and third-party consumers are downstream of the same neutral event shape.

CLI Reference

Command	Description
pnpm metrics:snapshot	Capture 5-metric DORA snapshot; emits NDJSON + syncs to cloud
pnpm flow:report	Generate DORA + gate + WU flow report
pnpm flow:bottlenecks	Identify workflow bottlenecks and critical path
pnpm cost:summary --brief	Summarize local wu:brief token metrics by WU, client, and date

Best Practices

1. Review metrics weekly

   Schedule a weekly review of flow metrics to identify trends before they become problems.

2. Set WIP limits appropriately

   If a lane is consistently at 100%+ capacity, consider splitting the lane, adding capacity, or reducing WU scope.

3. Address blockers quickly

   Blocked WUs create cascading delays. Prioritize unblocking over new work.

4. Track trends, not absolutes

   DORA research emphasises continuous improvement over hitting specific numbers. Watch the slope, not the intercept.

Next Steps

• DORA Metrics (concept) — formal definitions, formulas, rationale
• Initiatives — multi-phase project coordination
• Team Workflow — team practices and conventions
• Workspace spec — control_plane configuration