Skip to content

feat(web): add deployment system stats to service ping#1424

Merged
brendan-kellam merged 5 commits into
mainfrom
brendan/sou-1488-add-system-stats-to-service-ping
Jul 6, 2026
Merged

feat(web): add deployment system stats to service ping#1424
brendan-kellam merged 5 commits into
mainfrom
brendan/sou-1488-add-system-stats-to-service-ping

Conversation

@brendan-kellam

@brendan-kellam brendan-kellam commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Fixes SOU-1488

What

Collects a best-effort snapshot of the deployment's host/container resources and sends it as an optional systemInfo object on the daily service ping. Lighthouse forwards it into the lh_ping PostHog event, giving us a quick list of resource facts per deployment so we can diagnose issues (e.g. "this customer doesn't have enough RAM") and point them towards scaling.

How

New packages/web/src/features/billing/systemInfo.ts gathers:

Field Source
platform, arch, cpuCores, loadAverage1m os
totalMemoryBytes, freeMemoryBytes os.totalmem() / os.freemem()
cpuQuota cgroup v2 cpu.max / v1 cpu.cfs_quota_us + cpu.cfs_period_us
memoryLimitBytes, memoryUsedBytes cgroup v2 memory.max/memory.current, v1 memory.limit_in_bytes/memory.usage_in_bytes
diskTotalBytes, diskFreeBytes fs.statfs(DATA_CACHE_DIR)

Why cgroups? Sourcebot ships as a container (Docker/Kubernetes). os.totalmem()/os.cpus() report the host, not the container's limits, so the cgroup-derived limits/usage are what actually tell us whether a deployment is resource-constrained. Both cgroup v1 and v2 layouts are handled.

Safety / back-compat

  • The systemInfo field is .optional() on servicePingRequestSchema (mirrored in Lighthouse), so it stays backwards compatible in both directions.
  • Every value is best-effort: anything unreadable (e.g. cgroup files absent on local macOS dev, statfs failure) is reported as null, and the whole collection is wrapped so a failure can never block the ping.
  • No DB migration: the ping payload is already persisted as JSON in ServicePingEvent.payload.

Updated the Service Ping docs (docs/docs/misc/service-ping.mdx) with the new field table + example payload.

Paired PR

Lighthouse side (validates + captures systemInfo): sourcebot-dev/lighthouse#26

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Service ping now includes optional deployment resource stats when available, such as CPU, memory, disk, and load-related information.
    • Added support for collecting and sending a richer system information payload alongside existing ping data.
  • Documentation

    • Updated the service ping documentation with the new payload field and an example response.
  • Chores

    • Added a changelog entry noting the new resource statistics included in service ping responses.

Collects a best-effort snapshot of the deployment's host/container
resources (CPU cores + cgroup CPU quota, host + cgroup memory, disk
for the data volume, load average, platform) and includes it as an
optional `systemInfo` object on the service ping payload.

This surfaces resource facts in the `lh_ping` PostHog event so we can
quickly diagnose resource issues (e.g. a customer running out of RAM)
and point them towards scaling. Container-aware cgroup limits/usage are
read because os.totalmem()/os.cpus() report the host, not the
container's limits.

The `systemInfo` field is optional for back-compat with Lighthouse
deployments that predate it, and every collected value is best-effort
(reported as null when unreadable) so it can never break the ping.

Fixes SOU-1488

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: be59833a-1a9d-4bdc-8d90-c52bfb58a1e0

📥 Commits

Reviewing files that changed from the base of the PR and between 9c4780d and 9a3fa6c.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • docs/docs/misc/service-ping.mdx
  • packages/web/src/features/billing/servicePing.ts
  • packages/web/src/features/billing/systemInfo.ts
  • packages/web/src/features/billing/types.ts

Walkthrough

Adds a systemInfo field to the Service Ping payload, including a new schema, a getSystemInfo() collection module reading cgroup memory/CPU limits and disk stats, integration into syncWithLighthouse with best-effort error handling, plus documentation and changelog updates.

Changes

System resource stats in service ping

Layer / File(s) Summary
SystemInfo schema and payload type
packages/web/src/features/billing/types.ts
Adds systemInfoSchema and inferred SystemInfo type; extends servicePingRequestSchema with an optional systemInfo field.
System info collection module
packages/web/src/features/billing/systemInfo.ts
Implements getSystemInfo() gathering platform/arch, cgroup v1/v2 memory and CPU quota, and disk stats via statfs, with null fallbacks and rounding helpers.
Service ping integration and docs
packages/web/src/features/billing/servicePing.ts, docs/docs/misc/service-ping.mdx, CHANGELOG.md
Calls getSystemInfo() best-effort in syncWithLighthouse, conditionally includes it in the payload, and documents the new field with example JSON and a changelog entry.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • sourcebot-dev/sourcebot#1296: Both PRs modify the Service Ping request payload schema and syncWithLighthouse in packages/web/src/features/billing.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch brendan/sou-1488-add-system-stats-to-service-ping

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

brendan-kellam and others added 3 commits July 6, 2026 14:08
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Report the systemInfo memory and disk fields in whole MiB (binary,
1024-based units) instead of raw bytes, so the values are directly
human-readable when diagnosing resource issues (e.g. a 16Gi container
limit reads as 16384). Converts at the collection boundary via a
bytesToMiB helper and updates the mirrored docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drops the host/node-level readings from the systemInfo snapshot
(cpuCores, loadAverage1m, totalMemoryMiB, freeMemoryMiB). These reported
the underlying machine rather than the deployment's own allocation, which
leaked node sizing (especially in k8s) and added noise without aiding the
"is this deployment resource-constrained?" diagnosis.

What remains is purely container/volume-scoped: cpuQuota (cgroup CPU
limit), memoryLimitMiB/memoryUsedMiB (cgroup), and diskTotalMiB/
diskFreeMiB (statfs on the data volume).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@brendan-kellam brendan-kellam marked this pull request as ready for review July 6, 2026 23:09
@brendan-kellam brendan-kellam merged commit 11ef472 into main Jul 6, 2026
11 of 12 checks passed
@brendan-kellam brendan-kellam deleted the brendan/sou-1488-add-system-stats-to-service-ping branch July 6, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant