test(user): restore real modules from a pre-mock snapshot - #2031
Conversation
This suite's teardown re-installed its own mocks instead of undoing them.
`import * as realExeca from 'execa'` is a live namespace binding, and
mock.module repoints it. By the time afterEach ran, `realExeca` WAS the
mock, so `mock.module('execa', () => realExeca)` reinstalled the stub -- and
mock.module lasts for the life of the process, so every test file loaded
afterwards got it.
The stub returns { exitCode, stdout } with no stderr, which is what made it
visible elsewhere: collectTaskReportGitMetadata does
`inside.stderr.trim()` and threw "undefined is not an object". The two
task-report CLI handler tests and the two /ads command tests failed on any
run where this file happened to be ordered before them, which is why the
same four went red on unrelated PRs and intermittently on main itself
(6bef0e1, 0ff1d1c).
Snapshot each module surface into a plain object at load, before any mock is
installed, and restore through the snapshots. The stub definitions build on
the snapshot too -- a bare `import('execa')` inside the helper resolves to
whatever mock is current, so each stub was being layered on the last.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
📜 Recent review details⏰ Context from checks skipped due to timeout. (3)
📝 WalkthroughWalkthroughThe PR hardens module mocking in ChangesTest reliability updates
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
jatmn
left a comment
There was a problem hiding this comment.
I found an issue that needs to be addressed before this is ready.
Findings
- [P2] Remove the unrelated VCR fixture
fixtures/734ad7.json:1
This PR does not change the SDK lifecycle test that owns this capture. The fixture makes the existingtest undefined reasonpath replay an empty response instead of executing its normal request path, so it can silently hide regressions; moreover, the test disables built-in agents while the committed input contains an environment-dependent agent-listing reminder. Keep this mock-teardown repair focused and remove the generated fixture (or make an intentional, tested lifecycle-fixture change in its own PR).
The fixtures/734ad7.json capture was accidentally recorded while running the SDK suite locally and is unrelated to the mock-teardown repair. It replays an empty response for the 'test undefined reason' lifecycle path (hiding regressions) and embeds an environment-dependent agent-listing reminder. Remove it to keep this PR focused.
|
Good catch — that fixture was recorded by accident while I ran the SDK suite during verification; it isn't part of this change. Removed it in 59b25c8 so the PR stays scoped to the mock-teardown repair. Any intentional lifecycle-fixture change can go in its own PR. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/utils/user.test.ts`:
- Around line 82-83: Update the getHostPlatformForAnalytics mock in the affected
user test to return the valid Platform value win32 instead of windows, while
leaving the env.platform fixture unchanged unless required by the test contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 23ed3fd8-31fe-4d93-a007-4d381b3d43f1
📒 Files selected for processing (1)
src/utils/user.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: smoke-and-tests (22)
- GitHub Check: smoke-and-tests (24.11.x)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
TypeScript code in this repository must use strict mode and ESM imports.
**/*.{ts,tsx}: Follow the existing code style and architectural patterns in touched TypeScript and TSX files.
Add or update tests when TypeScript or TSX changes affect behavior.
Review AI-generated TypeScript and TSX changes for correctness beyond compilation, consistency with repository architecture and style, unnecessary generated noise, and subtle bugs before submission.
Files:
src/utils/user.test.ts
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Keep pull requests focused on one problem or feature; do not mix unrelated cleanup, fixes, features, or refactors into the same change.
Preserve existing repository patterns unless intentionally refactoring them, and prefer small, readable changes over broad rewrites.
Do not reformat unrelated files, and keep comments useful and concise.
Update documentation when setup, commands, or user-facing behavior changes.
When changing provider behavior, avoid breaking third-party providers, test the exact provider/model path changed when possible, explicitly identify affected providers, and document limitations or follow-up work.
Do not assign or use provider tags; provider tags are controlled and applied by maintainers.
Run the relevant validation checks locally before submitting; CI-required checks includebun run check,bun run test:full, provider tests when applicable, typechecks, andbun run security:pr-scan. Web changes additionally requirebun run web:typecheckandbun run web:build.
Dependency changes must have a concrete project benefit, such as fixing a bug, addressing a security issue, or supporting an approved feature; preference alone is insufficient.
Do not change the project's language, core runtime, dependency stack, or significantly restructure dependencies without prior maintainer agreement.
Before implementing a new feature or other non-trivial change, open an issue to establish scope and alignment with the project roadmap.
Files:
src/utils/user.test.ts
⚙️ CodeRabbit configuration file
**/*: Apply the OpenClaude maintainer review rubric from AGENTS.md. Review the current diff, not stale discussion context. Separate real blockers from suggestions. Do not request changes for vague style churn. Treat approval as merge-ready from CodeRabbit's side, pending required human review and GitHub Checks. If checks are failing or unavailable, say so clearly instead of implying the PR is fully ready.
Files:
src/utils/user.test.ts
**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run focused tests for changed behavior and ensure provider-specific changes include the relevant provider tests.
Files:
src/utils/user.test.ts
{src/**/*.test.ts,src/**/*.test.tsx,tests/**,scripts/**/*.test.ts,vscode-extension/**/*.test.js}
⚙️ CodeRabbit configuration file
{src/**/*.test.ts,src/**/*.test.tsx,tests/**,scripts/**/*.test.ts,vscode-extension/**/*.test.js}: Review tests for meaningful coverage of the changed behavior, isolation of global/env/config state, async cleanup, fake timers, provider profile leaks, and Windows-compatible assumptions. Block when risky runtime changes lack focused regression coverage or tests assert implementation details while missing the user-visible behavior.
Files:
src/utils/user.test.ts
🔇 Additional comments (1)
src/utils/user.test.ts (1)
12-23: LGTM!Also applies to: 33-44, 86-87, 114-118
Use win32 for the analytics platform mock (env.Platform contract) and include stderr on the async execa stub so a future leak fails soft. Rewrite the undefined-reason interrupt lifecycle assertion onto the deterministic queryLoop + stop-hook path so it no longer depends on an empty VCR fixture or SDK model-startup races after fixture removal.
The rewritten "undefined reason" interrupt test was an exact copy of the existing Stop-hook default-abort regression in the same file. Keep the single deterministic coverage path.
The failure
Four tests have been failing intermittently on
mainand on unrelated PRs:with
TypeError: undefined is not an object (evaluating 'inside.stderr.trim')attaskReport.ts:382. Seen onmainat 6bef0e1 and 0ff1d1c, and simultaneously on five of my open PRs that touch none of that code.Cause
src/utils/user.test.tsteardown reinstalls its own mocks instead of undoing them:import * asis a live namespace binding.mock.modulerepoints it, so by the timeafterEachruns,realExecais the stub — and passing it back tomock.modulereinstalls it.mock.moduleis process-global and is not undone bymock.restore(), so from that point every test file loaded afterwards gets the stub.Reduced:
The stub returns
{ exitCode, stdout }with nostderr, which is what makes it visible downstream:collectTaskReportGitMetadatareadsinside.stderr.trim()and throws. Whether it bites depends on whether bun happens to orderuser.test.tsbefore the affected suites — hence the intermittency, and why adding or removing any test file anywhere moves it.Fix
Snapshot each real module surface into a plain object at load, before any mock is installed, and restore through the snapshots. The stub definitions read from the snapshot too — a bare
import('execa')inside the helper resolves to whatever mock is currently installed, so each stub was being built on top of the previous one.This is the same shape as the fixes in #1667 and #1708.
Verification
Bisected the failure to this file: 162 test files in
src/utils, halved down touser.test.tsalone reproducing it againstreportTask.test.ts, and the four other candidates in the final group clean.bun test src/utils/user.test.ts src/utils/reportTask.test.ts— fails before, passes after.bun test src/utils/user.test.ts src/commands/ads*— passes after.bun test --feature=UNATTENDED_RETRY --max-concurrency=1, 7569 tests / 637 files): 26 failures fixed, 0 newly failing.The remaining failures are pre-existing and unrelated (the baseline itself is unstable — two runs on clean
maingave 48 and 69). The two task-report tests are deterministically fixed, verified by the targeted repro above rather than by the full-run diff.Note for a follow-up
22 other suites restore through a live namespace the same way —
grep -rlE "mock\.module\([^,]+,\s*\(\)\s*=>\s*(real|original)[A-Za-z]*\s*\)". Most are probably harmless today because their stubs happen to be shape-complete, but the pattern is a latent version of this bug. I have not touched them here: I only fixed what I could reproduce, and a sweep of that size deserves its own PR.Summary by CodeRabbit