Skip to content

Add weekly AAT and VRT flaky test reporting - #8368

Draft
joshblack with Copilot wants to merge 2 commits into
mainfrom
copilot/add-flaky-test-detector-script
Draft

Add weekly AAT and VRT flaky test reporting#8368
joshblack with Copilot wants to merge 2 commits into
mainfrom
copilot/add-flaky-test-detector-script

Conversation

Copilot AI commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Adds a read-only script that identifies Playwright tests marked flaky across AAT and VRT workflow runs from the past seven days.

Changelog

New

  • Query completed report workflows through the GitHub CLI.
  • Parse ANSI-colored Playwright flaky-test summaries.
  • Aggregate occurrences into a Markdown report with links to recent runs.
  • Support custom repositories, time ranges, and workflow files.
node script/find-flaky-tests.mts --repo primer/react

Changed

  • None.

Removed

  • None.

Rollout strategy

  • Patch release
  • Minor release
  • Major release; if selected, include a written rollout or migration plan
  • None; repository tooling only.

Testing & Reviewing

Review workflow selection, Playwright log parsing, and report aggregation.

Copilot AI and others added 2 commits September 1, 2026 17:02
Co-authored-by: joshblack <3901764+joshblack@users.noreply.github.com>
Co-authored-by: joshblack <3901764+joshblack@users.noreply.github.com>
@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 04a6cd0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@joshblack joshblack added the skip changeset This change does not need a changelog label Sep 1, 2026 — with Copilot SWE Agent
Copilot AI changed the title Add weekly AAT and VRT flaky test report script Add weekly AAT and VRT flaky test reporting Sep 1, 2026
Copilot AI requested a review from joshblack September 1, 2026 17:23
@github-actions github-actions Bot added the integration-tests: recommended This change needs to be tested for breaking changes. See https://arc.net/l/quote/tdmpakpm label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

⚠️ Action required

👋 Hi, this pull request contains changes to the source code that github/github-ui depends on. If you are GitHub staff, test these changes with github/github-ui using the integration workflow. Check the integration testing docs for step-by-step instructions. Or, apply the integration-tests: skipped manually label to skip these checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration-tests: recommended This change needs to be tested for breaking changes. See https://arc.net/l/quote/tdmpakpm skip changeset This change does not need a changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants