Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: github/codeql
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 84fc2d3
Choose a base ref
...
head repository: github/codeql
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: bc3b4e1
Choose a head ref
  • 2 commits
  • 14 files changed
  • 1 contributor

Commits on Sep 3, 2026

  1. Python: Add telemetry for parser usage 

    Adds statistics on how many files were extracted using the old parser
and using the tree-sitter parser.

Because parsing is done in parallel across many workers, I opted not to
consolidate these statistics for the entire run. Instead, we emit the
statistics for each worker and then need to aggregate themselves after
the telemetry has been ingested. (In practice the number of workers is
~16 at most, so is unlikely to be an issue.)

In terms of implementation, I opted to simply extend the existing
`DiagnosticsWriter` object (instantiatied once per worker) with methods
for counting the number of parsed files, and then thread this object
through to `modules.py` where the magic happens.

Finally, this also required instantiating such an object in cases where
we call directly into the extractor for debugging purposes (e.g. dumping
the AST or CFG). Note that in these cases we do not actually print any
diagnostics, so it's harmless to create these objects.
    @tausbn
    tausbn committed Sep 3, 2026
    Configuration menu
    Copy the full SHA
    9128443 View commit details
    Browse the repository at this point in the history

  2. Python: Add extractor flag telemetry 

    Records any non-default extractor flags (without their arguments) as a
normalised string. This will enable us to determine which flags are
actually used (and which ones we might therefore get rid of).

When there are no flags other than the ones the autobuilder injects, we
simply report the string `"default"`. That way, there's no need to
remember exactly which flags are enabled by default during extraction.
    @tausbn
    tausbn committed Sep 3, 2026
    Configuration menu
    Copy the full SHA
    bc3b4e1 View commit details
    Browse the repository at this point in the history
Loading