Skip to content

ST_Intersects pushdown: vortex.geo.intersects kernel, DuckDB lowering, spatial function overrides#8704

Open
HarukiMoriarty wants to merge 6 commits into
developfrom
nemo/geo-intersects
Open

ST_Intersects pushdown: vortex.geo.intersects kernel, DuckDB lowering, spatial function overrides#8704
HarukiMoriarty wants to merge 6 commits into
developfrom
nemo/geo-intersects

Conversation

@HarukiMoriarty

Copy link
Copy Markdown
Contributor

Rationale for this change

ST_Intersects over native geometry columns currently evaluates inside DuckDB, exporting every row as GEOMETRY just to be tested. This adds a native vortex.geo.intersects kernel and pushes the filter into the scan: on SpatialBench SF1 (6M points vs a literal polygon), 406ms (DuckDB filter) / 36.6ms (SPATIAL_JOIN) -> 12.4ms pushed. Part of the native geospatial lane, following the ST_Distance/ST_DWithin pushdown.

What changes are included in this PR?

  • vortex-geo: GeoIntersects kernel.
  • vortex-duckdb: lower st_intersects(native column, GEOMETRY literal) like st_distance.
  • vortex-duckdb: shadow spatial's ST_Intersects with a non-fallible copy — DuckDB won't push can-throw filters through the projection every view-registered table has. The ST_DWithin override generalizes into a table-driven registry (spatial_overrides.hpp) shared by registration and the join-condition restore pass.

Note: drop GeoDistance's columnar point fast paths. geo is row-oriented, so kernels materialize rows regardless; hand-rolled columnar paths belong in a future columnar geometry compute library (pushed Q1: 5.4ms -> 12.6ms, still far ahead of unpushed).

@codspeed-hq

codspeed-hq Bot commented Jul 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚡ 4 improved benchmarks
❌ 1 regressed benchmark
✅ 1625 untouched benchmarks
🆕 7 new benchmarks
⏩ 42 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation slice_empty_vortex 339.4 ns 397.8 ns -14.66%
Simulation chunked_varbinview_into_canonical[(100, 100)] 307 µs 271.9 µs +12.91%
Simulation encode_varbin[(1000, 8)] 160.7 µs 144.7 µs +11.05%
Simulation encode_varbin[(1000, 4)] 159 µs 144.2 µs +10.32%
Simulation encode_varbin[(1000, 32)] 164.7 µs 149.5 µs +10.13%
🆕 Simulation ilike_contains N/A 783.5 µs N/A
🆕 Simulation like_contains N/A 155.3 µs N/A
🆕 Simulation like_exact N/A 43 µs N/A
🆕 Simulation like_per_row_patterns N/A 167.3 µs N/A
🆕 Simulation like_prefix N/A 49.8 µs N/A
🆕 Simulation like_regex N/A 622 µs N/A
🆕 Simulation like_suffix N/A 65.8 µs N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing nemo/geo-intersects (840b7ac) with develop (2a58c67)2

Open in CodSpeed

Footnotes

  1. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (d2b2378) during the generation of this report, so 2a58c67 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Comment thread vortex-duckdb/cpp/scalar_fn_pushdown.cpp Outdated
Comment thread vortex-duckdb/cpp/expr.cpp Outdated
@myrrc

myrrc commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Let's add a sqllogictest which checks our queries don't break if we override the functions

…, spatial overrides

vortex-geo gains a GeoIntersects kernel (OGC semantics: not disjoint,
boundary contact counts; the function id is kept stable so a future
stats-pruning rewrite can key on it). Kernels stay thin
materialize-and-delegate wrappers over geo: GeoDistance's columnar point
fast paths are removed as well, since geo computes row by row and
hand-rolled columnar paths belong in a future columnar geometry compute
library (noted in scalar_fn/mod.rs).

vortex-duckdb lowers two-argument st_intersects like st_distance, and
shadows the spatial functions whose registrations block filter pushdown:
st_intersects is marked fallible, which DuckDB will not push through a
view's projection, and st_dwithin folds its radius into bind data. The
overrides live in their own translation unit
(cpp/spatial_overrides.{hpp,cpp}) as a (name, arity, tweak) table shared
by registration and the join-condition restore pass, keeping
scalar_fn_pushdown.* free of spatial code. e2e tests cover every lowered
geo filter on a direct file scan and through a view.

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
@HarukiMoriarty HarukiMoriarty requested a review from myrrc July 10, 2026 14:56

@myrrc myrrc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread vortex-duckdb/cpp/include/expr.h Outdated
Comment thread vortex-duckdb/cpp/spatial_overrides.cpp Outdated
HarukiMoriarty and others added 2 commits July 10, 2026 11:58
The C declaration moves to spatial_overrides.h, which bindgen consumes
alongside the other C API headers, and SpatialOverrideRestore is declared
in spatial_overrides.hpp with its methods implemented in the .cpp,
matching scalar_fn_pushdown's layout.

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
Comment thread vortex-geo/src/scalar_fn/intersects.rs Outdated
Comment thread vortex-geo/src/scalar_fn/intersects.rs Outdated
HarukiMoriarty and others added 2 commits July 10, 2026 13:37
Geometry arrays are never nullable, so GeoIntersects and GeoDistance
validate their operand dtypes in return_dtype, and the DuckDB lowering
no longer pushes nullable native geometry columns. The column length
checks use vortex_ensure_eq.

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants