[DE-8678] Run-free ("model v2") predictions on Model (#479)

* feat(model): add Model.model_runs() to list a model's run ids

The model-scoped counterpart to Dataset.model_runs(): lists every run under a
model via GET /nucleus/model/:modelId/modelRun. include_versions=True unions runs
across the model's version lineage (?family=true). Results are dataset-scoped
server-side. Adds a mock unit test, CHANGELOG entry, and version bump to 0.21.3.

Requires the matching scaleapi route (scaleapi#158386); live calls 404 until it deploys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(model): add run-free ("model v2") predictions [DE-8678]

Introduce a run-free prediction concept where predictions are tied directly
to a Model as (model, dataset_item) -> prediction, with no ModelRun or
Dataset. Purely additive: all existing model-run prediction paths are
unchanged.

- Model.upload_predictions(...) — upsert predictions onto model/{id}/predictions
  (sync + async), reusing the PredictionUploader batching machinery.
- Model.predictions_loc / predictions_refloc / predictions_iloc — model-scoped reads.
- Model.copy_predictions_from_run(model_run_id) — backfill from a v1 run (AsyncJob).
- create_benchmark_evaluation_v2 gains an optional model_id anchor (accepts a
  prj_* id or a Model) as an alternative to model_run_id; exactly one required.
- EvaluationV2 gains an optional model_id field; model_run_id now optional.
- serialize_and_write_to_presigned_url gains route_prefix for the model route.
- Docs + CHANGELOG; minor version bump to 0.22.0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(model): make v2 prediction upload + copy sync-only to match backend [DE-8678]

The live scaleapi backend for DE-8678 is synchronous-only for these routes:

- Model.upload_predictions: the model route has no async/signed-URL endpoint
  (?async=1 returns HTTP 400). Remove the assumed signed-URL async flow and
  raise NotImplementedError when asynchronous=True; keep the sync path as-is.
  Revert the now-unused route_prefix param added to
  serialize_and_write_to_presigned_url in nucleus/utils.py.
- Model.copy_predictions_from_run: the backend runs synchronously and returns
  {model_id, model_run_ids, predictions_copied, predictions_skipped_unsupported}.
  Return that dict directly (drop the AsyncJob wrapping and the unused
  asynchronous param); note it's synchronous in the docstring.

CHANGELOG updated to match; still additive, still v0.22.0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(model): parse run-free prediction reads in format_prediction_response [DE-8678]

Model.predictions_loc / predictions_refloc / predictions_iloc were shipped
non-functional: the run-free read endpoints return a flat {"predictions": [...]}
list (each element carrying its own "type"), but format_prediction_response only
understood the legacy type-keyed {"annotations": {"box": [...]}} shape. It fell
through to the "an error occurred" branch and returned the raw payload unparsed,
so these reads yielded the raw dict instead of the documented
{"box": [...], "polygon": [...], "cuboid": [...]}.

Add a flat-list branch that groups predictions by their per-element "type" into
that same shape. Legacy type-keyed reads and the error/empty case are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* remove dataset id refs

* deprecate model runs

* depracte allowed label matches

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

chore: bump pyproject version to 0.21.2 to match CHANGELOG + v0.21.2 …

…tag (#477)

The v0.21.2 tag never published to PyPI: the CircleCI pypi_publish job
validates CIRCLE_TAG == v${pyproject version}, but pyproject stayed at
0.21.0 while the tag and CHANGELOG were already 0.21.2, so the job exits 1
before poetry publish. Bump the version so a v0.21.2 tag build validates and
publishes. Also correct the 0.21.2 CHANGELOG release link (v0.21.0 -> v0.21.2).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

feat: async + multi-source create_benchmark (#470)

* feat: make create_benchmark asynchronous (poll build job)

The server now creates a benchmark in a 'building' state and streams its
members in via a background job, responding 202 with {benchmark_id, job_id}
(this removes the previous item-count ceiling on slice/dataset-sourced
benchmarks). Update create_benchmark to match:

- POST, then by default poll the build job to completion (reusing AsyncJob)
  and return the ready benchmark. Return type is unchanged, so existing
  blocking callers are unaffected; a failed build raises JobError.
- Add wait_for_completion (default True) to return the 'building' benchmark
  immediately for callers that want to poll themselves.
- Benchmark now exposes `status` ('building' | 'ready' | 'failed').

Bumps to 0.20.0. Stacked on #467.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: multi-source create_benchmark (slice_ids/dataset_ids, combinable)

Mirror the server's multi-source benchmark creation: create_benchmark now
accepts plural slice_ids and dataset_ids alongside the singular slice_id/
dataset_id/item_ids/items, and members from all provided sources are unioned
and de-duplicated server-side. Requirement relaxed from "exactly one source"
to "at least one source".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Update nucleus/__init__.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(benchmarks): restore the job_id lookup dropped by the guard commit

9f5ffd4 added the 'server returned no job_id' guard but deleted the
assignment it reads, so every wait_for_completion=True create_benchmark
raised NameError instead of polling.

* P1

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

[DE-8304] Deprecate synchronous upload capability (#468)

* Deprecate synchronous upload capability

* Disallow mixed file upload types in one call

* Address greptile

* Update tests that still use sync upload

* Remove sync append in deduplication test

* Update add_items_from_dir to be async only

* Fix test_create_update_dataset_from_dir

* Remove deprecated asynchronous=True flag from test files

* Update CLAUDE.md

Fix cibuildwheel build package shadowing (#465)

* Fix cibuildwheel build package shadowing

* Refresh Poetry lockfile

[codex] Add native pHash deduplication acceleration (#464)

* Add native pHash deduplication acceleration

* Name candidate mark states in native dedup index.

Replace magic 0/1/2 literals with a documented CandidateMark enum so Hamming-index query state is easier to follow.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address native dedup review nits

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

Bump version for local pHash dedup release (#463)

[DE-7859] Expose pHash on DatasetItem (v0.18.3) (#461)

* [DE-7859] Expose pHash on DatasetItem (v0.18.3)

Add a `phash` field to the DatasetItem dataclass and thread it through
`from_json`. Because every SDK method that returns a DatasetItem
(items_and_annotation_generator, items_generator, query_items,
dataset.items, iloc/refloc/loc) deserializes through DatasetItem.from_json,
exposing the field there is sufficient — no per-method changes required.

Also adds a top-level CLAUDE.md with release/branch conventions and
architecture pointers for future Claude Code sessions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Tighten phash field comment

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Loosen test_dataset_append_async — don't pin to step counts

The upload job pipeline plans with N total_steps initially, then
dynamically collapses to a single step once it knows how to
short-circuit (small input → batched upload). By the time
sleep_until_complete() returns, status() always reports total_steps=1,
completed_steps=1 — so the hard-coded expectation of 5/5 deterministically
fails on the current backend.

Drop the step-count assertions and keep the meaningful invariants:
job completed successfully, progress is 1.00, and
completed_steps == total_steps (whatever they are).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix flaky dedup tests: compare unique_item_ids as sets

`test_deduplicate_*_by_ids` runs dedup over the surviving set returned
from a prior dedup and asserts the second result equals the first.
The set of survivors is well-defined, but the backend doesn't guarantee
a stable list order across runs — the "kept" list depends on the order
in which the deduplication loop visits items, and that order can differ
between the whole-dataset (cursor-paginated) and by-ids (batched-by-input)
code paths.

Asserting list equality therefore fails intermittently when the same
items come back in a different order. Switch all four call sites
(image / video-scene / video-url / by-ids-returns-job) to set comparison.
The other invariants (length, `original_count`) still hold.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Exclude phash from DatasetItem __eq__

Adding `phash` as a regular dataclass field made every `item1 == item2`
comparison sensitive to whether the backend had populated the hash —
which it doesn't on every endpoint (some handlers cherry-pick columns
and exclude phash, others select all columns and include it). Tests
that constructed a DatasetItem locally and then compared it to the
backend round-trip (test_append_and_export, test_slice_dataset_item_iterator)
broke as a result.

phash is a derived value (computed from image_location), so two items
with the same source image should compare equal regardless of whether
their hashes happen to be populated. Mark the field `compare=False` so
auto-generated __eq__ ignores it, matching the natural semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test_dataset_tags: poll fresh get_tags() instead of asserting on remove_tags response

The DELETE /tags handler refetches the tag list immediately after the
delete and returns it. In prod that refetch can hit a read replica that
hasn't yet replayed the DELETE, so the response includes the just-deleted
tag — making the test fail. A separate follow-up request always sees the
correct state (verified against api.scale.com — first poll is already
consistent at ~25ms round-trip).

Tighten the test against the post-state by polling get_tags() with a 5s
settle window, rather than trusting the remove_tags response. Same change
applied to the idempotent-remove follow-up assertion. Backend deferred —
the inconsistency is bounded and not user-impacting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Add dataset tags to SDK for identification (DE-7033) (#456)

Expose dataset tags through the Python SDK so customers can identify
datasets labeled by Scale vs other vendors via the API.

- Add `tags` field to DatasetInfo model (returned by dataset.info())
- Add get_tags(), add_tags(), remove_tags() methods to Dataset class
- Use POST /tags/remove instead of DELETE to avoid proxy body-stripping
- Use pydantic v1/v2 compat shim for null-coercion validator
- Guard against passing a bare string instead of a list

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

[DE-7784] Migrate sdk image deduplication to async mode (#459)

* Add support for async dedup

* Update sphinx package versions

* Update sdk version to 0.18.1

* Remove support for sync dedup in sdk

* Updater CHANGELOG

* Address greptile