Add azure.ai.loom extension for Foundry experiment tracking

[HarshaVardhanBabu (babu-namburi)]

Fixes #9796

Summary

• Add a standalone first-party azure.ai.loom extension with all 18 Foundry experiment-tracking API surfaces under azd ai loom run.
• Organize operations into run inspection, trace, span, ingest, and wandb command groups.
• Preserve the existing azure.ai.projects command and provider behavior by removing experiment commands from that extension.
• Resolve the Foundry project endpoint from an explicit flag, the active azd environment, compatible projects context, or host environment variables.
• Use Azure Developer CLI bearer authentication by default, with optional AZURE_AI_PROJECT_API_KEY authentication.
• Add extension metadata, documentation, CODEOWNERS, lint workflow, Azure DevOps release pipeline wiring, and an end-to-end PowerShell smoke-test script.

CLI hierarchy

azd ai loom run
├── list
├── history-keys
├── summary
├── metrics
├── system-metrics
├── logs
├── log-records
├── compare
├── trace
│   ├── list
│   ├── show
│   └── chat
├── span
│   └── query
├── ingest
│   ├── metrics
│   ├── logs
│   ├── traces
│   └── agent-traces
└── wandb
    ├── graphql
    └── file-stream

Validation

• go build ./... and go test ./... from cli/azd/extensions/azure.ai.loom.
• go test ./... from cli/azd/extensions/azure.ai.projects.
• golangci-lint run ./... and extension cspell checks.
• mage checkDependencyVersions.
• azd x build for azure.ai.loom, including artifact packaging and local installation.
• Exercised all 18 APIs against the fdp-command-job-westus2-proj project using project-key authentication.
• Verified bearer-token request handling through unit tests.

Smoke testing

Run the PowerShell smoke-test script from the Loom extension directory:

.\test-all.ps1 `
  -ProjectEndpoint "https://<account>.services.ai.azure.com/api/projects/<project-id>" `
  -RunId "<run-id>" `
  -SecondRunId "<second-run-id>" `
  -TraceId "<trace-id>"

The script builds and installs the extension, generates temporary synthetic OTLP protobuf, agent-trace, GraphQL, and W&B file-stream payloads, exercises every CLI surface, and removes the fixtures afterward. Use -SkipWriteOperations to run only inspection, trace, and span checks.

Release handoff

This PR adds repository build and release wiring for azure.ai.loom. Publishing the extension to the official registry remains a separate release/registry step.

[azure-pipelines]

Azure Pipelines:
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[microsoft-github-policy-service]

Thank you for your contribution HarshaVardhanBabu (@babu-namburi)! We will review the pull request and get back to you soon.

[Copilot]

Pull request overview

Adds Foundry experiment-tracking operations to the azure.ai.projects extension.

Changes:

• Adds 18 run, ingestion, trace, span, and W&B API commands.
• Adds bearer/API-key authentication and Foundry request handling.
• Adds tests and user documentation.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
README.md	Documents experiment-tracking usage.
internal/exterrors/codes.go	Adds experiment error codes.
internal/experimenttracking/client.go	Implements the authenticated HTTP client.
internal/experimenttracking/client_test.go	Tests client behavior.
internal/cmd/root.go	Registers new command groups.
internal/cmd/experiment.go	Implements experiment commands.
internal/cmd/experiment_test.go	Tests command helpers and registration.
docs/environment-variables.md	Documents API-key authentication.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[HarshaVardhanBabu (babu-namburi)]

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 3 comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.loom/internal/cmd/project_endpoint.go:28

• url.Parse errors include the original URL, so a malformed endpoint containing user info or a sensitive query value can echo that secret in this structured error. Return a generic parse failure here; the existing suggestion already tells the user how to correct it.

			fmt.Sprintf("invalid project endpoint URL: %s", err),

cli/azd/extensions/azure.ai.loom/internal/cmd/run.go:337

• strconv.ParseFloat accepts NaN and infinities, and each bypasses this ordering check. The request then always fails in json.Marshal as an unsupported value and is reported as an internal request error; reject non-finite --min/--max values as invalid parameters before building the body.

			if maxStep < minStep {
				return invalidExperimentParameter("max", "--max must be greater than or equal to --min")
			}

[HarshaVardhanBabu (babu-namburi)]

Addressed the latest review feedback in c067118:

• disabled redirects for authenticated Loom HTTP clients to prevent credential forwarding;
• documented the host-shell AZURE_AI_PROJECT_ENDPOINT compatibility fallback;
• added command-level HTTP contract coverage for all 18 API surfaces;
• made malformed endpoint parse errors non-disclosing; and
• rejected NaN and infinite comparison bounds before JSON serialization.

[Copilot]

🟡 Changes recommended

Endpoint path safety and cancellation classification issues remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.loom/internal/exterrors/errors.go:72

• A canceled request reaches this fallback and is converted to an internal structured error, which removes the context.Canceled chain and makes Ctrl+C appear as an unexpected failure. Check cancellation before this fallback and return a user-cancellation error, as azure.ai.routines/internal/exterrors/errors.go:93-95 does.

• Files reviewed: 32/33 changed files
• Comments generated: 2
• Review effort level: Balanced

[HarshaVardhanBabu (babu-namburi)]

Also addressed the suppressed cancellation-classification finding in f877751. Wrapped context and gRPC cancellations now become user/cancelled errors instead of internal failures, and cancellation is checked before the access-token authentication heuristic so Ctrl+C during credential acquisition is not misreported as an authentication failure.

[Copilot]

🟡 Changes recommended

Optional comparison bounds and the ingestion timeout can cause valid operations to return incomplete results or fail.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.loom/internal/experimenttracking/client.go:28

• This timeout covers the entire upload and response, while ingestion accepts payloads up to 64 MiB. An allowed payload needs roughly 18 Mbit/s just to upload within 30 seconds, before server processing, so valid ingests can reliably time out on slower links. Use a longer/operation-specific timeout or align the accepted payload limit with this deadline.
  cli/azd/extensions/azure.ai.projects/internal/exterrors/codes.go:43
• This exported error code has no call site in azure.ai.projects; authentication failures for the moved experiment feature are defined and used in azure.ai.loom instead. Remove this orphaned constant so the projects extension remains unchanged and does not expose a dead telemetry code.

cli/azd/extensions/azure.ai.loom/internal/cmd/run.go:951

• Go map iteration order is unspecified, so commands with multiple missing required flags can report a different flag on different executions. Accept an ordered slice/variadic sequence, or validate each required flag explicitly so CLI errors are deterministic.

func requireValues(values map[string]string) error {
	for name, value := range values {
		if err := requireValue(name, value); err != nil {
			return err
		}

• Files reviewed: 33/34 changed files
• Comments generated: 1
• Review effort level: Balanced

[HarshaVardhanBabu (babu-namburi)]

Addressed the three suppressed findings in 64eb3fd:

• protobuf, agent-trace JSON, and W&B file-stream ingestion now use a five-minute upload timeout while ordinary API operations retain the 30-second timeout;
• required flags are validated explicitly in deterministic command order; and
• the unused CodeAuthenticationFailed constant was removed from azure.ai.projects.

[Copilot]

🔵 Needs a closer look

The new authenticated data-plane client, write operations, and release pipeline warrant final human validation.

Review details

• Files reviewed: 32/33 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[HarshaVardhanBabu (babu-namburi)]

Fixed the failing Loom golangci-lint gate in commit 3f58ca4 by removing the unused Client.do wrapper left by the ingestion-timeout refactor.

[Copilot]

🟡 Changes recommended

Error responses currently bypass the response-size limit and can cause excessive memory consumption.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 32/33 changed files
• Comments generated: 1
• Review effort level: Balanced

[HarshaVardhanBabu (babu-namburi)]

Completed an additional full-PR multi-lens review and addressed the remaining high-confidence findings in 13f8b0b:

• token acquisition now uses a typed sentinel, preventing server-controlled response text from being misclassified as a local authentication failure;
• host-shell endpoint fallbacks remain usable when persisted azd context cannot be read, while the original source error is still returned when no fallback exists;
• trace chat and span query now reject explicit flags that a complete --request-file body would otherwise ignore, including explicitly supplied empty, false, and default values;
• error response bodies are bounded before Azure response-error parsing.

The complete PR was reviewed across security, Go correctness, Azure/API behavior, CLI UX, testing, documentation, architecture, CI/release wiring, identity, and OTLP/observability concerns. The exact CI golangci-lint v2.11.4 configuration reports zero issues.

[Copilot]

🔵 Needs a closer look

The broad authenticated data-plane and ingestion surface warrants final human validation despite substantial automated coverage.

Review details

• Files reviewed: 32/33 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Environment selection, repeatable-value validation, service attribution, and lint enforcement need correction.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

cli/azd/extensions/azure.ai.loom/internal/cmd/run.go:342

• The repeatable flags are checked only by count, so two whitespace-only --run-id values or one whitespace-only --metric value pass validation and produce a malformed comparison request. Validate every entry before building the body.

			if len(runIDs) < 2 {
				return invalidExperimentParameter("run-id", "provide at least two run IDs")
			}
			if len(metricNames) == 0 {
				return invalidExperimentParameter("metric", "provide at least one metric name")

• Files reviewed: 32/33 changed files
• Comments generated: 4
• Review effort level: Balanced