Skip to content

feat(cli): embed core pack in wheel for offline/air-gapped deployment#1803

Open
mnriem wants to merge 2 commits intogithub:mainfrom
mnriem:fix/offline-install-1752
Open

feat(cli): embed core pack in wheel for offline/air-gapped deployment#1803
mnriem wants to merge 2 commits intogithub:mainfrom
mnriem:fix/offline-install-1752

Conversation

@mnriem
Copy link
Collaborator

@mnriem mnriem commented Mar 11, 2026

Summary

Closes #1711
Addresses #1752

Embeds templates, commands, and scripts inside the specify-cli Python wheel so that specify init works with zero network access by default. The pre-built wheel is also published as a GitHub release asset, providing a clean installation path for enterprise/air-gapped environments.


Problem

Two related air-gapped blockers were addressed together:

  1. Cannot install CLI (Unable to install CLI as wheel access is blocked by corp policy #1752) — PyPI is blocked (403); no .whl was available as a release asset
  2. Cannot run specify init (feat(cli): Embed core template pack in CLI package for air-gapped deployment #1711) — api.github.com is blocked; the init command unconditionally calls download_template_from_github() to fetch a release ZIP

A user who solved problem 1 (offline pip install) would immediately hit problem 2 on first use.


Solution

1. Bundle core assets in the wheel (pyproject.toml)

[tool.hatch.build.targets.wheel.force-include]
"templates"           = "specify_cli/core_pack/templates"
"templates/commands"  = "specify_cli/core_pack/commands"
"scripts/bash"        = "specify_cli/core_pack/scripts/bash"
"scripts/powershell"  = "specify_cli/core_pack/scripts/powershell"

2. New Python functions in __init__.py

  • _locate_core_pack() — finds bundled assets (wheel install) or falls back to repo-root trees (source checkout / editable install)
  • _generate_agent_commands() — Python port of generate_commands() from create-release-packages.sh: YAML frontmatter parsing, {SCRIPT} / {AGENT_SCRIPT} / {ARGS} / __AGENT__ substitution, path rewriting, Markdown / TOML / agent.md output formats, Copilot .prompt.md companion files
  • scaffold_from_core_pack() — copies scripts + page templates and generates agent command files, no network calls

3. specify init is now offline-first

init() detects bundled assets and uses scaffold_from_core_pack() by default. The GitHub download path is retained via a new --from-github flag:

# Default — works air-gapped:
specify init my-project --ai claude

# Opt-in to latest GitHub release:
specify init my-project --ai claude --from-github

If scaffolding unexpectedly fails, it automatically falls back to the GitHub download.

4. Wheel published as release asset

  • release.yml: build step added (python -m build --wheel)
  • create-github-release.sh: specify_cli-VERSION-py3-none-any.whl attached to every release

5. Documentation

  • docs/installation.md: new "Enterprise / Air-Gapped Installation" section
  • README.md: "Option 3: Enterprise / Air-Gapped Installation"

Acceptance criteria from #1711

Criterion Status
specify init scaffolds from embedded assets with no network calls
All supported agents produce correct command files (Markdown, TOML, agent.md) ✅ (_generate_agent_commands covers all formats)
specify init --from-github retains current GitHub-download behavior
pip install specify-cli includes all core templates, commands, and scripts ✅ (force-include in pyproject.toml)
Existing create-release-packages.sh continues to work ✅ (unchanged)
Air-gapped deployment works end-to-end ✅ (install wheel offline → specify init offline)

Testing

158 passed in 4.87s

All existing tests pass unchanged.

…github#1752)

Bundle templates, commands, and scripts inside the specify-cli wheel so
that `specify init` works without any network access by default.

Changes:
- pyproject.toml: add hatchling force-include for core_pack assets; bump
  version to 0.2.1
- __init__.py: add _locate_core_pack(), _generate_agent_commands() (Python
  port of generate_commands() shell function), and scaffold_from_core_pack();
  modify init() to scaffold from bundled assets by default; add --from-github
  flag to opt back in to the GitHub download path
- release.yml: build wheel during CI release job
- create-github-release.sh: attach .whl as a release asset
- docs/installation.md: add Enterprise/Air-Gapped Installation section
- README.md: add Option 3 enterprise install with accurate offline story

Closes github#1711
Addresses github#1752
Copilot AI review requested due to automatic review settings March 11, 2026 14:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes specify init work offline by bundling the core template pack (templates/commands/scripts) inside the specify-cli wheel, and updates the release workflow/docs to support air-gapped installation via a published .whl release asset.

Changes:

  • Bundle core templates/commands/scripts into the wheel and scaffold from those assets by default (with --from-github to force network download).
  • Add runtime generation of agent-specific command files (md/toml/agent.md + Copilot prompt companions).
  • Publish the wheel as a GitHub release asset and document enterprise/air-gapped install steps.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/specify_cli/__init__.py Adds core-pack discovery, offline scaffolding, and agent command generation; updates init to be offline-first with --from-github.
pyproject.toml Bumps version and force-includes core assets into the wheel.
docs/installation.md Adds enterprise/air-gapped installation instructions and offline init guidance.
README.md Documents the new air-gapped installation option via wheel.
CHANGELOG.md Notes offline-first init, --from-github, and wheel release asset.
.github/workflows/scripts/create-github-release.sh Attaches the built wheel to GitHub releases.
.github/workflows/release.yml Adds a wheel build step to the release workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +983 to +995
def _locate_core_pack() -> Path | None:
"""Return the filesystem path to the bundled core_pack directory.
Works for wheel installs (hatchling force-include puts the directory next to
__init__.py as specify_cli/core_pack/) and for source-checkout / editable
installs (falls back to the repo-root templates/ and scripts/ trees).
Returns None only when neither location exists.
"""
# Wheel install: core_pack is a sibling directory of this file
candidate = Path(__file__).parent / "core_pack"
if candidate.is_dir():
return candidate
return None
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_locate_core_pack()’s docstring says it also “falls back to the repo-root templates/ and scripts/ trees”, but the implementation only checks for specify_cli/core_pack and otherwise returns None. Either update the docstring to reflect what the function actually does, or implement the repo-root fallback inside this helper (so callers don’t need to duplicate fallback logic).

Copilot uses AI. Check for mistakes.
# Extract script command for this script variant
scripts_section = frontmatter.get("scripts") or {}
script_command = str(scripts_section.get(script_type, "")).strip()
if not script_command:
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a command template is missing a scripts.{sh|ps} entry, this generator silently substitutes a placeholder string into the output (and continues). In the release packaging script this condition emits a warning, which helps catch broken templates early. Consider surfacing this as a warning/error via tracker/console (or raising) so users don’t end up with generated commands that contain “(missing script command …)” without any explanation.

Suggested change
if not script_command:
if not script_command:
# Surface missing script commands as a warning so users can fix
# broken templates instead of silently generating placeholders.
Console().print(
f"[yellow]Warning:[/yellow] Template '{template_file.name}' is missing "
f"a 'scripts.{script_type}' entry; using placeholder command.",
style="yellow",
)

Copilot uses AI. Check for mistakes.
Comment on lines +35 to +40
- name: Build Python wheel
if: steps.check_release.outputs.exists == 'false'
run: |
pip install build
python -m build --wheel --outdir .genreleases/

Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step builds the wheel into .genreleases/, but the next step (create-release-packages.sh) clears that directory at startup (rm -rf "$GENRELEASES_DIR"/*). As a result, the wheel will be deleted before create-github-release.sh tries to attach it, causing releases to fail or omit the wheel. Fix by either building the wheel after the release packages step, outputting the wheel to a different directory, or updating the packaging script to preserve existing artifacts.

See below for a potential fix:

      - name: Create release package variants
        if: steps.check_release.outputs.exists == 'false'
        run: |
          chmod +x .github/workflows/scripts/create-release-packages.sh
          .github/workflows/scripts/create-release-packages.sh ${{ steps.version.outputs.tag }}

      - name: Build Python wheel
        if: steps.check_release.outputs.exists == 'false'
        run: |
          pip install build
          python -m build --wheel --outdir .genreleases/

Copilot uses AI. Check for mistakes.
full_content = re.sub(r"/?scripts/", ".specify/scripts/", full_content)
full_content = re.sub(r"/?templates/", ".specify/templates/", full_content)
# Fix any accidental double-prefix introduced by the substitution
full_content = full_content.replace(".specify/.specify/", ".specify/")
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The double-prefix cleanup here doesn’t match the behavior in create-release-packages.sh: the shell version fixes .specify.specify/ (no slash between) but this code only replaces .specify/.specify/. With the current regex rewrites above, already-prefixed paths like .specify/scripts/... can turn into .specify.specify/scripts/... and won’t be corrected. Update the cleanup to match the shell rewrite (.specify.specify/.specify/).

Suggested change
full_content = full_content.replace(".specify/.specify/", ".specify/")
full_content = full_content.replace(".specify.specify/", ".specify/")

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(cli): Embed core template pack in CLI package for air-gapped deployment

2 participants