feat(cli): embed core pack in wheel for offline/air-gapped deployment#1803
feat(cli): embed core pack in wheel for offline/air-gapped deployment#1803mnriem wants to merge 2 commits intogithub:mainfrom
Conversation
…github#1752) Bundle templates, commands, and scripts inside the specify-cli wheel so that `specify init` works without any network access by default. Changes: - pyproject.toml: add hatchling force-include for core_pack assets; bump version to 0.2.1 - __init__.py: add _locate_core_pack(), _generate_agent_commands() (Python port of generate_commands() shell function), and scaffold_from_core_pack(); modify init() to scaffold from bundled assets by default; add --from-github flag to opt back in to the GitHub download path - release.yml: build wheel during CI release job - create-github-release.sh: attach .whl as a release asset - docs/installation.md: add Enterprise/Air-Gapped Installation section - README.md: add Option 3 enterprise install with accurate offline story Closes github#1711 Addresses github#1752
There was a problem hiding this comment.
Pull request overview
This PR makes specify init work offline by bundling the core template pack (templates/commands/scripts) inside the specify-cli wheel, and updates the release workflow/docs to support air-gapped installation via a published .whl release asset.
Changes:
- Bundle core templates/commands/scripts into the wheel and scaffold from those assets by default (with
--from-githubto force network download). - Add runtime generation of agent-specific command files (md/toml/agent.md + Copilot prompt companions).
- Publish the wheel as a GitHub release asset and document enterprise/air-gapped install steps.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/__init__.py |
Adds core-pack discovery, offline scaffolding, and agent command generation; updates init to be offline-first with --from-github. |
pyproject.toml |
Bumps version and force-includes core assets into the wheel. |
docs/installation.md |
Adds enterprise/air-gapped installation instructions and offline init guidance. |
README.md |
Documents the new air-gapped installation option via wheel. |
CHANGELOG.md |
Notes offline-first init, --from-github, and wheel release asset. |
.github/workflows/scripts/create-github-release.sh |
Attaches the built wheel to GitHub releases. |
.github/workflows/release.yml |
Adds a wheel build step to the release workflow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def _locate_core_pack() -> Path | None: | ||
| """Return the filesystem path to the bundled core_pack directory. | ||
| Works for wheel installs (hatchling force-include puts the directory next to | ||
| __init__.py as specify_cli/core_pack/) and for source-checkout / editable | ||
| installs (falls back to the repo-root templates/ and scripts/ trees). | ||
| Returns None only when neither location exists. | ||
| """ | ||
| # Wheel install: core_pack is a sibling directory of this file | ||
| candidate = Path(__file__).parent / "core_pack" | ||
| if candidate.is_dir(): | ||
| return candidate | ||
| return None |
There was a problem hiding this comment.
_locate_core_pack()’s docstring says it also “falls back to the repo-root templates/ and scripts/ trees”, but the implementation only checks for specify_cli/core_pack and otherwise returns None. Either update the docstring to reflect what the function actually does, or implement the repo-root fallback inside this helper (so callers don’t need to duplicate fallback logic).
| # Extract script command for this script variant | ||
| scripts_section = frontmatter.get("scripts") or {} | ||
| script_command = str(scripts_section.get(script_type, "")).strip() | ||
| if not script_command: |
There was a problem hiding this comment.
When a command template is missing a scripts.{sh|ps} entry, this generator silently substitutes a placeholder string into the output (and continues). In the release packaging script this condition emits a warning, which helps catch broken templates early. Consider surfacing this as a warning/error via tracker/console (or raising) so users don’t end up with generated commands that contain “(missing script command …)” without any explanation.
| if not script_command: | |
| if not script_command: | |
| # Surface missing script commands as a warning so users can fix | |
| # broken templates instead of silently generating placeholders. | |
| Console().print( | |
| f"[yellow]Warning:[/yellow] Template '{template_file.name}' is missing " | |
| f"a 'scripts.{script_type}' entry; using placeholder command.", | |
| style="yellow", | |
| ) |
| - name: Build Python wheel | ||
| if: steps.check_release.outputs.exists == 'false' | ||
| run: | | ||
| pip install build | ||
| python -m build --wheel --outdir .genreleases/ | ||
|
|
There was a problem hiding this comment.
This step builds the wheel into .genreleases/, but the next step (create-release-packages.sh) clears that directory at startup (rm -rf "$GENRELEASES_DIR"/*). As a result, the wheel will be deleted before create-github-release.sh tries to attach it, causing releases to fail or omit the wheel. Fix by either building the wheel after the release packages step, outputting the wheel to a different directory, or updating the packaging script to preserve existing artifacts.
See below for a potential fix:
- name: Create release package variants
if: steps.check_release.outputs.exists == 'false'
run: |
chmod +x .github/workflows/scripts/create-release-packages.sh
.github/workflows/scripts/create-release-packages.sh ${{ steps.version.outputs.tag }}
- name: Build Python wheel
if: steps.check_release.outputs.exists == 'false'
run: |
pip install build
python -m build --wheel --outdir .genreleases/
| full_content = re.sub(r"/?scripts/", ".specify/scripts/", full_content) | ||
| full_content = re.sub(r"/?templates/", ".specify/templates/", full_content) | ||
| # Fix any accidental double-prefix introduced by the substitution | ||
| full_content = full_content.replace(".specify/.specify/", ".specify/") |
There was a problem hiding this comment.
The double-prefix cleanup here doesn’t match the behavior in create-release-packages.sh: the shell version fixes .specify.specify/ (no slash between) but this code only replaces .specify/.specify/. With the current regex rewrites above, already-prefixed paths like .specify/scripts/... can turn into .specify.specify/scripts/... and won’t be corrected. Update the cleanup to match the shell rewrite (.specify.specify/ → .specify/).
| full_content = full_content.replace(".specify/.specify/", ".specify/") | |
| full_content = full_content.replace(".specify.specify/", ".specify/") |
Summary
Closes #1711
Addresses #1752
Embeds templates, commands, and scripts inside the
specify-cliPython wheel so thatspecify initworks with zero network access by default. The pre-built wheel is also published as a GitHub release asset, providing a clean installation path for enterprise/air-gapped environments.Problem
Two related air-gapped blockers were addressed together:
.whlwas available as a release assetspecify init(feat(cli): Embed core template pack in CLI package for air-gapped deployment #1711) —api.github.comis blocked; the init command unconditionally callsdownload_template_from_github()to fetch a release ZIPA user who solved problem 1 (offline pip install) would immediately hit problem 2 on first use.
Solution
1. Bundle core assets in the wheel (
pyproject.toml)2. New Python functions in
__init__.py_locate_core_pack()— finds bundled assets (wheel install) or falls back to repo-root trees (source checkout / editable install)_generate_agent_commands()— Python port ofgenerate_commands()fromcreate-release-packages.sh: YAML frontmatter parsing,{SCRIPT}/{AGENT_SCRIPT}/{ARGS}/__AGENT__substitution, path rewriting, Markdown / TOML /agent.mdoutput formats, Copilot.prompt.mdcompanion filesscaffold_from_core_pack()— copies scripts + page templates and generates agent command files, no network calls3.
specify initis now offline-firstinit()detects bundled assets and usesscaffold_from_core_pack()by default. The GitHub download path is retained via a new--from-githubflag:If scaffolding unexpectedly fails, it automatically falls back to the GitHub download.
4. Wheel published as release asset
release.yml: build step added (python -m build --wheel)create-github-release.sh:specify_cli-VERSION-py3-none-any.whlattached to every release5. Documentation
docs/installation.md: new "Enterprise / Air-Gapped Installation" sectionREADME.md: "Option 3: Enterprise / Air-Gapped Installation"Acceptance criteria from #1711
specify initscaffolds from embedded assets with no network calls_generate_agent_commandscovers all formats)specify init --from-githubretains current GitHub-download behaviorpip install specify-cliincludes all core templates, commands, and scriptsforce-includein pyproject.toml)create-release-packages.shcontinues to workspecify initoffline)Testing
All existing tests pass unchanged.