From 24f23e09ae8c9a1646c196c3e783f7fd11435d77 Mon Sep 17 00:00:00 2001 From: "SREDevOps.org Agent" Date: Fri, 30 Oct 2026 12:31:50 -0300 Subject: [PATCH] docs: align readmes, docs and agent guidance with current behavior (#37) Remove obsolete layout/migration prose and legacy gateway-alias references, reconcile the required-check policy, fix version-label/RBAC/workspace/channel drift, and de-duplicate the example and gateway docs. Rename the minimal manifest to 10-sandboxtemplate.yaml and mount the dsh-home PVC's workspace subPath at /workspace so the documented persistence is real. Enforce numbered top-level manifests in check-examples.sh. Verified with make check, yamllint, shellcheck, a Markdown link/anchor scan, and git diff --check. Co-authored-by: SREDevOpsAgents --- .../dsh-agent-sandbox-platform/SKILL.md | 11 +- .env.sample | 27 +-- .github/ISSUE_TEMPLATE/bug_report.yml | 2 +- .github/pull_request_template.md | 3 +- .github/workflows/ci-manifests.yaml | 5 +- .github/workflows/ci-python.yaml | 3 +- .github/workflows/release.yaml | 27 --- AGENTS.md | 36 +-- CONTRIBUTING.md | 22 +- README.md | 29 +-- SECURITY.md | 2 +- cliff.toml | 2 +- docs/README.md | 5 +- docs/examples.md | 1 + docs/gitflow.md | 45 ++-- docs/main-image.md | 21 +- docs/versioning.md | 32 +-- examples/README.md | 43 ++-- examples/dsh-agents-as-a-service/README.md | 219 ++++++------------ .../docs/architecture.md | 5 +- .../docs/optional-hardening.md | 6 +- .../dsh-agents-as-a-service/gateway/README.md | 10 +- .../manifests/10-sandboxtemplate.yaml | 4 +- examples/minimal-single-tenant/README.md | 23 +- ...-template.yaml => 10-sandboxtemplate.yaml} | 8 + scripts/check-examples.sh | 7 + scripts/check-versions.sh | 2 +- 27 files changed, 228 insertions(+), 372 deletions(-) rename examples/minimal-single-tenant/manifests/{sandbox-template.yaml => 10-sandboxtemplate.yaml} (93%) diff --git a/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md b/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md index 42abf74..42b6562 100644 --- a/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md +++ b/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md @@ -126,13 +126,14 @@ Read `docs/examples.md` and the testing table in `AGENTS.md` before editing. alternatives, never a stack. - **New example:** follow `docs/examples.md`, add the index entry, keep scripts executable/idempotent/self-cleaning, run `make check`. -- **Version bump:** edit `versions.env` only, update prerequisites, run - `make check` and the e2e. Never mix a bump with an unrelated feature. +- **Version bump:** use `make set-version`; never edit pins by hand. Update + prerequisites, run `make check` and the e2e on a cluster. Never mix a bump + with an unrelated feature. - **MCP integration:** pin the build to the cluster's tag, least-privilege SA, ClusterIP + auth proxy, raised DSH timeout, no secrets in the repo. -- **Optional hardening:** gVisor, `nodeSelector`, NetworkPolicy and extra - provider keys stay opt-in; the sandbox policy lives under - `manifests/optional/`. All four are documented in +- **Optional hardening:** gVisor, `nodeSelector`, NetworkPolicy, extra + provider keys and failed-Pod recovery stay opt-in; the sandbox policy lives + under `manifests/optional/`. All of them are documented in `examples/dsh-agents-as-a-service/docs/optional-hardening.md`. - **Docs:** update the affected doc in the same change. diff --git a/.env.sample b/.env.sample index 5b29cf1..289c792 100644 --- a/.env.sample +++ b/.env.sample @@ -1,4 +1,6 @@ -# Copy to .env and fill in. Never commit a real key. +# Reference environment names for local runs. Nothing sources this file +# automatically: export the values or pass them with `docker run -e/--env-file`. +# Copy it to .env if you prefer --env-file. Never commit a real key. # # The version values below are documentation only: versions.env is the single # source of truth, and scripts/check-versions.sh fails if they drift. @@ -29,22 +31,7 @@ DSH_PORT=3080 # public hostname). localhost and 127.0.0.1 are always trusted. DSH_TRUSTED_HOSTS= -# --- agents-as-a-service example --- -NAMESPACE=dsh-demo -POOL=dsh-pool -# Seconds a sandbox may stay idle before the sweeper suspends it, and the max -# time a proxied request waits for a suspended sandbox to become Ready. -IDLE_TIMEOUT=360 -WAKE_TIMEOUT=240 -DSH_CONTAINER=dsh-web -DSH_LOG_TAIL_LINES=2000 -COOKIE_NAME=dsh_gw_session -# Set to true only behind HTTPS; secure cookies are not sent over plain HTTP -# (localhost counts as a secure context in browsers). -COOKIE_SECURE=false -COOKIE_MAX_AGE=604800 -MAX_BODY_MB=32 -HTTP_TIMEOUT=300 -# SSO is opt-in. Set this only when the gateway Service is reachable -# exclusively through oauth2-proxy (see the example exposure docs). -# AUTH_PROXY_HEADER=X-Forwarded-User +# The example gateway's runtime settings (NAMESPACE, POOL, IDLE_TIMEOUT, +# WAKE_TIMEOUT, COOKIE_*, AUTH_PROXY_HEADER, ...) are documented in +# examples/dsh-agents-as-a-service/gateway/README.md and set in +# examples/dsh-agents-as-a-service/manifests/50-gateway.yaml. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 383a90d..02ce4ba 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -28,7 +28,7 @@ body: id: image-tag attributes: label: Image or release - description: The pinned tag or digest, for example dsh-v0.2.1-rc.5. + description: The tag or digest you deployed (from the manifest or release metadata). - type: input id: cluster attributes: diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 2979071..2251874 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -19,8 +19,7 @@ ## Checklist - [ ] make check passes (versions, example layout, manifest YAML) -- [ ] make test passes (gateway unit tests) -- [ ] make lint passes (ruff, shellcheck; yamllint/actionlint where installed) +- [ ] make test / make lint pass when the change touches code (gateway unit tests, ruff, shellcheck; yamllint/actionlint where installed) - [ ] Docs updated for the change (root README, docs/, example README) - [ ] No secrets or usable credentials committed - [ ] CHANGELOG.md was not edited by hand (generated by git-cliff) diff --git a/.github/workflows/ci-manifests.yaml b/.github/workflows/ci-manifests.yaml index 9dd68d9..dfb7253 100644 --- a/.github/workflows/ci-manifests.yaml +++ b/.github/workflows/ci-manifests.yaml @@ -1,7 +1,8 @@ # Manifest shape validation and server-side dry-run. Split from ci.yaml so this # job can use native on.pull_request.paths filtering and skip PRs that do not -# touch manifests, scripts or versions.env. The job name is the required -# branch-protection check `manifests`. +# touch manifests, scripts or versions.env. The job name is `manifests`; because +# it is path-filtered it is an on-demand check, not a required branch-protection +# check (see docs/gitflow.md). name: CI (manifests) on: diff --git a/.github/workflows/ci-python.yaml b/.github/workflows/ci-python.yaml index 1d563ad..0a14248 100644 --- a/.github/workflows/ci-python.yaml +++ b/.github/workflows/ci-python.yaml @@ -1,6 +1,7 @@ # Gateway unit tests and lint. Split from ci.yaml so this job can use native # on.pull_request.paths filtering and skip PRs that do not touch the gateway. -# The job name is the required branch-protection check `python`. +# The job name is `python`; because it is path-filtered it is an on-demand check, +# not a required branch-protection check (see docs/gitflow.md). name: CI (python) on: diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index ed30eb5..e3515dc 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -16,11 +16,6 @@ on: description: Existing tag to release (must match versions.env) required: true type: string - publish_legacy_gateway_alias: - description: Also publish the legacy :gateway alias (one release only) - required: false - default: true - type: boolean permissions: contents: write @@ -193,28 +188,6 @@ jobs: ${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }} ${{ needs.meta.outputs.gateway_image }}:latest - legacy-gateway-alias: - name: Publish legacy :gateway alias (temporary) - needs: [meta, build-gateway] - if: github.event_name != 'workflow_dispatch' || inputs.publish_legacy_gateway_alias - runs-on: ubuntu-latest - timeout-minutes: 15 - steps: - - name: Login to GitHub Container Registry - uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - name: Retag gateway as the legacy package tag - env: - SOURCE: ${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }} - LEGACY: ${{ needs.meta.outputs.ghcr_image }}:gateway - run: | - set -euo pipefail - docker buildx imagetools create -t "$LEGACY" "$SOURCE" - echo "Deprecated alias $LEGACY now points at $SOURCE" >> "$GITHUB_STEP_SUMMARY" - publish-release: name: Changelog and GitHub Release needs: [meta, build-main, build-gateway] diff --git a/AGENTS.md b/AGENTS.md index 0605709..26b0c62 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,8 +9,8 @@ before changing anything; it encodes the conventions the CI enforces. Harness Web (dsh web) as a slim, non-root, multi-arch image. Everything else exists to support it or to demonstrate an implementation on top of it. - **Every implementation is a self-contained example** under examples//. - There is exactly one example for the agents-as-a-service platform; do not - split it into parallel directories. + The multi-tenant platform lives in exactly one directory; do not split it into + parallel examples. - **versions.env is the single source of truth** for the DSH ref, pnpm, the control-plane versions and the image names. - **Docs describe the shipped behavior**, not intentions. If you change a @@ -43,7 +43,7 @@ before changing anything; it encodes the conventions the CI enforces. | versions.env | Every pinned version and image name | | Makefile | Canonical local entrypoint | | build.sh | Deprecated shim that forwards to make | -| docs/ | main-image.md, examples.md, gitflow.md, versioning.md | +| docs/ | main-image.md, examples.md, gitflow.md, versioning.md (index: README.md) | | examples/ | One implementation per directory (contract in docs/examples.md) | | examples/dsh-agents-as-a-service/ | The multi-tenant platform example (manifests, gateway, docs, scripts) | | examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart | @@ -54,7 +54,7 @@ before changing anything; it encodes the conventions the CI enforces. ## Commands make check # versions, example layout and manifest YAML (run before every commit) - make test # gateway unit tests (109+ tests) + make test # gateway unit tests (pytest) make lint # ruff + shellcheck (+ yamllint/actionlint when installed) make image # build the main DSH Web image (slow: clones and builds DSH) make gateway # build the example gateway image (fast) @@ -72,21 +72,18 @@ needs kind plus the agent-sandbox CRDs (see the example README). | --- | --- | | Documentation only | make check; verify relative links | | Example manifest | make check; kubectl apply --dry-run=server in CI | -| Optional hardening (gVisor, nodeSelector, NetworkPolicy, provider keys) | make check; server dry-run in CI; manual verification on a capable cluster | +| Optional hardening (gVisor, nodeSelector, NetworkPolicy, provider keys, failed-Pod recovery) | make check; server dry-run in CI; manual verification on a capable cluster | | Gateway code | make test; make lint; add or update tests | | Dockerfile / versions.env | make check; the CI `build-amd64` job (`ci-image.yaml`) builds and smoke-tests the image on the PR; run `make image` locally only when you need the image | | Workflow | actionlint in CI; keep actions SHA-pinned; no hard-coded versions | | New example | make check; all files described in docs/examples.md | -| Project skill (.dsh/skills/) | make check; frontmatter and size check; shellcheck skill scripts; load it with the skill tool | +| Project skill (.dsh/skills/) | make check; validate frontmatter; keep SKILL.md under ~8 KB; shellcheck skill scripts; load it with the skill tool | ## Versioning and releases -- Release identity = DSH_VERSION, optionally with a repository revision suffix - for re-releases at the same DSH ref (current identity: - dsh-v0.2.1-rc.5). -- The current identity is dsh-v0.2.1-rc.5 (stored in versions.env). It is the - first release at this DSH ref, so it carries no revision suffix; a - repository-only change at the same ref publishes -r2. +- Release identity = DSH_VERSION plus an optional -rN repository revision. + The current identity is dsh-v0.2.1-rc.5 (first release at this DSH ref, so no + suffix); a repository-only change at the same ref publishes -r2. - Channels: release tags are immutable; latest follows the newest release; main is the development channel for both image packages. - A broken release is fixed by publishing -r(N+1), never by re-tagging. @@ -99,7 +96,9 @@ needs kind plus the agent-sandbox CRDs (see the example README). - Branch names: feat/, fix/, docs/, chore/, refactor/, test/. - Conventional Commit subjects and PR titles (feat(scope): ..., fix(scope): ...). - PRs are squash-merged; main stays linear and protected. -- Required checks: lint, python, manifests, pr-title, CodeQL, dependency-review. +- Required on every PR: lint, pr-title, CodeQL (both Analyze jobs) and + dependency-review. python, manifests and build-amd64 are path-filtered + on-demand checks; run them when the change touches their scope. - Keep diffs scoped; update docs with the code; never mix a version bump with unrelated changes. @@ -107,9 +106,9 @@ needs kind plus the agent-sandbox CRDs (see the example README). 1. Read docs/examples.md and examples/README.md. 2. Create examples// with README.md, manifests/NN-*.yaml, optional - manifests/exposure// (plus README), optional components with their - own Dockerfile/README/tests, and scripts/ (executable, idempotent, - self-cleaning). + manifests/optional// or manifests/exposure// (each with a + README), optional components with their own Dockerfile/README/tests, and + scripts/ (executable, idempotent, self-cleaning). 3. Publish component images under ////. 4. Add the example to examples/README.md; scripts/check-examples.sh enforces the layout. @@ -124,8 +123,9 @@ needs kind plus the agent-sandbox CRDs (see the example README). - The example shares provider keys (dsh-provider-keys plus the optional dsh-provider-keys-extra) across sandboxes; do not describe it as tenant-isolated. -- gVisor, nodeSelector and the NetworkPolicies are opt-in and are not active in - the base manifests; do not describe the base example as hardened. +- gVisor, nodeSelector, the NetworkPolicies, extra provider keys and failed-Pod + recovery are opt-in and are not active in the base manifests; do not describe + the base example as hardened. - TLS terminates outside the gateway; set COOKIE_SECURE=true for HTTPS and keep the base manifest safe for local HTTP. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cc9404b..d5f39a1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -43,8 +43,10 @@ Install the gateway dev dependencies once: - Use a Conventional Commit title, for example feat(gateway): add wake metrics or docs(examples): clarify SSO trust boundary. PR titles are linted. - Keep the diff scoped. Squash merge into main; history stays linear. -- Required checks: lint, python, manifests, pr-title, CodeQL, and - dependency-review. Branch protection is described in docs/gitflow.md. +- Required on every PR: lint, pr-title, CodeQL (both Analyze jobs) and + dependency-review. python, manifests and build-amd64 are path-filtered + on-demand checks; run them when the change touches their scope. Branch + protection is described in docs/gitflow.md. - Update the docs that the change affects; docs are part of the change, not a follow-up. - Never commit a real secret. Secret-shaped YAML belongs in a *.yaml.example @@ -60,16 +62,12 @@ it by hand; the release workflow regenerates it. ## Adding an example -Read docs/examples.md, then add examples// with: - -- README.md (what it demonstrates, prerequisites, steps, teardown) -- manifests/NN-*.yaml (numbered, plain kubectl) -- optional manifests/exposure// with its own README.md -- optional components (a Dockerfile plus README and tests per component) -- scripts/ with executable, idempotent, self-cleaning scripts -- docs/ for deeper material - -List it in examples/README.md. scripts/check-examples.sh enforces this layout. +Read docs/examples.md, then add examples// with a README.md, +numbered manifests/NN-*.yaml, optional manifests/optional// or +manifests/exposure// (each with a README), optional components (a +Dockerfile plus README and tests), and executable, idempotent, self-cleaning +scripts. Add the literal examples// entry to examples/README.md; +scripts/check-examples.sh enforces the layout. ## Documentation diff --git a/README.md b/README.md index fbda0eb..97e8879 100644 --- a/README.md +++ b/README.md @@ -96,10 +96,7 @@ ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway | --- | --- | --- | | `dsh-v0.2.1-rc.5` | DSH Web, port `3080` | **Current release identity.** Immutable: once published it never moves. | | `latest` | DSH Web | Most recent release; same digest as the current identity. | -| `main` | DSH Web | Development channel rebuilt from every push to `main`. Not a release; not for production. | - -Tags published before the release-identity scheme remain on GHCR but are not -updated; new deployments use the current identity. +| `main` | DSH Web | Development channel rebuilt when image-definition files change on `main`. Not a release; not for production. | The example gateway is published under its nested path with the same release identity, e.g. @@ -134,8 +131,8 @@ Self-contained implementations that consume the image. Index and contract: | Example | What it demonstrates | | --- | --- | -| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Full multi-tenant platform: SandboxTemplate with per-user PVCs, warm pool, one claim per user, suspend-when-idle, wake-on-return, browser gateway with signup + `HttpOnly` sessions + HTTP/WebSocket proxy, Ingress or Gateway API + GitHub SSO exposure, and opt-in gVisor/NetworkPolicy/provider-key hardening. Start with the [walkthrough](examples/dsh-agents-as-a-service/README.md) and the [architecture](examples/dsh-agents-as-a-service/docs/architecture.md). | -| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | The smallest useful setup: one `SandboxTemplate` and one `SandboxClaim`, reached through `kubectl port-forward` + the launch-token URL. Manifest: [`manifests/sandbox-template.yaml`](examples/minimal-single-tenant/manifests/sandbox-template.yaml). | +| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Full multi-tenant platform: SandboxTemplate with per-user PVCs, warm pool, one claim per user, suspend-when-idle, wake-on-return, browser gateway with signup + `HttpOnly` sessions + HTTP/WebSocket proxy, Ingress or Gateway API + GitHub SSO exposure, and opt-in gVisor/NetworkPolicy/provider-key hardening and failed-Pod recovery. Start with the [walkthrough](examples/dsh-agents-as-a-service/README.md) and the [architecture](examples/dsh-agents-as-a-service/docs/architecture.md). | +| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | The smallest useful setup: one `SandboxTemplate` and one `SandboxClaim`, reached through `kubectl port-forward` + the launch-token URL. Manifest: [`manifests/10-sandboxtemplate.yaml`](examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml). | The gateway is a component of the multi-tenant example and has its own [README](examples/dsh-agents-as-a-service/gateway/README.md) (API, configuration, @@ -166,7 +163,7 @@ guide and troubleshooting notes live beside `SKILL.md`. | [`scripts/`](scripts/) | Repository checks: versions, example layout, manifest YAML, version bump. | | [`.dsh/skills/`](.dsh/skills/) | Project-local DSH skills, auto-discovered from the git root. | | [`.github/`](.github/) | CI, release automation, dependency updates, PR/issue templates. | -| [`.env.sample`](.env.sample) | Documented local configuration (copy to `.env`; never commit it). | +| [`.env.sample`](.env.sample) | Reference environment names for local runs; nothing reads it automatically. Never commit a real key. | ## Development @@ -200,24 +197,6 @@ Conventional Commits and is never edited by hand. See [`CHANGELOG.md`](CHANGELOG.md), [docs/versioning.md](docs/versioning.md) and [cliff.toml](cliff.toml). -## Migrating from the previous layout - -If you are coming from an older checkout, these paths moved. Nothing else changed -semantically; the content was split into the example it belongs to. - -| Old path | New path | -| --- | --- | -| `gateway-api/dsh-demo/` (Gateway API + oauth2-proxy / GitHub SSO) | `examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/` (now `70-gateway.yaml` ... `74-tls-secret.yaml.example`) | -| `examples/dsh-agents-as-a-service/60-ingress.yaml` | `examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml` | -| `examples/dsh-agents-as-a-service/NN-*.yaml` (`00-` ... `50-`) | `examples/dsh-agents-as-a-service/manifests/NN-*.yaml` | -| `examples/dsh-agents-as-a-service/run-test-kind.sh` | `examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` | -| `deploy/k8s/sandbox-template.yaml` | `examples/minimal-single-tenant/manifests/sandbox-template.yaml` | -| `docs/architecture.md` | `examples/dsh-agents-as-a-service/docs/architecture.md` | - -The SSO walkthrough that lived in `gateway-api/dsh-demo/README.md` is merged -into the [multi-tenant example README](examples/dsh-agents-as-a-service/README.md) -(step 9) and the [exposure README](examples/dsh-agents-as-a-service/manifests/exposure/README.md). - ## Contributing - [CONTRIBUTING.md](CONTRIBUTING.md) — local setup, PR workflow, what a change needs. diff --git a/SECURITY.md b/SECURITY.md index 7942ffa..8ed1e37 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -21,7 +21,7 @@ disclosure once a fix is available. from pod logs (pods/log). Anyone with access to those logs can impersonate a tenant. Keep the gateway's RBAC namespace-scoped and treat pod logs as sensitive. Replacing log scraping with an upstream token API is tracked in - the example architecture document. + the [example architecture document](examples/dsh-agents-as-a-service/docs/architecture.md#upstream-feature-request-remove-log-scraping). - **SSO header trust boundary.** AUTH_PROXY_HEADER makes the gateway trust an identity header (for example X-Forwarded-User). Enable it only when the gateway Service is reachable exclusively through the authenticating proxy; diff --git a/cliff.toml b/cliff.toml index bb55a2c..a9e14fa 100644 --- a/cliff.toml +++ b/cliff.toml @@ -44,6 +44,6 @@ commit_parsers = [ ] protect_breaking_commits = false filter_commits = false -# Release tags are dsh-vX.Y.Z[-rN]; the single legacy pre-scheme tag is v0.0.1. +# Release tags are dsh-vX.Y.Z[-rN]; the single pre-scheme tag is v0.0.1. tag_pattern = "^(v|dsh-v).*$" sort_commits = "oldest" diff --git a/docs/README.md b/docs/README.md index b90b7ac..d9a618f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,8 +2,7 @@ Repository-wide documentation lives here. Example-specific material lives with its example under [`examples//docs/`](../examples/) — see the -[agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md) -for the deep-dive that used to live in this directory. +[agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md). If you are new, read the [root README](../README.md) first: it is image-first and gets a DSH Web container running before anything else. @@ -34,6 +33,8 @@ DSH discovers project skills from `/.dsh/skills//SKILL.md` | [dsh-agents-as-a-service README](../examples/dsh-agents-as-a-service/README.md) | The multi-tenant platform walkthrough: warm pool, claims, suspend/resume, gateway, exposure and troubleshooting. | | [dsh-agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md) | Container/sequence diagrams, ADRs, security and RBAC, failure modes and the evolution path. | | [gateway README](../examples/dsh-agents-as-a-service/gateway/README.md) | The gateway's HTTP/WebSocket API, configuration, auth (including SSO), and limitations. | +| [optional hardening](../examples/dsh-agents-as-a-service/docs/optional-hardening.md) | Opt-in gVisor, nodeSelector, NetworkPolicy, provider keys and failed-Pod recovery. | +| [exposure variants](../examples/dsh-agents-as-a-service/manifests/exposure/README.md) | The mutually exclusive Ingress and Gateway API + SSO exposure paths. | ## Project policies diff --git a/docs/examples.md b/docs/examples.md index 9f81eea..a34fc56 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -41,6 +41,7 @@ examples// example directory; - an example has no `README.md` or no `manifests/` directory; - `manifests/` contains no top-level `*.yaml` files; +- a top-level `manifests/*.yaml` is not named `NN-*.yaml`; - a `*.yaml` file sits at the example root instead of under `manifests/`; - `manifests/exposure/` exists without a `manifests/exposure/README.md` describing the variants; diff --git a/docs/gitflow.md b/docs/gitflow.md index 8f0654d..5620988 100644 --- a/docs/gitflow.md +++ b/docs/gitflow.md @@ -30,7 +30,7 @@ branch; the PR is squash-merged, so individual branch commits are never part of `docs(scope): ...`, `build: ...`, `ci: ...`, `refactor: ...`, `test: ...`, `chore: ...`. Breaking changes use `feat(scope)!: ...` or a `BREAKING CHANGE:` footer. -3. **CI must be green.** All required checks below must pass before merge. +3. **CI must be green.** All applicable checks below must pass before merge. 4. **Squash merge only.** `main` stays linear and protected; merged PRs are one commit each. 5. **No direct pushes to `main`** or to a release branch. Both are protected. @@ -40,21 +40,20 @@ branch; the PR is squash-merged, so individual branch commits are never part of ## Required checks -| Check | Workflow / job | What it verifies | -| --- | --- | --- | -| `lint` | `ci.yaml` | `scripts/check-versions.sh` (version pins and markdown tokens), `scripts/check-examples.sh` (example layout), shellcheck, actionlint, yamllint, hadolint and documentation links. | -| `python` | `ci-python.yaml` | Gateway ruff check/format and unit tests (`pytest` with coverage). Runs only when the gateway or this workflow changes. | -| `manifests` | `ci-manifests.yaml` | `scripts/check-manifests.py` (manifest shape) and a server-side dry-run of every example manifest against the agent-sandbox CRDs. Runs only when manifests, check scripts or `versions.env` change. | -| `pr-title` | `pr-title.yaml` | Conventional Commit PR title. | -| `CodeQL` | `codeql.yml` | Static analysis for Python and GitHub Actions. | -| `dependency-review` | `ci.yaml` | Dependency changes introduced by the PR (job `dependency-review`; runs on pull requests only). | +| Check | Workflow / job | Required on every PR? | What it verifies | +| --- | --- | --- | --- | +| `lint` | `ci.yaml` | yes | `scripts/check-versions.sh` (version pins and markdown tokens), `scripts/check-examples.sh` (example layout and numbered manifests), shellcheck, actionlint, yamllint, hadolint and documentation links. | +| `python` | `ci-python.yaml` | no (path-filtered) | Gateway ruff check/format and unit tests (`pytest` with coverage). Runs only when the gateway or this workflow changes. | +| `manifests` | `ci-manifests.yaml` | no (path-filtered) | `scripts/check-manifests.py` (manifest shape) and a server-side dry-run of every example manifest against the agent-sandbox CRDs. Runs when manifest files, `scripts/check-manifests.py` or `versions.env` change. | +| `pr-title` | `pr-title.yaml` | yes | Conventional Commit PR title. | +| `Analyze (python)`, `Analyze (actions)` | `codeql.yml` | yes (both jobs) | CodeQL static analysis for Python and GitHub Actions. | +| `dependency-review` | `ci.yaml` | yes | Dependency changes introduced by the PR (job `dependency-review`; runs on pull requests only). | `python`, `manifests` and `build-amd64` use native `on.pull_request.paths` filtering, so PRs that do not touch the gateway, the manifests or the image -definition skip them. The gateway image build and `/healthz` are no longer -smoke-tested in a dedicated job: `/healthz` is covered by the gateway unit tests, -the Dockerfile is linted by `lint` (hadolint), and the multi-arch image build is -published by `build-gateway.yaml` on merge. +definition skip them. The gateway image is built and published by +`build-gateway.yaml` on merge; `/healthz` is covered by the gateway unit tests and +the Dockerfile is linted by `lint` (hadolint). `ci-image.yaml` (`build-amd64`) is the PR-side check for the main image: when `Dockerfile`, `.dockerignore` or `versions.env` change it builds `linux/amd64` @@ -64,9 +63,9 @@ the result (`--version`, non-root UID, `/workspace` + `DSH_HOME`, the container). The published multi-arch build still happens only in `build-main.yaml` on merge. -Because a skipped path-filtered check is not a passing check, only the always-on -checks (`lint`, `pr-title`, `CodeQL`, `dependency-review`) should be enforced as -required status checks; `python`, `manifests` and `build-amd64` are on-demand +Because a skipped path-filtered check is not a passing check, require only the +always-on set: `lint`, `pr-title`, both CodeQL `Analyze` jobs and +`dependency-review`. Keep `python`, `manifests` and `build-amd64` as on-demand checks that skip when unrelated to the change. The image build workflows (`_build-image.yaml`, `build-main.yaml`, @@ -109,7 +108,8 @@ Rollback is also a re-release, never a retag: see ## Tag protection -All release tags match `dsh-*` and are protected: +Release tags follow `dsh-*` and are protected (the pre-scheme `v0.0.1` tag is +historical and unmaintained): - only maintainers (the release workflow) may create them; - they cannot be deleted or force-updated; @@ -123,11 +123,12 @@ respectively. Configure these on the repository before relying on the workflow: -- [ ] **Branches → main**: require a pull request; require the checks `lint`, - `python`, `manifests`, `pr-title`, `CodeQL`, `dependency-review`; require - branches to be up to date; require conversation resolution; restrict direct - pushes; allow only squash merges and enable "Automatically delete head - branches". +- [ ] **Branches → main**: require a pull request; require the always-on checks + `lint`, `pr-title`, both CodeQL `Analyze` jobs and `dependency-review` + (`python`, `manifests` and `build-amd64` are path-filtered and must not be + required); require branches to be up to date; require conversation + resolution; restrict direct pushes; allow only squash merges and enable + "Automatically delete head branches". - [ ] **Branches → release/***: same protections, plus allow the release workflow to push tags. - [ ] **Tags → dsh-***: restrict creation/update/deletion to maintainers; a tag diff --git a/docs/main-image.md b/docs/main-image.md index 4573ece..58d3e52 100644 --- a/docs/main-image.md +++ b/docs/main-image.md @@ -47,8 +47,8 @@ docker build \ | Arg | Default / source | Purpose | | --- | --- | --- | | `DSH_REPO` | `https://github.com/deepseek-ai/deepseek-harness.git` | Upstream DSH repository to clone. | -| `DSH_VERSION` | `DSH_VERSION` in [versions.env](../versions.env) | Upstream DSH git tag/ref to package; recorded as `org.opencontainers.image.version`. | -| `RELEASE_REVISION` | `RELEASE_REVISION` in [versions.env](../versions.env) | Repository revision suffix (`rN`); the release identity is `DSH_VERSION-RELEASE_REVISION`, recorded in the `io.sredevopsorg.release.revision` label. See [versioning](versioning.md). | +| `DSH_VERSION` | `DSH_VERSION` in [versions.env](../versions.env) | Upstream DSH git tag/ref to package. The Dockerfile records it as `io.sredevopsorg.dsh.version` and (for local builds) `org.opencontainers.image.version`; published builds set the version label to the full release identity. | +| `RELEASE_REVISION` | `RELEASE_REVISION` in [versions.env](../versions.env) | Repository revision suffix (`rN`), recorded in `io.sredevopsorg.release.revision`. The release identity is `DSH_VERSION` plus `-RELEASE_REVISION` when set. See [versioning](versioning.md). | | `PNPM_VERSION` | `PNPM_VERSION` in [versions.env](../versions.env) | pnpm version activated with corepack in both stages. | | `VCS_REF` | `git rev-parse --short HEAD` (via `make image`) | Source revision recorded for traceability. | | `NODE_BUILD_IMAGE` | `docker.io/node:24-trixie` | Build-stage base image. | @@ -90,9 +90,10 @@ includes `/app/node_modules/.bin`. the picker's browse backend lists `$HOME` when the dialog opens and labels that listing *Home*. With the image default (`/home/node`) that listing is outside a mounted volume; the example sandbox templates -([minimal](../examples/minimal-single-tenant/manifests/sandbox-template.yaml), +([minimal](../examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml), [agents as a service](../examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml)) -set `HOME=/workspace` so the picker opens on the persisted agent workspace. `~` +mount the same PVC's `workspace` subPath at `/workspace` and set `HOME=/workspace` +so the picker opens on the persisted agent workspace. `~` and every tool that keeps caches or config under `$HOME` (`npm`/`pnpm`, `uv`, `git`, `gh`, DSH's global `~/.agents` instructions and skills) follow, so those dotfiles then live on the workspace volume too. `DSH_HOME` is separate and stays @@ -141,7 +142,7 @@ procedure. | --- | --- | --- | | Exposed port | `3080/tcp` | `EXPOSE 3080`; one port serves the UI, assets, API, SSE and WebSocket traffic. | | State volume | `/home/node/.dsh` | Declared with `VOLUME`; use a named volume (Docker) or PVC (Kubernetes). | -| Working directory | `/workspace` | The agent workspace. Mount a volume here if agents should keep files; the agents-as-a-service example mounts the same PVC's `workspace` subPath here and points `HOME` at it, so DSH's folder picker opens on it. | +| Working directory | `/workspace` | The agent workspace. Mount a volume here if agents should keep files; the shipped example templates mount the same PVC's `workspace` subPath here and point `HOME` at it, so DSH's folder picker opens on it. | | Stop signal | `SIGTERM` | `tini` forwards it to `dsh web`; allow a graceful shutdown period. | | User | `node`, UID/GID `1000` | The process is non-root. A PVC mounted at `DSH_HOME` must be writable by UID 1000. | @@ -164,8 +165,8 @@ livenessProbe: periodSeconds: 15 ``` -The shipped templates use exactly this shape; see -[`examples/minimal-single-tenant/manifests/sandbox-template.yaml`](../examples/minimal-single-tenant/manifests/sandbox-template.yaml). +The shipped templates use the same path and startup budget; see +[`examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml`](../examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml). A `startupProbe` is important because plugin activation (or the first start of a large sandbox) can exceed a default liveness budget. @@ -199,10 +200,7 @@ GitHub Container Registry: | --- | --- | | `dsh-v0.2.1-rc.5` | Current release identity. **Immutable**: once published it never moves. | | `latest` | Most recent release; points at the same digest as the current identity. | -| `main` | Development channel rebuilt from every push to `main`; not a release, not for production. | - -Tags published before the release-identity scheme remain published but are not -updated; new deployments use the current identity. +| `main` | Development channel rebuilt when image-definition files change on `main`; not a release, not for production. | Release identities, channels and rollback rules are defined in [versioning](versioning.md). Tags are immutable, so production deployments should @@ -220,6 +218,7 @@ docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbo docker inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.5 \ --format '{{json .Config.Labels}}' # org.opencontainers.image.title / .version / .source +# io.sredevopsorg.dsh.version / io.sredevopsorg.release.revision ``` Release builds also publish SBOM and provenance attestations from the release diff --git a/docs/versioning.md b/docs/versioning.md index 66dfce2..a316884 100644 --- a/docs/versioning.md +++ b/docs/versioning.md @@ -15,7 +15,7 @@ release identity = DSH_VERSION + optional "-" + RELEASE_REVISION | Piece | Meaning | | --- | --- | -| `DSH_VERSION` | The upstream DeepSeek Harness git ref being packaged, e.g. `dsh-v0.2.1-rc.5`. It mirrors upstream's own tag, which is what "mirrored-DSH scheme" means. | +| `DSH_VERSION` | The upstream DeepSeek Harness git ref being packaged, e.g. `dsh-v0.2.1-rc.5`. It mirrors upstream's own tag. | | `RELEASE_REVISION` | An optional repository revision at that DSH ref: empty for the first release, `rN` (`N >= 2`) for a repository-only re-release. | Examples: @@ -33,23 +33,14 @@ with an empty revision); a repository-only change at the same DSH ref increments ## Why release identities are immutable **Container tags are immutable** in this project: re-pushing a tag would silently -change what an existing deployment runs. A DSH upgrade therefore produces a new -base identity with an empty revision; a repository-only change at an -already-released ref publishes `-rN`, where `r2` is the first re-release (the -plain tag is conceptually revision 1 and is taken). +change what an existing deployment runs. A fix therefore publishes a new identity; +see the rules above. The current identity is `dsh-v0.2.1-rc.5`, the first release at this upstream ref, so it carries no revision suffix. The identity is stored in -[versions.env](../versions.env) and consumed everywhere: the Dockerfile label, -the example manifests, the Makefile and the release workflow. Tags published -before the release-identity scheme remain published but are not updated; new -deployments use the current identity. - -The gateway image moved from the repository-level `:gateway` tag to a nested -package (`/dsh-agents-as-a-service/gateway`). The release workflow -also republishes the legacy `:gateway` tag as a deprecated -compatibility alias; it may be removed in a later release, so deployments -should pin the nested package. +[versions.env](../versions.env) and consumed everywhere: the Dockerfile labels +(version and revision), the example manifests, the Makefile and the release +workflow. ## Channels @@ -57,7 +48,7 @@ should pin the nested package. | --- | --- | --- | | Release | `dsh-v0.2.1-rc.5` | The current immutable release identity. What production should pin (preferably by digest). | | `latest` | `latest` | Alias for the most recent release; moves only when a new release is published. | -| `main` | `main` | Development channel, rebuilt from every push to `main`. Never treat it as a release; it exists so `main` can be tested. | +| `main` | `main` | Development channel, rebuilt when image or gateway sources change on `main`. Never treat it as a release; it exists so `main` can be tested. | The same channels exist for every published package, including the nested example component image. @@ -110,11 +101,12 @@ workflows with a literal version. matches [versions.env](../versions.env). 6. The workflow builds and pushes multi-arch images for the main image and the example components (the identity tag plus `latest`), creates the GitHub - Release from the git-cliff notes, records the published digests, and opens a - `chore(changelog)` PR that refreshes `CHANGELOG.md` on `main`. + Release from the git-cliff notes, and opens a `chore(changelog)` PR that + refreshes `CHANGELOG.md` on `main`. The publish log shows each image manifest + and digest. 7. Merge the changelog PR. Verify: - `docker buildx imagetools inspect :` shows both platforms, - and the GitHub Release references the tag and the digest. + `docker buildx imagetools inspect :` shows both platforms; + the GitHub Release references the tag, and the workflow log shows the digest. 8. Announce the identity and the digest; deployments pin the digest. ## Rollback diff --git a/examples/README.md b/examples/README.md index 066daca..66bb764 100644 --- a/examples/README.md +++ b/examples/README.md @@ -11,7 +11,7 @@ reference material, not libraries. | [`examples/minimal-single-tenant/`](minimal-single-tenant/) | The smallest useful setup: a single `SandboxTemplate` and one `SandboxClaim`, no gateway — reach DSH Web with a launch-token URL through `kubectl port-forward`. | agent-sandbox with extensions. | The multi-tenant example also documents opt-in gVisor, `nodeSelector`, -NetworkPolicy and extra provider-key hardening in +NetworkPolicy, extra provider-key hardening and failed-Pod recovery in [optional hardening](dsh-agents-as-a-service/docs/optional-hardening.md). ## The example contract (short version) @@ -19,37 +19,26 @@ NetworkPolicy and extra provider-key hardening in Full rules: [docs/examples.md](../docs/examples.md). `scripts/check-examples.sh` enforces the layout, and `make check` runs it. -``` -examples// -+-- README.md # walkthrough: prerequisites, steps, troubleshooting, cleanup -+-- manifests/ # NN-*.yaml, applied with plain kubectl -| +-- optional// # optional opt-in manifests... -| +-- optional/README.md # ...required whenever optional/ exists -| +-- exposure// # optional per-variant manifests... -| +-- exposure/README.md # ...required whenever exposure/ exists -+-- scripts/ # optional executable, idempotent, self-cleaning *.sh -+-- docs/ # optional architecture/ADR material -+-- / # optional component with Dockerfile + README + tests -``` - -- Secrets never ship populated: use `*.yaml.example` files with placeholders. -- Manifests are numbered (`00-`, `10-`, ...) and read as an ordered walkthrough. -- Component images are published nested below the main image: - `////`. -- Every example must be listed above with its literal `examples//` path. -- Scripts under `scripts/` must be executable, idempotent and clean up after - themselves. +- `README.md` is the walkthrough; `manifests/NN-*.yaml` is applied with plain + kubectl and read in order. +- Optional `manifests/optional//` and `manifests/exposure//` + each ship a README; exposure variants are mutually exclusive. +- Optional `scripts/` must be executable, idempotent and self-cleaning; `docs/` + holds deeper material. +- Components with a Dockerfile need a README and tests; images are published + nested below the main image (`////`). +- Secrets are `*.yaml.example` with placeholders; every example is listed above + with its literal `examples//` path. ## Running an example -```bash -# e.g. the multi-tenant platform example -kubectl apply -f examples/dsh-agents-as-a-service/manifests/ +Each example README has the exact prerequisites, commands, optional hardening, +exposure options and teardown. The multi-tenant example also has a full kind +walkthrough that cleans up after itself: -# or run the whole walkthrough (kind + agent-sandbox required) and let it clean up +```bash examples/dsh-agents-as-a-service/scripts/e2e-kind.sh ``` `kubectl apply -f manifests/` is non-recursive: `optional/` and `exposure/` -subdirectories are applied explicitly. Each example README has the exact -prerequisites, optional hardening, exposure options and teardown. +subdirectories are applied explicitly. diff --git a/examples/dsh-agents-as-a-service/README.md b/examples/dsh-agents-as-a-service/README.md index 92bc1f0..2f3b29d 100644 --- a/examples/dsh-agents-as-a-service/README.md +++ b/examples/dsh-agents-as-a-service/README.md @@ -8,6 +8,8 @@ personal, long-lived DSH Web instance that is **provisioned in ~2 seconds from a warm pool, suspended when idle to save cost, and woken transparently when the user returns** — with all state (conversations, sessions, config, and the agent workspace) surviving on the per-user PVC at `/home/node/.dsh` + `/workspace`. +Suspend-when-idle plus the warm pool keep cost proportional to use: a return +after absence pays a short cold-start pause, not an always-on instance. It ships in three layers: @@ -79,12 +81,14 @@ What it demonstrates, feature by feature: ## Prerequisites +Run the walkthrough from this example's directory (`examples/dsh-agents-as-a-service/`). + - A Kubernetes cluster with a default StorageClass (kind and GKE both work as-is). - agent-sandbox with extensions installed: ```bash - VERSION=v1.0.8 # AGENT_SANDBOX_VERSION in versions.env - kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$VERSION/sandbox-with-extensions.yaml + . ../../versions.env + kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${AGENT_SANDBOX_VERSION}/sandbox-with-extensions.yaml" kubectl -n agent-sandbox-system rollout status deployment --timeout=180s ``` @@ -243,12 +247,9 @@ kubectl -n dsh-demo exec "$SB" -- cat /home/node/.dsh/marker.txt # "remember m kubectl -n dsh-demo exec "$SB" -- cat /workspace/marker.txt # "remember me too" ``` -> **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`. -> DSH requires its `.credentials.yaml` to be `0600`, while the kubelet's default -> fsGroup policy widens every file to `0660` on each pod start — a resumed sandbox -> would crash-loop. See [Troubleshooting](#troubleshooting) for the one-time fix -> on sandboxes created before this setting existed. The `/workspace` mount does not -> change that: `subPath` is a separate directory, and only the volume root is walked. +> **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`; +> without it the kubelet widens `.credentials.yaml` to `0660` and DSH crash-loops. +> See [Troubleshooting](#troubleshooting) for the one-time fix on older sandboxes. ## 7. Injection policy rejects env-carrying claims @@ -273,7 +274,7 @@ kind load docker-image "$GW" --name # ... or build it locally and tag it exactly as manifests/50-gateway.yaml # expects (make gateway builds dsh-aaas-gateway:dev): -make gateway +make -C ../.. gateway docker tag dsh-aaas-gateway:dev "$GW" kind load docker-image "$GW" --name @@ -351,44 +352,21 @@ API clients that prefer bearer tokens can use `POST /users` with a JSON body (`{"user":"bob"}`) and the returned one-time token, `POST /login` to exchange a token for a cookie, and `Authorization: Bearer ` on subsequent calls. -### Gateway API +### Gateway routes -| Route | Purpose | -| --- | --- | -| `GET /healthz` | Liveness/readiness; no cluster access | -| `GET /` | Landing page: signup, token login, logout. With SSO it auto-provisions and connects instead. | -| `POST /users` | Signup. JSON or form; `201` (JSON), `202` (waking), `303` (browser redirect), `409` if user exists; sets the session cookie | -| `POST /login` | Exchange a bearer token for the session cookie | -| `POST /logout` | Clear the session cookie | -| `POST /recreate` | SSO only: delete the authenticated user's claim (wiping workspace data) and provision a fresh one; `303` to `/u/{user}/`. Cookie-authenticated calls must be same-origin. | -| `GET /users/{user}` | Derived state (`Ready`/`Waking`/`Suspended`) | -| `DELETE /users/{user}` | Delete claim (cascades sandbox + PVC) | -| `GET /u/{user}` | `308` redirect to `/u/{user}/` | -| `GET /u/{user}/**` | HTTP/SSE proxy to the user's DSH Web | -| `WS /u/{user}/**` | WebSocket proxy to the user's DSH Web | - -The full API reference, configuration table, auth semantics and failure matrix -live in the [gateway README](gateway/README.md). +The browser flow above uses the gateway's main routes. The full route table, +status codes, configuration and failure matrix live in the +[gateway README](gateway/README.md). ### How the DSH handshake works -DSH does not expose an API for its launch token; it prints a URL containing -`?token=…` in the pod logs. For each `(user, pod UID, Host)` the gateway: - -1. resolves the claim → sandbox → pod (annotation `agents.x-k8s.io/pod-name`, or - the sandbox name as pod name), -2. reads the pod's logs (last `DSH_LOG_TAIL_LINES` lines) and parses the most - recent `dsh web: …?token=…` line, -3. requests `http://:3080/?token=…` with the browser's `Host`, -4. captures DSH's `dsh-auth-*` `Set-Cookie` response and caches it, -5. injects that cookie (and strips the browser's `Cookie`/`Authorization` and any - `?token=`) on every proxied HTTP and WebSocket request, -6. refreshes the handshake once if DSH answers `401`, and drops the cached cookie - when the sandbox suspends or is deleted. - -This is why the gateway's RBAC includes `pods` `get`/`list` and `pods/log` -`get`. It is also the most fragile part of the design; see -[Limitations](#limitations). +DSH prints its launch token only in pod logs. The gateway resolves +claim → sandbox → pod, reads the pod-log tail, performs the `?token=` handshake +under the browser's `Host`, caches `dsh-auth-*` per `(user, pod UID, Host)`, and +refreshes once on a `401`. That is why its RBAC includes `pods` `get`/`list` and +`pods/log` `get`; it is also the most fragile part of the design (see +[Limitations](#limitations) and the +[architecture ADR](docs/architecture.md#adr-3-server-side-launch-token-handshake-instead-of-exposing-the-token)). ### DSH_TRUSTED_HOSTS @@ -407,27 +385,18 @@ The same applies to a local `docker run`: the printed token URL uses ### WebSocket -The gateway is a Starlette ASGI app and proxies WebSocket upgrades with a -bidirectional pump, so the DSH web GUI is fully interactive. Requirements: +The gateway proxies WebSocket upgrades with a bidirectional pump, so the DSH web +GUI is fully interactive. The entry point must pass `Upgrade`/`Connection` +headers; [`manifests/exposure/ingress/60-ingress.yaml`](manifests/exposure/ingress/60-ingress.yaml) +is a starting point (Traefik handles upgrades by default). Idle suspension is +disabled while a proxied request or socket is in flight, so a long-running turn is +not cut off. -- the ingress/load balancer in front of the gateway must pass `Upgrade`/`Connection` - headers; [`manifests/exposure/ingress/60-ingress.yaml`](manifests/exposure/ingress/60-ingress.yaml) - is a starting point (Traefik handles upgrades by default), -- idle suspension is disabled for users with in-flight proxied requests or open - WebSockets, so a long-running chat stream is not cut off mid-turn, -- the upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`, - and performs the launch-token handshake under that same `pod-IP:3080` authority. - DSH fences on `Host` (only its bind address, loopback, or an allow-listed host - passes) and binds its auth cookie to the request authority - (`payload.authority === authority`), while `websockets` derives `Host` from the - URI and cannot be overridden. A browser-host cookie or forwarded origin makes - the upgrade fail (`401`/`403`, surfaced as close `1011`), which the client - retries forever as `[connection] connection lost, retry #N`, -- missing/invalid gateway auth is rejected right after the upgrade with close code - `4401` (WebSocket auth has no HTTP `401` path); cross-origin or Origin-less - upgrades with the cookie are rejected the same way (CSWSH protection), so - clients must send a matching `Origin`; upstream bridging failures close with - `1011`. +The upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`, +and performs the launch-token handshake under the same `pod-IP:3080` authority. +Missing or cross-origin auth closes with `4401`; upstream bridging failures close +with `1011`. See the [gateway README](gateway/README.md#proxy-failure-semantics) +for the full matrix. ### Path routing @@ -457,27 +426,18 @@ and the gateway. They are part of the variant; see the ### Option A — plain Ingress (gateway's own session auth) -```bash -# Replace the dsh.example.com host and TLS Secret (or drop the tls: block for -# plain HTTP), then: -kubectl apply -f manifests/exposure/ingress/ -``` - -1. Set `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml` to the same - public host (the gateway forwards the browser's `Host`) and re-apply the - template. -2. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS: +1. Set the Ingress host/TLS (or drop the `tls:` block), and set + `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml` to the same public + host; re-apply the template. +2. Set `COOKIE_SECURE=true` once the Ingress serves HTTPS: `kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`. - (The base manifest keeps `false` so the plain-HTTP port-forward demo works.) -3. Verify: `curl -sI https:///healthz` returns `200`; sign up in a browser - at `https:///`. -4. `manifests/exposure/ingress/61-networkpolicy.yaml` is part of this variant: - it allows only Traefik pods to reach the gateway on `8080`. Adjust its - selector if your ingress controller uses a different pod label or - `hostNetwork`; if kubelet probes fail, see the hardening doc. +3. `kubectl apply -f manifests/exposure/ingress/`; verify + `curl -sI https:///healthz` returns `200`. Keep the router free of buffering/compression middlewares: they break WebSocket -upgrades and streaming. +upgrades and streaming. The variant's `61-networkpolicy.yaml` allows only Traefik +pods to reach the gateway; see the [exposure README](manifests/exposure/README.md) +for selector and probe caveats. ### Option B — Gateway API + GitHub SSO (oauth2-proxy) @@ -487,84 +447,41 @@ front of **oauth2-proxy**, which authenticates the browser against GitHub `X-Forwarded-User` set. The gateway trusts that header only when `AUTH_PROXY_HEADER` is set (SSO is **opt-in**). -Prerequisites: - -- Gateway API CRDs and a Gateway implementation; the manifests assume - `gatewayClassName: traefik` (the CRD version is pinned as - `GATEWAY_API_VERSION` in `versions.env`). -- A GitHub OAuth App whose homepage/callback match the public FQDN - (`https:///` and `https:///oauth2/callback`). -- The oauth2-proxy credentials Secret (template: - `73-oauth2-proxy-secret.yaml.example`) and a TLS Secret for the Gateway - listener (template: `74-tls-secret.yaml.example`). - -Steps: - -```bash -# 1. Real Secrets; the committed *.yaml.example files are templates only. -kubectl -n dsh-demo create secret generic oauth2-proxy \ - --from-literal=client-id='' \ - --from-literal=client-secret='' \ - --from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')" -kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key - -# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72). - -# 3. Enable SSO on the gateway and trust the public host in the template. -kubectl -n dsh-demo set env deploy/dsh-gateway \ - AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true -# ... and set DSH_TRUSTED_HOSTS=dsh-demo.example.com in -# manifests/10-sandboxtemplate.yaml, then re-apply the template. - -# 4. Apply the variant. -kubectl apply -f manifests/exposure/gateway-api-sso/ -``` - -Verification: - -```bash -kubectl get gateway dsh-gateway -n dsh-demo -o yaml # Programmed=True -kubectl get httproute -n dsh-demo -o wide # backend oauth2-proxy:4180, Accepted -kubectl get deploy oauth2-proxy -n dsh-demo # 1/1 Ready -kubectl get endpoints oauth2-proxy -n dsh-demo # backend present -kubectl -n dsh-demo logs deploy/oauth2-proxy --tail=50 # no config errors -``` - -Browser check: open `https:///` — you are redirected to GitHub, and after -signing in as a member of the allowed organization the gateway auto-provisions -your sandbox (the first page can show `Provisioning`) and connects you. A -non-member is denied before reaching the gateway. GitHub logins are lowercase -DNS-1123 labels, so they map 1:1 onto the gateway's `user` validation. The SSO -landing page also offers **Recreate sandbox** (`POST /recreate`), which deletes -the claim — wiping the workspace PVC — and provisions a fresh one. - -Sign-out: the gateway `/logout` clears `dsh_gw_session`; oauth2-proxy's session -can be cleared at `/oauth2/sign_out`. +1. Install Gateway API CRDs and a Gateway implementation; the manifests assume + `gatewayClassName: traefik` (the CRD version is `GATEWAY_API_VERSION` in + `versions.env`). +2. Create a GitHub OAuth App whose homepage/callback match the public FQDN, and + create the oauth2-proxy and TLS Secrets from + `73-oauth2-proxy-secret.yaml.example` and `74-tls-secret.yaml.example`. +3. Replace `dsh-demo.example.com` and `your-github-org`, set + `DSH_TRUSTED_HOSTS` in the template, and enable SSO: + `kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true`. +4. `kubectl apply -f manifests/exposure/gateway-api-sso/`; verify the Gateway, + HTTPRoute, oauth2-proxy Deployment and endpoints as shown in the + [exposure README](manifests/exposure/README.md). > **SSO trust boundary — read this before enabling it.** After SSO is on, the > `dsh-gateway` Service must be reachable **only** through oauth2-proxy (it is > ClusterIP). The variant's `75-networkpolicies.yaml` restricts pod traffic > accordingly: the gateway accepts traffic only from oauth2-proxy pods, -> oauth2-proxy only from the Gateway implementation, and oauth2-proxy egress is -> limited to DNS, public HTTPS and the gateway. Anyone who can reach -> `dsh-gateway:8080` directly can still send `X-Forwarded-User: ` and be -> treated as that user, so never add a second path (the Ingress variant, a -> shared-host port-forward, a permissive policy). To turn SSO off, remove the -> variable (`kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER-`) -> and delete the SSO variant; the gateway falls back to bearer-token/session -> auth. +> oauth2-proxy only from the Gateway implementation, and its egress only to DNS, +> public HTTPS and the gateway. Anyone who can reach `dsh-gateway:8080` directly +> can send `X-Forwarded-User: `, so never add a second path. To turn SSO +> off, remove the variable +> (`kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER-`) and +> delete the SSO variant. ### Switching variants -Delete the variant you had before applying the other one -(`kubectl delete -f manifests/exposure/ingress/`), revert +Delete the variant you had before applying the other one, revert `AUTH_PROXY_HEADER`/`COOKIE_SECURE`, and verify with `kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to the -gateway remains. +gateway remains. Full commands and verification are in the +[exposure README](manifests/exposure/README.md). ## Optional hardening -The base walkthrough is deliberately portable. Four opt-in isolation layers are +The base walkthrough is deliberately portable. Five opt-in settings are documented in [optional hardening](docs/optional-hardening.md): - **gVisor** — uncomment `runtimeClassName: gvisor` before applying, or patch an @@ -582,7 +499,7 @@ documented in [optional hardening](docs/optional-hardening.md): the sandbox at `Finished=True`. Requires agent-sandbox v1.0.8+. Apply each independently and re-run the walkthrough. `kubectl apply -f manifests/` -is non-recursive, so the optional manifest is not applied by the base step. +is non-recursive, so the optional manifests are not applied by the base step. ## Troubleshooting @@ -599,7 +516,7 @@ is non-recursive, so the optional manifest is not applied by the base step. | `manifest.webmanifest` returns `401` in the console | Browsers fetch the Web App Manifest without credentials, so the gateway's session cookie is absent and the request fails closed. The UI is unaffected; only the PWA install prompt is lost | | Signup is slow / `202` then `Waking` | Warm pool exhausted (`kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox`); scale the pool | | `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `manifests/50-gateway.yaml`) | -| Gateway pod `ErrImagePull` / `ImagePullBackOff` (especially on kind) | The manifest uses `imagePullPolicy: IfNotPresent` with the release identity. Build with `make gateway`, retag to exactly the manifest's image, `kind load docker-image `, or set the Deployment image to your local tag / pinned digest. On a real cluster, make sure the GHCR package is readable (public or a pull secret) | +| Gateway pod `ErrImagePull` / `ImagePullBackOff` (especially on kind) | The manifest uses `imagePullPolicy: IfNotPresent` with the release identity. Build with `make -C ../.. gateway`, retag to exactly the manifest's image, `kind load docker-image `, or set the Deployment image to your local tag / pinned digest. On a real cluster, make sure the GHCR package is readable (public or a pull secret) | | Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` | | Resume crash-loops: `credentials-local: … is readable beyond its owner (mode 660)` | The pod template lacks `fsGroupChangePolicy: OnRootMismatch`, so the kubelet's default fsGroup policy rewrote `.credentials.yaml` to `0660`; DSH requires `0600` and refuses to start. New installs get the policy from the template. For an existing sandbox: suspend it, run a throwaway pod on `dsh-home-$SB` to `chmod 600 /home/node/.dsh/.credentials.yaml`, patch the sandbox's `podTemplate` with the policy, then resume | | Agents get `Permission denied` writing to `/workspace` on a fresh or resumed pod | The `/workspace` mount is the `dsh-home` PVC's `workspace/` subPath, and the kubelet is known to leave a `subPath` directory at `0755` if it is interrupted between creating and chmodding it ([kubernetes#142464](https://github.com/kubernetes/kubernetes/issues/142464)); later mounts never correct it. Suspend the sandbox, run a throwaway pod on `dsh-home-$SB` mounted at `/data`, `chmod 2775 /data/workspace` (do **not** chown/chmod recursively — that rewrites `.credentials.yaml`), then resume. If you recreate the sandbox instead, its current `/workspace` contents go with the PVC | @@ -666,12 +583,6 @@ Deliberately **not** production code: - **Bearer tokens in URLs.** Avoid putting the signup token in a URL/log; the browser flow uses the `HttpOnly` cookie instead. -## Cost model - -The same economics as the upstream pattern: suspend-when-idle plus warm pools turn -an always-on instance into a sub-second cold-start pause per return-after-absence, -at a fraction of the cost. - ## Cleanup ```bash diff --git a/examples/dsh-agents-as-a-service/docs/architecture.md b/examples/dsh-agents-as-a-service/docs/architecture.md index 9ef2413..8028c4c 100644 --- a/examples/dsh-agents-as-a-service/docs/architecture.md +++ b/examples/dsh-agents-as-a-service/docs/architecture.md @@ -6,7 +6,8 @@ Kubernetes, and DSH Web fit together, and why the design looks the way it does. Scope: one Kubernetes namespace (`dsh-demo`), one shared DSH Web image, one gateway deployment, and one DSH Web pod + PVC per user. The cluster is assumed to run -[agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) v1.0.8 with the extensions +[agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) with the pinned +`AGENT_SANDBOX_VERSION` from [`versions.env`](../../../versions.env) and the extensions (`SandboxTemplate`, `SandboxWarmPool`, `SandboxClaim`). ## Container view @@ -340,7 +341,7 @@ Notes and limits: | Probe on `/` | Pod `CrashLoopBackOff`, `401` in probe logs | Use `/favicon.svg` for readiness/liveness (shipped templates do) | | Warm pool exhausted | JSON signup returns `202` (state `Waking`); form signup is slow | Increase `replicas` in the warm pool | | Gateway restart | Idle sandboxes stop suspending; handshake cache lost | Rebuilt lazily on next request; consider a shared store for production | -| Browser aborts a request (tab reload, back navigation) during a slow wake or handshake | No response reaches the client; nothing appears in the access log | If the abort lands before the response starts, the body read raises `ClientDisconnect` and the app answers `499` (`CLIENT_CLOSED_REQUEST`), which cannot reach the departed peer. Nothing reached DSH, no ERROR traceback is logged, and the in-flight slot is released. A request already streaming ends with its upstream status. Before the fix every mid-body abort logged a `ClientDisconnect` traceback | +| Browser aborts a request (tab reload, back navigation) during a slow wake or handshake | No response reaches the client; nothing appears in the access log | If the abort lands before the response starts, the body read raises `ClientDisconnect` and the app answers `499` (`CLIENT_CLOSED_REQUEST`), which cannot reach the departed peer. Nothing reached DSH, no ERROR traceback is logged, and the in-flight slot is released. A request already streaming ends with its upstream status. | | DSH auth cookie expired mid-session | Proxy `401` once, then recovers | Gateway refreshes the handshake once | | Upstream pod name changed after resume | Cached cookie keyed by old pod UID is useless | Cache is invalidated on suspend; next request re-handshakes | | Two gateway replicas | Double suspend/wake races | Keep `replicas: 1` until state is externalized | diff --git a/examples/dsh-agents-as-a-service/docs/optional-hardening.md b/examples/dsh-agents-as-a-service/docs/optional-hardening.md index 8f41d2a..46a38ce 100644 --- a/examples/dsh-agents-as-a-service/docs/optional-hardening.md +++ b/examples/dsh-agents-as-a-service/docs/optional-hardening.md @@ -3,7 +3,7 @@ The base example is deliberately portable: it runs on kind and GKE without a gVisor runtime, a tainted node pool or a NetworkPolicy-capable CNI, and it boots with only the required `dsh-provider-keys` Secret. This document covers -four **opt-in** isolation layers, plus one availability setting: +four **opt-in** controls plus failed-Pod recovery: | Setting | Where it lives | What it changes | | --- | --- | --- | @@ -52,7 +52,7 @@ PodSpec; no CRD change is involved. - [`manifests/50-gateway.yaml`](../manifests/50-gateway.yaml) - [`manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml`](../manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml) (only with the SSO variant) -- [minimal sandbox-template.yaml](../../minimal-single-tenant/manifests/sandbox-template.yaml) +- [minimal template](../../minimal-single-tenant/manifests/10-sandboxtemplate.yaml) **Existing install:** a template edit does not update existing Sandboxes, and warm-pool spares use `OnReplenish`, so patch explicitly. Plan a maintenance @@ -604,7 +604,7 @@ Required locally for any change to this example: ```bash make check # versions, example layout, manifest YAML shape yamllint . -make test # gateway unit tests (gateway code is untouched) +make test # only when the gateway code changed ``` CI additionally parses every manifest and runs `kubectl apply diff --git a/examples/dsh-agents-as-a-service/gateway/README.md b/examples/dsh-agents-as-a-service/gateway/README.md index 9185183..8a91504 100644 --- a/examples/dsh-agents-as-a-service/gateway/README.md +++ b/examples/dsh-agents-as-a-service/gateway/README.md @@ -22,7 +22,7 @@ It does four things: The DSH version pin lives in [`versions.env`](../../../versions.env) (`DSH_VERSION` plus the optional repository `RELEASE_REVISION`); the example manifests and published images use the release identity built from it (currently -`dsh-v0.2.1-rc.5`). DSH binds `:3080`, serves `/`, gates `/api` behind a +`dsh-v0.2.1-rc.5`). DSH binds `:3080`, gates the UI and `/api` behind a `dsh-auth-*` cookie and requires a WebSocket at `/api/remote.mux`. ## Flow @@ -174,13 +174,17 @@ can also be injected into `create_app` for tests. | `SWEEP_INTERVAL` | `15` | Seconds between idle sweeps. | | `AUTH_PROXY_HEADER` | unset | Opt-in SSO: name of the trusted identity header set by an authenticating reverse proxy (e.g. `X-Forwarded-User`). Unset disables SSO and the header is never trusted. Enable only when the Service is reachable exclusively through that proxy (see [SSO](#sso-opt-in)). | +The shipped `manifests/50-gateway.yaml` sets `IDLE_TIMEOUT=360` and +`WAKE_TIMEOUT=240` for the demo; the e2e script overrides them to 60/120 so the +smoke test does not wait minutes for the sweeper. + ### Required RBAC The gateway's ServiceAccount needs, in its namespace: * `sandboxclaims` (`extensions.agents.x-k8s.io`): get, list, create, delete -* `sandboxes` (`agents.x-k8s.io`): get, patch -* `pods`: get +* `sandboxes` (`agents.x-k8s.io`): get, list, patch +* `pods`: get, list * `pods/log`: get ## Running locally diff --git a/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml b/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml index eeb485e..4ab56f0 100644 --- a/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml +++ b/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml @@ -69,7 +69,7 @@ spec: # OnRootMismatch skips that walk once the volume root already carries # the fsGroup ownership/mode applied on the first mount. # Keep in sync with the minimal variant: - # examples/minimal-single-tenant/manifests/sandbox-template.yaml. + # examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml. fsGroupChangePolicy: OnRootMismatch containers: - name: dsh-web @@ -103,7 +103,7 @@ spec: # sessions, config and credentials are unaffected. Existing # sandboxes stamp pod env at creation and keep the old HOME until # recreated. Keep in sync with the minimal variant: - # examples/minimal-single-tenant/manifests/sandbox-template.yaml. + # examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml. - name: HOME value: /workspace # Empty => DSH derives links from the request Host, which the diff --git a/examples/minimal-single-tenant/README.md b/examples/minimal-single-tenant/README.md index c807b80..701e534 100644 --- a/examples/minimal-single-tenant/README.md +++ b/examples/minimal-single-tenant/README.md @@ -48,7 +48,7 @@ kubectl -n dsh-demo create secret generic dsh-provider-keys-extra \ --from-literal=NVIDIA_API_KEY="nvapi-..." # 2. The template itself (image pinned in the manifest). -kubectl apply -f manifests/sandbox-template.yaml +kubectl apply -f manifests/10-sandboxtemplate.yaml # 3. A template only describes pods; a warm pool turns it into running # Sandboxes and a claim adopts one. One replica is enough here. @@ -103,16 +103,19 @@ kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080 ## Folder picker and the workspace -The template sets `HOME=/workspace`. DSH's directory picker (the folder explorer -used to choose a workspace) has no start-path setting: it lists `$HOME` when it -opens, so with the image default (`/home/node`) it would start on a directory that -is not persisted. Pointing `HOME` at the PVC-backed workspace makes the picker -open there (labelled *Home* in its breadcrumbs) and makes `~` in agent commands -resolve to `/workspace`. The trade-off is that `$HOME`-relative tool caches and -config (`.cache`, `.config`, `.local`, `.npm`, `.agents`) are written into the -workspace on the PVC. DSH state is unaffected: `DSH_HOME` stays +The template mounts the `dsh-home` PVC's `workspace/` subPath at +`/workspace` and sets `HOME=/workspace`. DSH's directory picker (the folder +explorer used to choose a workspace) has no start-path setting: it lists `$HOME` +when it opens, so with the image default (`/home/node`) it would start on a +directory that is not persisted. Pointing `HOME` at the PVC-backed workspace makes +the picker open there (labelled *Home* in its breadcrumbs) and makes `~` in agent +commands resolve to `/workspace`. The trade-off is that `$HOME`-relative tool +caches and config (`.cache`, `.config`, `.local`, `.npm`, `.agents`) are written +into the workspace on the PVC. DSH state is unaffected: `DSH_HOME` stays `/home/node/.dsh`. The rationale and the list of affected paths are in [`10-sandboxtemplate.yaml`](../dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml). +Pod specs are stamped at creation, so an existing sandbox keeps its old mounts and +`HOME` until it is recreated; the PVC retains DSH state and workspace data. Verify inside a running sandbox: @@ -133,7 +136,7 @@ NetworkPolicy are documented in ## Probes and trusted hosts -Two settings in `manifests/sandbox-template.yaml` are easy to get wrong: +Two settings in `manifests/10-sandboxtemplate.yaml` are easy to get wrong: - **Probe path.** `/` returns `401` without the launch-token cookie, so a probe there crash-loops the pod. The template probes `/favicon.svg` (served diff --git a/examples/minimal-single-tenant/manifests/sandbox-template.yaml b/examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml similarity index 93% rename from examples/minimal-single-tenant/manifests/sandbox-template.yaml rename to examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml index 6337a86..482ddc8 100644 --- a/examples/minimal-single-tenant/manifests/sandbox-template.yaml +++ b/examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml @@ -107,6 +107,14 @@ spec: volumeMounts: - name: dsh-home mountPath: /home/node/.dsh # DSH_HOME + # The agent workspace (the image's WORKDIR) is the same PVC under + # its own subPath, so DSH state - including .credentials.yaml - + # never shows up as workspace data. Kubelet creates workspace/ on + # first mount with the volume's fsGroup permissions; no init + # container is needed. See the multi-tenant template for details. + - name: dsh-home + mountPath: /workspace + subPath: workspace # / is behind DSH's launch-token cookie (401 without it); use an # unauthenticated path for probes. The startup probe gives a cold # start (plugin activation on a small node, or the first resume of a diff --git a/scripts/check-examples.sh b/scripts/check-examples.sh index e0dcf40..8e06068 100755 --- a/scripts/check-examples.sh +++ b/scripts/check-examples.sh @@ -20,6 +20,13 @@ for dir in examples/*/; do if [ -d "${dir}manifests" ]; then count=$(find "${dir}manifests" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ') [ "$count" -gt 0 ] || err "$name: manifests/ has no top-level *.yaml files" + for manifest in "${dir}"manifests/*.yaml; do + [ -e "$manifest" ] || continue + case "$(basename "$manifest")" in + [0-9][0-9]-*.yaml) ;; + *) err "$name: top-level manifest $(basename "$manifest") must be named NN-*.yaml" ;; + esac + done fi stray=$(find "$dir" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ') diff --git a/scripts/check-versions.sh b/scripts/check-versions.sh index 6e124b3..002248d 100755 --- a/scripts/check-versions.sh +++ b/scripts/check-versions.sh @@ -33,7 +33,7 @@ expect .env.sample "RELEASE_REVISION=$RELEASE_REVISION" # Image references in the example manifests for f in \ examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml \ - examples/minimal-single-tenant/manifests/sandbox-template.yaml; do + examples/minimal-single-tenant/manifests/10-sandboxtemplate.yaml; do expect "$f" "image: $GHCR_IMAGE:$IDENTITY" done expect examples/dsh-agents-as-a-service/manifests/50-gateway.yaml "image: $GATEWAY_IMAGE:$IDENTITY"