From e6e109d9a91b3d0e91d4ea84e23cba76eea52137 Mon Sep 17 00:00:00 2001 From: SREDevOpsAgents Date: Sun, 25 Oct 2026 06:37:28 -0300 Subject: [PATCH] feat(repo)!: restructure around the dsh-harness image and one example per implementation - the root Dockerfile is the primary artifact; versions.env is the single source of truth - merge gateway-api/dsh-demo into examples/dsh-agents-as-a-service (manifests/, exposure variants) - move the single-tenant template into examples/minimal-single-tenant - publish the gateway as a nested package under the repository image - mirror the DSH ref for release identities (first release dsh-v0.2.1-rc.2-r2) - CI: lint/python/manifests/gateway-image, reusable multi-arch build, tag-driven release, weekly e2e - docs: README, docs/{main-image,examples,gitflow,versioning}, example READMEs, AGENTS.md, governance - SSO is opt-in again; gateway pull policy and cookie defaults match the documented demo BREAKING CHANGE: example manifest paths, the gateway image name, and the removal of deploy/ and gateway-api/ affect existing deployments; see the migration table in README.md. --- .dockerignore | 155 +----- .editorconfig | 15 + .env.sample | 30 +- .github/ISSUE_TEMPLATE/bug_report.yml | 42 ++ .github/ISSUE_TEMPLATE/config.yml | 8 + .github/ISSUE_TEMPLATE/feature_request.yml | 35 ++ .github/dependabot.yml | 28 + .github/pull_request_template.md | 31 ++ .github/workflows/_build-image.yaml | 160 ++++++ .github/workflows/build-gateway.yaml | 69 +++ .github/workflows/build-main.yaml | 83 +++ .github/workflows/ci.yaml | 257 +++++++++ .github/workflows/codeql.yml | 6 +- .github/workflows/e2e-kind.yaml | 78 +++ .github/workflows/multi-build-gateway.yaml | 277 ---------- .github/workflows/multi-build.yaml | 295 ----------- .github/workflows/pr-title.yaml | 38 ++ .github/workflows/release.yaml | 268 ++++++++++ .github/workflows/validate.yaml | 137 ----- .gitignore | 5 + .hadolint.yaml | 6 + .lychee.toml | 13 + .yamllint.yml | 19 + AGENTS.md | 140 +++++ CODE_OF_CONDUCT.md | 46 ++ CONTRIBUTING.md | 88 ++++ Dockerfile | 49 +- LICENSE | 21 + Makefile | 85 ++- README.md | 324 ++++++------ SECURITY.md | 41 ++ build.sh | 69 +-- cliff.toml | 46 ++ docs/README.md | 38 ++ docs/examples.md | 149 ++++++ docs/gitflow.md | 128 +++++ docs/main-image.md | 169 ++++++ docs/versioning.md | 163 ++++++ examples/README.md | 47 ++ examples/dsh-agents-as-a-service/README.md | 487 ++++++++++++------ .../docs}/architecture.md | 24 +- .../gateway/.dockerignore | 11 + .../dsh-agents-as-a-service/gateway/README.md | 66 ++- .../gateway/gateway.py | 147 ++---- .../gateway/requirements-dev.txt | 1 + .../gateway/test_gateway.py | 140 ++--- .../{ => manifests}/00-prereqs.yaml | 2 +- .../{ => manifests}/10-sandboxtemplate.yaml | 18 +- .../{ => manifests}/20-sandboxwarmpool.yaml | 0 .../{ => manifests}/30-claim-alice.yaml | 0 .../{ => manifests}/40-claim-rejected.yaml | 0 .../{ => manifests}/50-gateway.yaml | 55 +- .../manifests/exposure/README.md | 135 +++++ .../exposure/gateway-api-sso/70-gateway.yaml | 5 + .../gateway-api-sso/71-httproute.yaml | 2 + .../gateway-api-sso/72-oauth2-proxy.yaml | 21 +- .../73-oauth2-proxy-secret.yaml.example | 0 .../74-tls-secret.yaml.example | 2 +- .../exposure/ingress}/60-ingress.yaml | 22 +- .../{run-test-kind.sh => scripts/e2e-kind.sh} | 62 ++- examples/minimal-single-tenant/README.md | 127 +++++ .../manifests}/sandbox-template.yaml | 24 +- gateway-api/dsh-demo/README.md | 91 ---- scripts/check-examples.sh | 56 ++ scripts/check-manifests.py | 47 ++ scripts/check-versions.sh | 70 +++ scripts/set-version.sh | 57 ++ versions.env | 26 + 68 files changed, 3713 insertions(+), 1643 deletions(-) create mode 100644 .editorconfig create mode 100644 .github/ISSUE_TEMPLATE/bug_report.yml create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/ISSUE_TEMPLATE/feature_request.yml create mode 100644 .github/dependabot.yml create mode 100644 .github/pull_request_template.md create mode 100644 .github/workflows/_build-image.yaml create mode 100644 .github/workflows/build-gateway.yaml create mode 100644 .github/workflows/build-main.yaml create mode 100644 .github/workflows/ci.yaml create mode 100644 .github/workflows/e2e-kind.yaml delete mode 100644 .github/workflows/multi-build-gateway.yaml delete mode 100644 .github/workflows/multi-build.yaml create mode 100644 .github/workflows/pr-title.yaml create mode 100644 .github/workflows/release.yaml delete mode 100644 .github/workflows/validate.yaml create mode 100644 .hadolint.yaml create mode 100644 .lychee.toml create mode 100644 .yamllint.yml create mode 100644 AGENTS.md create mode 100644 CODE_OF_CONDUCT.md create mode 100644 CONTRIBUTING.md create mode 100644 LICENSE create mode 100644 SECURITY.md create mode 100644 cliff.toml create mode 100644 docs/README.md create mode 100644 docs/examples.md create mode 100644 docs/gitflow.md create mode 100644 docs/main-image.md create mode 100644 docs/versioning.md create mode 100644 examples/README.md rename {docs => examples/dsh-agents-as-a-service/docs}/architecture.md (90%) create mode 100644 examples/dsh-agents-as-a-service/gateway/.dockerignore rename examples/dsh-agents-as-a-service/{ => manifests}/00-prereqs.yaml (95%) rename examples/dsh-agents-as-a-service/{ => manifests}/10-sandboxtemplate.yaml (88%) rename examples/dsh-agents-as-a-service/{ => manifests}/20-sandboxwarmpool.yaml (100%) rename examples/dsh-agents-as-a-service/{ => manifests}/30-claim-alice.yaml (100%) rename examples/dsh-agents-as-a-service/{ => manifests}/40-claim-rejected.yaml (100%) rename examples/dsh-agents-as-a-service/{ => manifests}/50-gateway.yaml (71%) create mode 100644 examples/dsh-agents-as-a-service/manifests/exposure/README.md rename gateway-api/dsh-demo/gateway.yaml => examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/70-gateway.yaml (54%) rename gateway-api/dsh-demo/httproute.yaml => examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/71-httproute.yaml (79%) rename gateway-api/dsh-demo/oauth2-proxy.yaml => examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml (76%) rename gateway-api/dsh-demo/oauth2-proxy-secret.yaml.example => examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/73-oauth2-proxy-secret.yaml.example (100%) rename gateway-api/dsh-demo/tls-secret.yaml.example => examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/74-tls-secret.yaml.example (91%) rename examples/dsh-agents-as-a-service/{ => manifests/exposure/ingress}/60-ingress.yaml (59%) rename examples/dsh-agents-as-a-service/{run-test-kind.sh => scripts/e2e-kind.sh} (81%) create mode 100644 examples/minimal-single-tenant/README.md rename {deploy/k8s => examples/minimal-single-tenant/manifests}/sandbox-template.yaml (78%) delete mode 100644 gateway-api/dsh-demo/README.md create mode 100755 scripts/check-examples.sh create mode 100644 scripts/check-manifests.py create mode 100755 scripts/check-versions.sh create mode 100755 scripts/set-version.sh create mode 100644 versions.env diff --git a/.dockerignore b/.dockerignore index 1301e01..d1558dd 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,149 +1,6 @@ -# Logs -logs -*.log -npm-debug.log* -yarn-debug.log* -yarn-error.log* -lerna-debug.log* - -# Diagnostic reports (https://nodejs.org/api/report.html) -report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json - -# Runtime data -pids -*.pid -*.seed -*.pid.lock - -# Directory for instrumented libs generated by jscoverage/JSCover -lib-cov - -# Coverage directory used by tools like istanbul -coverage -*.lcov - -# nyc test coverage -.nyc_output - -# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) -.grunt - -# Bower dependency directory (https://bower.io/) -bower_components - -# node-waf configuration -.lock-wscript - -# Compiled binary addons (https://nodejs.org/api/addons.html) -build/Release - -# Dependency directories -node_modules/ -jspm_packages/ - -# Snowpack dependency directory (https://snowpack.dev/) -web_modules/ - -# TypeScript cache -*.tsbuildinfo - -# Optional npm cache directory -.npm - -# Optional eslint cache -.eslintcache - -# Optional stylelint cache -.stylelintcache - -# Optional REPL history -.node_repl_history - -# Output of 'npm pack' -*.tgz - -# Yarn Integrity file -.yarn-integrity - -# dotenv environment variable files -.env -.env.* -!.env.example - -# parcel-bundler cache (https://parceljs.org/) -.cache -.parcel-cache - -# Next.js build output -.next -out - -# Nuxt.js build / generate output -.nuxt -dist -.output - -# Gatsby files -.cache/ -# Comment in the public line in if your project uses Gatsby and not Next.js -# https://nextjs.org/blog/next-9-1#public-directory-support -# public - -# vuepress build output -.vuepress/dist - -# vuepress v2.x temp directory -.temp - -# Sveltekit cache directory -.svelte-kit/ - -# vitepress build output -**/.vitepress/dist - -# vitepress cache directory -**/.vitepress/cache - -# Docusaurus cache and generated files -.docusaurus - -# Serverless directories -.serverless/ - -# FuseBox cache -.fusebox/ - -# DynamoDB Local files -.dynamodb/ - -# Firebase cache directory -.firebase/ - -# TernJS port file -.tern-port - -# Stores Visual Studio Code versions used for testing Visual Studio Code extensions -.vscode-test - -# pnpm -.pnpm-store - -# yarn v3 -.pnp.* -.yarn/* -!.yarn/patches -!.yarn/plugins -!.yarn/releases -!.yarn/sdks -!.yarn/versions - -# Vite files -vite.config.js.timestamp-* -vite.config.ts.timestamp-* -.vite/ - -# Astro generated types -.astro/ - -# Turbo Repo -.turbo +# The main image build clones DeepSeek Harness from git and copies nothing +# from this repository, so the build context is intentionally reduced to the +# Dockerfile alone. This keeps builds fast and prevents the rest of the repo +# (examples, docs, git history) from being sent to the builder. +* +!Dockerfile diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..2e0d78c --- /dev/null +++ b/.editorconfig @@ -0,0 +1,15 @@ +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true +indent_style = space +indent_size = 2 + +[*.py] +indent_size = 4 + +[Makefile] +indent_style = tab diff --git a/.env.sample b/.env.sample index 2f919ab..504df86 100644 --- a/.env.sample +++ b/.env.sample @@ -1,29 +1,35 @@ # Copy to .env and fill in. Never commit a real key. +# +# The version values below are documentation only: versions.env is the single +# source of truth, and scripts/check-versions.sh fails if they drift. -# --- DSH Web image (Dockerfile / build.sh) --- +# --- Main DSH Web image (root Dockerfile) --- DEEPSEEK_API_KEY= DSH_VERSION=dsh-v0.2.1-rc.2 +RELEASE_REVISION=r2 PNPM_VERSION=11.28.5 -# DSH Web port inside the container, and the local port published by -# build.sh / docker run. The gateway also uses it as the upstream port. +# Container port, and the host port published by "make run". DSH_PORT=3080 -# Comma/space separated extra Host allow-list entries for DSH (for an -# Ingress/public hostname). localhost and 127.0.0.1 are always trusted. +# Comma/space separated extra Host allow-list entries for DSH (for a proxy or +# public hostname). localhost and 127.0.0.1 are always trusted. DSH_TRUSTED_HOSTS= -# --- Gateway (examples/dsh-agents-as-a-service) --- -GATEWAY_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway +# --- agents-as-a-service example --- NAMESPACE=dsh-demo POOL=dsh-pool -# Seconds a sandbox may stay idle before the sweeper suspends it. -IDLE_TIMEOUT=60 -# Seconds a waking request may wait for the sandbox to become Ready. -WAKE_TIMEOUT=120 +# Seconds a sandbox may stay idle before the sweeper suspends it, and the max +# time a proxied request waits for a suspended sandbox to become Ready. +IDLE_TIMEOUT=360 +WAKE_TIMEOUT=240 DSH_CONTAINER=dsh-web DSH_LOG_TAIL_LINES=2000 COOKIE_NAME=dsh_gw_session -# Set to true only behind HTTPS; secure cookies are not sent over plain HTTP. +# Set to true only behind HTTPS; secure cookies are not sent over plain HTTP +# (localhost counts as a secure context in browsers). COOKIE_SECURE=false COOKIE_MAX_AGE=604800 MAX_BODY_MB=32 HTTP_TIMEOUT=300 +# SSO is opt-in. Set this only when the gateway Service is reachable +# exclusively through oauth2-proxy (see the example exposure docs). +# AUTH_PROXY_HEADER=X-Forwarded-User diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000..03b166c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,42 @@ +name: Bug report +description: Report a reproducible problem with the image, an example or the tooling. +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + Do not include API keys, tokens or other secrets. Report security + issues privately via SECURITY.md instead. + - type: textarea + id: what-happened + attributes: + label: What happened + description: The observed behavior and the expected behavior. + validations: + required: true + - type: textarea + id: reproduce + attributes: + label: Reproduction steps + description: Commands, manifests and versions. Redact secrets. + placeholder: | + make image + docker run ... + validations: + required: true + - type: input + id: image-tag + attributes: + label: Image or release + description: The pinned tag or digest, for example dsh-v0.2.1-rc.2-r2. + - type: input + id: cluster + attributes: + label: Environment + description: Kubernetes distribution/version, kind version, agent-sandbox version. + - type: textarea + id: logs + attributes: + label: Relevant logs + description: Paste the smallest useful excerpt. Remove tokens and API keys. + render: shell diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..c22a459 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: false +contact_links: + - name: Security vulnerability + url: https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/security/advisories/new + about: Report security issues privately; do not open a public issue. + - name: Documentation + url: https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/tree/main/docs + about: Read the repository docs before opening an issue. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 0000000..a715804 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,35 @@ +name: Feature request +description: Propose a new example, image capability or workflow improvement. +labels: ["enhancement"] +body: + - type: textarea + id: problem + attributes: + label: Problem or use case + description: What are you trying to do, and what blocks you today? + validations: + required: true + - type: textarea + id: proposal + attributes: + label: Proposed solution + description: Describe the shape you have in mind. For a new example, say which implementation it demonstrates. + validations: + required: true + - type: dropdown + id: area + attributes: + label: Area + options: + - Main DSH Web image + - Existing example + - New example + - CI / release automation + - Documentation + - Other + validations: + required: true + - type: textarea + id: alternatives + attributes: + label: Alternatives considered diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..e42954b --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,28 @@ +version: 2 + +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + groups: + actions: + patterns: ["*"] + + - package-ecosystem: pip + directory: /examples/dsh-agents-as-a-service/gateway + schedule: + interval: weekly + groups: + gateway-python: + patterns: ["*"] + + - package-ecosystem: docker + directory: / + schedule: + interval: weekly + + - package-ecosystem: docker + directory: /examples/dsh-agents-as-a-service/gateway + schedule: + interval: weekly diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..2979071 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,31 @@ +## Summary + + + +## Type of change + +- [ ] feat: new capability or example +- [ ] fix: bug fix +- [ ] docs: documentation only +- [ ] refactor: no behavior change +- [ ] build / ci: tooling, images, workflows +- [ ] chore: maintenance + +## Breaking change + +- [ ] Yes, and the migration is documented in the description +- [ ] No + +## Checklist + +- [ ] make check passes (versions, example layout, manifest YAML) +- [ ] make test passes (gateway unit tests) +- [ ] make lint passes (ruff, shellcheck; yamllint/actionlint where installed) +- [ ] Docs updated for the change (root README, docs/, example README) +- [ ] No secrets or usable credentials committed +- [ ] CHANGELOG.md was not edited by hand (generated by git-cliff) +- [ ] Version pins changed only through versions.env / make set-version + +## Testing + + diff --git a/.github/workflows/_build-image.yaml b/.github/workflows/_build-image.yaml new file mode 100644 index 0000000..0864591 --- /dev/null +++ b/.github/workflows/_build-image.yaml @@ -0,0 +1,160 @@ +# Reusable multi-arch image build: one workflow, called by build-main, +# build-gateway and release. Builds natively per platform (no QEMU), pushes by +# digest, then merges the manifest list with SBOM and provenance attestations. +name: Reusable multi-arch image build + +on: + workflow_call: + inputs: + image: + description: Fully qualified image name without a tag + required: true + type: string + version: + description: Version recorded in the OCI labels + required: true + type: string + context: + description: Docker build context + required: true + type: string + dockerfile: + description: Dockerfile path, relative to the context + required: false + default: Dockerfile + type: string + build-args: + description: Newline-separated build args + required: false + default: "" + type: string + tags: + description: Newline-separated fully qualified tags to publish + required: true + type: string + +permissions: + contents: read + packages: write + id-token: write + attestations: write + +jobs: + build: + name: Build ${{ matrix.platform }} + runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || 'ubuntu-24.04-arm' }} + timeout-minutes: 120 + strategy: + fail-fast: false + matrix: + platform: + - linux/amd64 + - linux/arm64 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Prepare environment + env: + IMAGE: ${{ inputs.image }} + run: | + set -euo pipefail + pair=$(printf '%s' "${{ matrix.platform }}" | tr '/' '-') + echo "PLATFORM_PAIR=$pair" >> "$GITHUB_ENV" + echo "CACHE_SCOPE=$(printf '%s' "$IMAGE" | tr '/' '-')" >> "$GITHUB_ENV" + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 + + - name: Login to GitHub Container Registry + uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Build and push by digest + id: build + uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 + with: + context: ${{ inputs.context }} + file: ${{ inputs.context }}/${{ inputs.dockerfile }} + platforms: ${{ matrix.platform }} + build-args: ${{ inputs.build-args }} + labels: | + org.opencontainers.image.source=https://github.com/${{ github.repository }} + org.opencontainers.image.version=${{ inputs.version }} + org.opencontainers.image.revision=${{ github.sha }} + outputs: type=image,name=${{ inputs.image }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true + sbom: true + provenance: mode=max + cache-from: type=gha,scope=${{ env.CACHE_SCOPE }}-${{ matrix.platform }} + cache-to: type=gha,mode=max,scope=${{ env.CACHE_SCOPE }}-${{ matrix.platform }} + + - name: Export digest + run: | + set -euo pipefail + mkdir -p /tmp/digests + digest="${{ steps.build.outputs.digest }}" + touch "/tmp/digests/${digest#sha256:}" + + - name: Upload digest + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: digests-${{ env.PLATFORM_PAIR }} + path: /tmp/digests/* + if-no-files-found: error + retention-days: 1 + + merge: + name: Merge manifest list + needs: build + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - name: Download digests + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + path: /tmp/digests + pattern: digests-* + merge-multiple: true + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 + + - name: Login to GitHub Container Registry + uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Create and push manifest list + working-directory: /tmp/digests + env: + IMAGE: ${{ inputs.image }} + TAGS: ${{ inputs.tags }} + run: | + set -euo pipefail + args=() + while IFS= read -r tag; do + if [ -n "$tag" ]; then + args+=("-t" "$tag") + fi + done <<< "$TAGS" + refs=() + for digest in *; do + refs+=("$IMAGE@sha256:$digest") + done + docker buildx imagetools create "${args[@]}" "${refs[@]}" + + - name: Inspect image + env: + IMAGE: ${{ inputs.image }} + TAGS: ${{ inputs.tags }} + run: | + set -euo pipefail + first=$(printf '%s\n' "$TAGS" | head -n1) + docker buildx imagetools inspect "$first" + echo "### Published ${{ inputs.image }}" >> "$GITHUB_STEP_SUMMARY" + printf '%s\n' "$TAGS" | sed 's/^/- /' >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/build-gateway.yaml b/.github/workflows/build-gateway.yaml new file mode 100644 index 0000000..9d898a1 --- /dev/null +++ b/.github/workflows/build-gateway.yaml @@ -0,0 +1,69 @@ +# Development channel for the example gateway image: a merge that touches the +# gateway publishes ghcr.io///dsh-agents-as-a-service/gateway:main. +name: Build gateway image + +on: + workflow_dispatch: + push: + branches: [main] + paths: + - examples/dsh-agents-as-a-service/gateway/** + - versions.env + - .github/workflows/build-gateway.yaml + - .github/workflows/_build-image.yaml + +permissions: + contents: read + packages: write + id-token: write + attestations: write + +concurrency: + group: build-gateway-${{ github.ref }} + cancel-in-progress: true + +jobs: + meta: + name: Read versions + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + identity: ${{ steps.meta.outputs.identity }} + gateway_image: ${{ steps.meta.outputs.gateway_image }} + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Read versions.env + id: meta + shell: bash + run: | + set -euo pipefail + while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + echo "$line" >> "$GITHUB_ENV" + done < versions.env + identity="$DSH_VERSION" + [ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION" + { + echo "identity=$identity" + echo "gateway_image=$GATEWAY_IMAGE" + } >> "$GITHUB_OUTPUT" + + build: + name: Build and push gateway image + needs: meta + uses: ./.github/workflows/_build-image.yaml + permissions: + contents: read + packages: write + id-token: write + attestations: write + with: + image: ${{ needs.meta.outputs.gateway_image }} + version: ${{ needs.meta.outputs.identity }} + context: examples/dsh-agents-as-a-service/gateway + dockerfile: Dockerfile + tags: | + ${{ needs.meta.outputs.gateway_image }}:main diff --git a/.github/workflows/build-main.yaml b/.github/workflows/build-main.yaml new file mode 100644 index 0000000..d36d700 --- /dev/null +++ b/.github/workflows/build-main.yaml @@ -0,0 +1,83 @@ +# Development channel for the main DSH Web image: every merge to main that +# touches the image definition publishes ghcr.io//:main. +# Releases are published by release.yaml with immutable version tags. +name: Build main image + +on: + workflow_dispatch: + push: + branches: [main] + paths: + - Dockerfile + - versions.env + - .github/workflows/build-main.yaml + - .github/workflows/_build-image.yaml + +permissions: + contents: read + packages: write + id-token: write + attestations: write + +concurrency: + group: build-main-${{ github.ref }} + cancel-in-progress: true + +jobs: + meta: + name: Read versions + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + identity: ${{ steps.meta.outputs.identity }} + dsh_version: ${{ steps.meta.outputs.dsh_version }} + pnpm_version: ${{ steps.meta.outputs.pnpm_version }} + release_revision: ${{ steps.meta.outputs.release_revision }} + ghcr_image: ${{ steps.meta.outputs.ghcr_image }} + gateway_image: ${{ steps.meta.outputs.gateway_image }} + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Read versions.env + id: meta + shell: bash + run: | + set -euo pipefail + while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + echo "$line" >> "$GITHUB_ENV" + done < versions.env + identity="$DSH_VERSION" + [ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION" + { + echo "identity=$identity" + echo "dsh_version=$DSH_VERSION" + echo "pnpm_version=$PNPM_VERSION" + echo "release_revision=$RELEASE_REVISION" + echo "ghcr_image=$GHCR_IMAGE" + echo "gateway_image=$GATEWAY_IMAGE" + } >> "$GITHUB_OUTPUT" + + build: + name: Build and push main image + needs: meta + uses: ./.github/workflows/_build-image.yaml + permissions: + contents: read + packages: write + id-token: write + attestations: write + with: + image: ${{ needs.meta.outputs.ghcr_image }} + version: ${{ needs.meta.outputs.identity }} + context: . + dockerfile: Dockerfile + build-args: | + DSH_VERSION=${{ needs.meta.outputs.dsh_version }} + RELEASE_REVISION=${{ needs.meta.outputs.release_revision }} + PNPM_VERSION=${{ needs.meta.outputs.pnpm_version }} + VCS_REF=${{ github.sha }} + tags: | + ${{ needs.meta.outputs.ghcr_image }}:main diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml new file mode 100644 index 0000000..4ca480f --- /dev/null +++ b/.github/workflows/ci.yaml @@ -0,0 +1,257 @@ +# Continuous integration: lint, unit tests, manifest dry-run, gateway image +# smoke test and dependency review. Job names are the required branch-protection +# checks documented in docs/gitflow.md. +name: CI + +on: + pull_request: + push: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + lint: + name: lint + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.12" + - name: Install Python linters + run: | + set -euo pipefail + python -m pip install --upgrade pip + pip install "yamllint==1.38.0" "pyyaml==6.0.3" + - name: Install actionlint + env: + ACTIONLINT_VERSION: v1.7.12 + run: | + set -euo pipefail + version="${ACTIONLINT_VERSION#v}" + curl -fsSL -o /tmp/actionlint.tar.gz "https://github.com/rhysd/actionlint/releases/download/$ACTIONLINT_VERSION/actionlint_${version}_linux_amd64.tar.gz" + tar -xzf /tmp/actionlint.tar.gz -C /tmp actionlint + install -m 0755 /tmp/actionlint /usr/local/bin/actionlint + actionlint -version + - name: Install hadolint + env: + HADOLINT_VERSION: v2.12.0 + run: | + set -euo pipefail + curl -fsSL -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/$HADOLINT_VERSION/hadolint-Linux-x86_64" + install -m 0755 /tmp/hadolint /usr/local/bin/hadolint + hadolint --version + - name: Repository checks + run: | + set -euo pipefail + ./scripts/check-versions.sh + ./scripts/check-examples.sh + python scripts/check-manifests.py + - name: Shellcheck + run: shellcheck -x build.sh scripts/*.sh examples/*/scripts/*.sh + - name: Actionlint + run: actionlint -color + - name: Yamllint + run: yamllint . + - name: Hadolint + run: | + set -euo pipefail + hadolint Dockerfile + hadolint examples/dsh-agents-as-a-service/gateway/Dockerfile + - name: Documentation links + uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2 + with: + args: >- + --offline --no-progress --verbose + ./README.md ./CONTRIBUTING.md ./SECURITY.md ./AGENTS.md ./docs ./examples + + python: + name: python + runs-on: ubuntu-latest + timeout-minutes: 20 + defaults: + run: + working-directory: examples/dsh-agents-as-a-service/gateway + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.12" + cache: pip + cache-dependency-path: | + examples/dsh-agents-as-a-service/gateway/requirements.txt + examples/dsh-agents-as-a-service/gateway/requirements-dev.txt + - name: Install dependencies + run: | + set -euo pipefail + python -m pip install --upgrade pip + pip install -r requirements.txt -r requirements-dev.txt + - name: Lint + run: | + set -euo pipefail + python -m ruff check . + python -m ruff format --check . + - name: Unit tests with coverage + run: >- + python -m pytest -q + --cov=. --cov-report=xml:coverage.xml --cov-report=term + --junitxml=junit.xml + - name: Coverage summary + run: | + set -euo pipefail + { + echo "### Gateway test coverage" + python -m coverage report + } >> "$GITHUB_STEP_SUMMARY" + - name: Upload test artefacts + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: gateway-test-results + path: | + examples/dsh-agents-as-a-service/gateway/junit.xml + examples/dsh-agents-as-a-service/gateway/coverage.xml + if-no-files-found: warn + retention-days: 7 + + manifests: + name: manifests + runs-on: ubuntu-latest + timeout-minutes: 25 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Create kind cluster + uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1 + with: + version: v0.33.0 + cluster_name: dsh-ci + - name: Read versions.env + run: | + set -euo pipefail + while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + echo "$line" >> "$GITHUB_ENV" + done < versions.env + - name: Install agent-sandbox and Gateway API CRDs + run: | + set -euo pipefail + kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml" + kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s + kubectl apply -f "https://github.com/kubernetes-sigs/gateway-api/releases/download/$GATEWAY_API_VERSION/standard-install.yaml" + kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/httproutes.gateway.networking.k8s.io --timeout=180s + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.12" + - name: Install check dependencies + run: pip install "pyyaml==6.0.3" + - name: Validate manifest shape + run: python scripts/check-manifests.py + - name: Validate manifests (server dry-run) + run: | + set -euo pipefail + kubectl create namespace dsh-demo + shopt -s globstar nullglob + count=0 + for file in examples/*/manifests/**/*.yaml; do + case "$file" in *.yaml.example) continue ;; esac + echo "== $file ==" + kubectl apply --dry-run=server -f "$file" + count=$((count + 1)) + done + if [ "$count" -eq 0 ]; then + echo "no manifests were validated" >&2 + exit 1 + fi + echo "validated $count manifests" + + gateway-image: + name: gateway-image + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Build gateway image + run: docker build -t dsh-aaas-gateway:ci examples/dsh-agents-as-a-service/gateway/ + - name: Smoke-test /healthz without a cluster + run: | + set -euo pipefail + cat > /tmp/kubeconfig <<'EOF' + apiVersion: v1 + kind: Config + clusters: + - name: dummy + cluster: + server: https://127.0.0.1:6443 + insecure-skip-tls-verify: true + contexts: + - name: dummy + context: + cluster: dummy + user: dummy + users: + - name: dummy + user: {} + current-context: dummy + EOF + docker run -d --name gateway-smoke \ + -p 18080:8080 \ + -e NAMESPACE=dsh-demo \ + -e POOL=dsh-pool \ + -e KUBECONFIG=/tmp/kubeconfig \ + -v /tmp/kubeconfig:/tmp/kubeconfig:ro \ + dsh-aaas-gateway:ci + trap 'docker logs gateway-smoke || true; docker rm -f gateway-smoke || true' EXIT + code="" + for _ in $(seq 1 30); do + code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:18080/healthz || true) + [ "$code" = "200" ] && break + sleep 1 + done + if [ "$code" != "200" ]; then + echo "gateway /healthz returned '$code' instead of 200" >&2 + exit 1 + fi + echo "gateway image healthy: GET /healthz -> 200" + + dependency-review: + name: dependency-review + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + pull-requests: write + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Dependency review + uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4 diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 9c424c5..ba03c11 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -57,7 +57,7 @@ jobs: # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages steps: - name: Checkout repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 # Add any setup steps before running the `github/codeql-action/init` action. # This includes steps like installing compilers or runtimes (`actions/setup-node` @@ -67,7 +67,7 @@ jobs: # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@v4 + uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4 with: languages: ${{ matrix.language }} build-mode: ${{ matrix.build-mode }} @@ -96,6 +96,6 @@ jobs: exit 1 - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v4 + uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4 with: category: "/language:${{matrix.language}}" diff --git a/.github/workflows/e2e-kind.yaml b/.github/workflows/e2e-kind.yaml new file mode 100644 index 0000000..9a514ba --- /dev/null +++ b/.github/workflows/e2e-kind.yaml @@ -0,0 +1,78 @@ +# Weekly end-to-end run of the agents-as-a-service example against a real kind +# cluster with agent-sandbox installed. Scheduled (not PR-blocking) so upstream +# controller churn does not block merges; also available on demand. +name: E2E (kind) + +on: + schedule: + - cron: "0 6 * * 1" + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: e2e-kind + cancel-in-progress: false + +jobs: + e2e: + name: Example walkthrough on kind + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Create kind cluster + uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1 + with: + version: v0.33.0 + cluster_name: dsh-e2e + - name: Read versions.env + run: | + set -euo pipefail + while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + echo "$line" >> "$GITHUB_ENV" + done < versions.env + - name: Install agent-sandbox + run: | + set -euo pipefail + kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml" + kubectl -n agent-sandbox-system rollout status deployment --timeout=180s + kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s + kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s + - name: Set up Python (WebSocket checks) + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.12" + - name: Install WebSocket client + run: pip install "websockets==17.3" + - name: Run the example walkthrough + env: + KIND_CLUSTER: dsh-e2e + run: examples/dsh-agents-as-a-service/scripts/e2e-kind.sh + - name: Collect cluster state + if: always() + run: | + set -euo pipefail + { + echo "== resources ==" + kubectl get all,pvc -n dsh-demo -o wide 2>&1 || true + echo "== events ==" + kubectl get events -n dsh-demo --sort-by=.lastTimestamp 2>&1 || true + echo "== gateway logs ==" + kubectl -n dsh-demo logs deploy/dsh-gateway --tail=200 2>&1 || true + } > /tmp/e2e-cluster.txt + - name: Upload cluster state + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: e2e-cluster-state + path: /tmp/e2e-cluster.txt + if-no-files-found: warn + retention-days: 7 diff --git a/.github/workflows/multi-build-gateway.yaml b/.github/workflows/multi-build-gateway.yaml deleted file mode 100644 index 261d525..0000000 --- a/.github/workflows/multi-build-gateway.yaml +++ /dev/null @@ -1,277 +0,0 @@ -# This workflow builds the multi-arch gateway image (Starlette ASGI, served by -# uvicorn on :8080) and pushes it to GHCR as -# ghcr.io//:gateway. The DSH Web image is built by multi-build.yaml. -# It uses Docker Buildx with native AMD64 and ARM64 runners, without QEMU. -name: Gateway Build multi arch Docker Image with separate Github Runners - -on: - workflow_dispatch: - push: - branches: - - main - paths: - - 'examples/dsh-agents-as-a-service/gateway/**' - - '.github/workflows/multi-build-gateway.yaml' -env: - # ghcr.io//; this workflow always publishes the `:gateway` tag. - GHCR_IMAGE: ghcr.io/${{ github.repository }} -permissions: - # Global permissions for the workflow, which can be overridden at the job level - contents: read - -concurrency: - # This concurrency group ensures that only one job in the group runs at a time. - # If a new job is triggered, the previous one will be canceled. - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - # The build job builds the Docker image for each platform specified in the matrix. - build: - strategy: - fail-fast: false - matrix: - platform: - - linux/amd64 - - linux/arm64 - # The matrix includes two platforms: linux/amd64 and linux/arm64. - # The build job will run for each platform in the matrix. - - permissions: - # Permissions for the build job, which can be overridden at the step level - # The permissions are set to allow the job to write to the GitHub Container Registry (GHCR) and read from the repository. - attestations: write - actions: read - checks: write - contents: write - deployments: none - id-token: write - issues: read - discussions: read - packages: write - pages: none - pull-requests: read - repository-projects: read - security-events: read - statuses: read - - runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' }} - # The job runs on different runners based on the platform. - # For linux/amd64, it runs on the latest Ubuntu runner. - # For linux/arm64, it runs on an Ubuntu 24.04 ARM runner. - # The runner is selected based on the platform specified in the matrix. - - name: Build Docker image for ${{ matrix.platform }} - - steps: - - - name: Prepare environment for current platform - # This step sets up the environment for the current platform being built. - # It replaces the '/' character in the platform name with '-' and sets it as an environment variable. - # This is useful for naming artifacts and other resources that cannot contain '/'. - # The environment variable PLATFORMS_PAIR will be used later in the workflow. - id: prepare - run: | - platform=${{ matrix.platform }} - echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV - - - name: Checkout - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # This step checks out the code from the repository. - # It uses the actions/checkout action to clone the repository into the runner's workspace. - - - name: Docker meta default - # This step generates metadata for the Docker image. - # It uses the docker/metadata-action to create metadata based on the repository information. - # The metadata includes information such as the image name, tags, and labels. - # The metadata will be used later in the workflow to build and push the Docker image. - id: meta - uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0 - with: - images: ${{ env.GHCR_IMAGE }} - tags: | - type=raw,value=gateway - - - - name: Set up Docker Context for Buildx - # This step sets up a Docker context for Buildx. - # It creates a new context named "builders" that will be used for building the Docker image. - # The context allows Buildx to use the Docker daemon for building images. - id: buildx-context - run: | - docker context create builders - - - name: Set up Docker Buildx - # This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit. - # It uses the docker/setup-buildx-action to configure Buildx with the specified context and platforms. - # The platforms are specified in the matrix and will be used for building the Docker image. - uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 - with: - endpoint: builders - platforms: ${{ matrix.platform }} - - - name: Login to GitHub Container Registry - # This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action. - # It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication. - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - - name: Build and push by digest - # This step builds and pushes the Docker image using Buildx. - # It uses the docker/build-push-action to build the image with the specified context and platforms. - # The image is built with the labels and annotations generated in the previous steps. - # The outputs are configured to push the image by digest, which allows for better caching and versioning. - # The cache-from and cache-to options are used to enable caching for the build process. - # The cache is stored in GitHub Actions cache and is scoped to the repository, branch, and platform. - id: build - uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 - env: - DOCKER_BUILDKIT: 1 - with: - context: ./examples/dsh-agents-as-a-service/gateway/ - platforms: ${{ matrix.platform }} - labels: ${{ steps.meta.outputs.labels }} - annotations: ${{ steps.meta.outputs.annotations }} - outputs: type=image,name=${{ env.GHCR_IMAGE }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true - cache-from: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }} - cache-to: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }} - - - - name: Export digest - # This step exports the digest of the built image to a file. - # It creates a directory in /tmp/digests and saves the digest of the image to a file. - # The digest is obtained from the output of the build step. - # The digest is used to uniquely identify the built image and can be used for further processing or verification. - run: | - mkdir -p /tmp/digests - digest="${{ steps.build.outputs.digest }}" - touch "/tmp/digests/${digest#sha256:}" - - - name: Upload digest - # This step uploads the digest file to the GitHub Actions artifact storage. - # It uses the actions/upload-artifact action to upload the file created in the previous step. - # The artifact is named digests-${{ env.PLATFORM_PAIR }}, where PLATFORM_PAIR is the platform name with '/' replaced by '-'. - # The artifact is retained for 1 day, and if no files are found, it will throw an error. - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 - with: - name: digests-${{ env.PLATFORM_PAIR }} - path: /tmp/digests/* - if-no-files-found: error - retention-days: 1 - - - merge: - # This job merges the Docker manifests for the different platforms built in the previous job. - name: Merge Docker manifests - runs-on: ubuntu-latest - permissions: - attestations: write - actions: read - checks: read - contents: read - deployments: none - id-token: write - issues: read - discussions: read - packages: write - pages: none - pull-requests: read - repository-projects: read - security-events: read - statuses: read - - needs: - - build - # This job depends on the build job to complete before it starts. - # It ensures that the Docker images for all platforms are built before merging the manifests. - steps: - - name: Download digests - # This step downloads the digest files uploaded in the build job. - # It uses the actions/download-artifact action to download the artifacts with the pattern digests-*. - # The downloaded files are merged into the /tmp/digests directory. - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 - with: - path: /tmp/digests - pattern: digests-* - merge-multiple: true - - - - name: Docker meta - # This step generates metadata for the Docker image. - # It uses the docker/metadata-action to create metadata based on the repository information. - # The metadata includes information such as the image name, tags, and labels. - id: meta - uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0 - with: - images: ${{ env.GHCR_IMAGE }} - annotations: | - type=org.opencontainers.image.description,value=${{ github.event.repository.description || 'No description provided' }} - tags: | - type=raw,value=gateway - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 - # This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit. - with: - driver-opts: | - network=host - - - name: Login to GitHub Container Registry - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - # This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action. - # It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication. - # The login is necessary to push the merged manifest list to GHCR. - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Get execution timestamp with RFC3339 format - # This step gets the current execution timestamp in RFC3339 format. - # It uses the date command to get the current UTC time and formats it as a string. - # The timestamp is used for annotating the Docker manifest list. - id: timestamp - run: | - echo "timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")" >> $GITHUB_OUTPUT - - - name: Create manifest list and pushs - # This step creates a manifest list for the Docker images built for different platforms. - # It uses the docker buildx imagetools create command to create the manifest list. - # The manifest list is annotated with metadata such as description, creation timestamp, and source URL. - # The annotations are obtained from the metadata generated in the previous steps. - # The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags. - working-directory: /tmp/digests - id: manifest-annotate - continue-on-error: true - run: | - docker buildx imagetools create \ - $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ - --annotation='index:org.opencontainers.image.description=${{ github.event.repository.description }}' \ - --annotation='index:org.opencontainers.image.created=${{ steps.timestamp.outputs.timestamp }}' \ - --annotation='index:org.opencontainers.image.url=${{ github.event.repository.url }}' \ - --annotation='index:org.opencontainers.image.source=${{ github.event.repository.url }}' \ - $(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *) - - - - name: Create manifest list and push without annotations - # This step creates a manifest list for the Docker images built for different platforms. - # It uses the docker buildx imagetools create command to create the manifest list. - # The manifest list is created without annotations if the previous step fails. - # The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags. - if: steps.manifest-annotate.outcome == 'failure' - working-directory: /tmp/digests - run: | - docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ - $(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *) - - - name: Inspect image - # This step inspects the created manifest list to verify its contents. - # It uses the docker buildx imagetools inspect command to display information about the manifest list. - # The inspection output will show the platforms and tags associated with the manifest list. - id: inspect - run: | - docker buildx imagetools inspect '${{ env.GHCR_IMAGE }}:${{ steps.meta.outputs.version }}' diff --git a/.github/workflows/multi-build.yaml b/.github/workflows/multi-build.yaml deleted file mode 100644 index a0638f1..0000000 --- a/.github/workflows/multi-build.yaml +++ /dev/null @@ -1,295 +0,0 @@ -# This workflow builds a multi-arch Docker image using GitHub Actions and separated Github Runners with native support for ARM64 and AMD64 architectures, without using QEMU emulation. -# It uses Docker Buildx to build and push the image to GitHub Container Registry (GHCR). -name: Build multi arch Docker Image with separate Github Runners - -on: - workflow_dispatch: - inputs: - dsh_version: - description: "DSH git ref to build (tag or branch)" - required: true - default: dsh-v0.2.1-rc.2 - type: string - pnpm_version: - description: "pnpm version used to build DSH" - required: true - default: "11.28.5" - type: string - push: - branches: - - main - paths: - - 'Dockerfile' - - '.github/workflows/multi-build.yaml' -env: - # The name of the Docker image to be built and pushed to GHCR - # The image name is derived from the GitHub repository name and the GitHub Container Registry (GHCR) URL. - # The image name will be in the format: ghcr.io// - GHCR_IMAGE: ghcr.io/${{ github.repository }} - # workflow_dispatch inputs win; pushes to main fall back to the pinned - # defaults. Both values flow into the Docker build args and the image tags. - DSH_VERSION: ${{ inputs.dsh_version || 'dsh-v0.2.1-rc.2' }} - PNPM_VERSION: ${{ inputs.pnpm_version || '11.28.5' }} -permissions: - # Global permissions for the workflow, which can be overridden at the job level - contents: read - -concurrency: - # This concurrency group ensures that only one job in the group runs at a time. - # If a new job is triggered, the previous one will be canceled. - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - # The build job builds the Docker image for each platform specified in the matrix. - build: - strategy: - fail-fast: false - matrix: - platform: - - linux/amd64 - - linux/arm64 - # The matrix includes two platforms: linux/amd64 and linux/arm64. - # The build job will run for each platform in the matrix. - - permissions: - # Permissions for the build job, which can be overridden at the step level - # The permissions are set to allow the job to write to the GitHub Container Registry (GHCR) and read from the repository. - attestations: write - actions: read - checks: write - contents: write - deployments: none - id-token: write - issues: read - discussions: read - packages: write - pages: none - pull-requests: read - repository-projects: read - security-events: read - statuses: read - - runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' }} - # The job runs on different runners based on the platform. - # For linux/amd64, it runs on the latest Ubuntu runner. - # For linux/arm64, it runs on an Ubuntu 24.04 ARM runner. - # The runner is selected based on the platform specified in the matrix. - - name: Build Docker image for ${{ matrix.platform }} - - steps: - - - name: Prepare environment for current platform - # This step sets up the environment for the current platform being built. - # It replaces the '/' character in the platform name with '-' and sets it as an environment variable. - # This is useful for naming artifacts and other resources that cannot contain '/'. - # The environment variable PLATFORMS_PAIR will be used later in the workflow. - id: prepare - run: | - platform=${{ matrix.platform }} - echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV - - - name: Checkout - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # This step checks out the code from the repository. - # It uses the actions/checkout action to clone the repository into the runner's workspace. - - - name: Docker meta default - # This step generates metadata for the Docker image. - # It uses the docker/metadata-action to create metadata based on the repository information. - # The metadata includes information such as the image name, tags, and labels. - # The metadata will be used later in the workflow to build and push the Docker image. - id: meta - uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0 - with: - images: ${{ env.GHCR_IMAGE }} - - - - name: Set up Docker Context for Buildx - # This step sets up a Docker context for Buildx. - # It creates a new context named "builders" that will be used for building the Docker image. - # The context allows Buildx to use the Docker daemon for building images. - id: buildx-context - run: | - docker context create builders - - - name: Set up Docker Buildx - # This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit. - # It uses the docker/setup-buildx-action to configure Buildx with the specified context and platforms. - # The platforms are specified in the matrix and will be used for building the Docker image. - uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 - with: - endpoint: builders - platforms: ${{ matrix.platform }} - - - name: Login to GitHub Container Registry - # This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action. - # It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication. - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - - name: Build and push by digest - # This step builds and pushes the Docker image using Buildx. - # It uses the docker/build-push-action to build the image with the specified context and platforms. - # The image is built with the labels and annotations generated in the previous steps. - # The outputs are configured to push the image by digest, which allows for better caching and versioning. - # The cache-from and cache-to options are used to enable caching for the build process. - # The cache is stored in GitHub Actions cache and is scoped to the repository, branch, and platform. - id: build - uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 - env: - DOCKER_BUILDKIT: 1 - DSH_PORT: 3080 - with: - context: . - platforms: ${{ matrix.platform }} - labels: ${{ steps.meta.outputs.labels }} - build-args: | - DSH_VERSION=${{ env.DSH_VERSION }} - PNPM_VERSION=${{ env.PNPM_VERSION }} - annotations: ${{ steps.meta.outputs.annotations }} - outputs: type=image,name=${{ env.GHCR_IMAGE }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true - cache-from: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }} - cache-to: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }} - - - - name: Export digest - # This step exports the digest of the built image to a file. - # It creates a directory in /tmp/digests and saves the digest of the image to a file. - # The digest is obtained from the output of the build step. - # The digest is used to uniquely identify the built image and can be used for further processing or verification. - run: | - mkdir -p /tmp/digests - digest="${{ steps.build.outputs.digest }}" - touch "/tmp/digests/${digest#sha256:}" - - - name: Upload digest - # This step uploads the digest file to the GitHub Actions artifact storage. - # It uses the actions/upload-artifact action to upload the file created in the previous step. - # The artifact is named digests-${{ env.PLATFORM_PAIR }}, where PLATFORM_PAIR is the platform name with '/' replaced by '-'. - # The artifact is retained for 1 day, and if no files are found, it will throw an error. - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 - with: - name: digests-${{ env.PLATFORM_PAIR }} - path: /tmp/digests/* - if-no-files-found: error - retention-days: 1 - - - merge: - # This job merges the Docker manifests for the different platforms built in the previous job. - name: Merge Docker manifests - runs-on: ubuntu-latest - permissions: - attestations: write - actions: read - checks: read - contents: read - deployments: none - id-token: write - issues: read - discussions: read - packages: write - pages: none - pull-requests: read - repository-projects: read - security-events: read - statuses: read - - needs: - - build - # This job depends on the build job to complete before it starts. - # It ensures that the Docker images for all platforms are built before merging the manifests. - steps: - - name: Download digests - # This step downloads the digest files uploaded in the build job. - # It uses the actions/download-artifact action to download the artifacts with the pattern digests-*. - # The downloaded files are merged into the /tmp/digests directory. - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 - with: - path: /tmp/digests - pattern: digests-* - merge-multiple: true - - - - name: Docker meta - # This step generates metadata for the Docker image. - # It uses the docker/metadata-action to create metadata based on the repository information. - # The metadata includes information such as the image name, tags, and labels. - id: meta - uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0 - with: - images: ${{ env.GHCR_IMAGE }} - annotations: | - type=org.opencontainers.image.description,value=${{ github.event.repository.description || 'No description provided' }} - tags: | - type=raw,value=main,enable=${{ github.ref_name == 'main' }} - type=raw,value=latest,enable=${{ github.ref_name == 'main' }} - type=raw,value=${{ env.DSH_VERSION }},enable=${{ github.ref_name == 'main' || github.event_name == 'workflow_dispatch' }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 - # This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit. - with: - driver-opts: | - network=host - - - name: Login to GitHub Container Registry - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - # This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action. - # It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication. - # The login is necessary to push the merged manifest list to GHCR. - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Get execution timestamp with RFC3339 format - # This step gets the current execution timestamp in RFC3339 format. - # It uses the date command to get the current UTC time and formats it as a string. - # The timestamp is used for annotating the Docker manifest list. - id: timestamp - run: | - echo "timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")" >> $GITHUB_OUTPUT - - - name: Create manifest list and pushs - # This step creates a manifest list for the Docker images built for different platforms. - # It uses the docker buildx imagetools create command to create the manifest list. - # The manifest list is annotated with metadata such as description, creation timestamp, and source URL. - # The annotations are obtained from the metadata generated in the previous steps. - # The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags. - working-directory: /tmp/digests - id: manifest-annotate - continue-on-error: true - run: | - docker buildx imagetools create \ - $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ - --annotation='index:org.opencontainers.image.description=${{ github.event.repository.description }}' \ - --annotation='index:org.opencontainers.image.created=${{ steps.timestamp.outputs.timestamp }}' \ - --annotation='index:org.opencontainers.image.url=${{ github.event.repository.url }}' \ - --annotation='index:org.opencontainers.image.source=${{ github.event.repository.url }}' \ - $(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *) - - - name: Create manifest list and push without annotations - # This step creates a manifest list for the Docker images built for different platforms. - # It uses the docker buildx imagetools create command to create the manifest list. - # The manifest list is created without annotations if the previous step fails. - # The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags. - if: steps.manifest-annotate.outcome == 'failure' - working-directory: /tmp/digests - run: | - docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ - $(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *) - - - name: Inspect image - # This step inspects the created manifest list to verify its contents. - # It uses the docker buildx imagetools inspect command to display information about the manifest list. - # The inspection output will show the platforms and tags associated with the manifest list. - id: inspect - run: | - docker buildx imagetools inspect '${{ env.GHCR_IMAGE }}:${{ steps.meta.outputs.version }}' diff --git a/.github/workflows/pr-title.yaml b/.github/workflows/pr-title.yaml new file mode 100644 index 0000000..a16cb1f --- /dev/null +++ b/.github/workflows/pr-title.yaml @@ -0,0 +1,38 @@ +# Enforce Conventional Commit pull request titles so git-cliff can group +# release notes. The PR is squash-merged, so the title becomes the commit. +name: PR title + +on: + pull_request: + types: [opened, edited, synchronize, reopened] + +permissions: + contents: read + pull-requests: read + +jobs: + lint: + name: pr-title + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Validate title + uses: amannn/action-semantic-pull-request@e32d7e603df1aa1ba07e981f2a23455dee596825 # v5 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + types: | + feat + fix + docs + refactor + perf + test + build + ci + chore + revert + requireScope: false + subjectPattern: ^(?![A-Z]).+$ + subjectPatternError: | + The PR title subject must start in lowercase and must not end with a period. diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml new file mode 100644 index 0000000..5f61bd1 --- /dev/null +++ b/.github/workflows/release.yaml @@ -0,0 +1,268 @@ +# Release pipeline. Pushing a tag that matches the identity in versions.env +# publishes both images (version + latest), creates the GitHub Release from +# git-cliff, and opens a PR that refreshes CHANGELOG.md on main. +# +# The tag must be dsh-vX.Y.Z or dsh-vX.Y.Z-rN and must match versions.env; +# tags are immutable, so a broken release is fixed with a new -rN revision. +name: Release + +on: + push: + tags: + - "dsh-v*" + workflow_dispatch: + inputs: + tag: + description: Existing tag to release (must match versions.env) + required: true + type: string + publish_legacy_gateway_alias: + description: Also publish the legacy :gateway alias (one release only) + required: false + default: true + type: boolean + +permissions: + contents: write + packages: write + id-token: write + attestations: write + +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +jobs: + meta: + name: Validate release identity + runs-on: ubuntu-latest + timeout-minutes: 15 + outputs: + identity: ${{ steps.meta.outputs.identity }} + dsh_version: ${{ steps.meta.outputs.dsh_version }} + pnpm_version: ${{ steps.meta.outputs.pnpm_version }} + release_revision: ${{ steps.meta.outputs.release_revision }} + ghcr_image: ${{ steps.meta.outputs.ghcr_image }} + gateway_image: ${{ steps.meta.outputs.gateway_image }} + tag: ${{ steps.meta.outputs.tag }} + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Validate tag against versions.env + id: meta + shell: bash + run: | + set -euo pipefail + tag="${{ inputs.tag || github.ref_name }}" + while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + echo "$line" >> "$GITHUB_ENV" + done < versions.env + identity="$DSH_VERSION" + [ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION" + if [ "$tag" != "$identity" ]; then + echo "tag '$tag' does not match versions.env identity '$identity'" >&2 + exit 1 + fi + if git rev-parse -q --verify "refs/tags/$tag" >/dev/null; then + git fetch origin main '+refs/heads/release/*:refs/remotes/origin/release/*' + commit=$(git rev-list -n1 "refs/tags/$tag") + allowed=0 + for ref in origin/main origin/release/*; do + if git rev-parse -q --verify "$ref" >/dev/null && git merge-base --is-ancestor "$commit" "$ref"; then + allowed=1 + break + fi + done + if [ "$allowed" -ne 1 ]; then + echo "tag commit $commit is not on main or a release branch" >&2 + exit 1 + fi + else + echo "tag '$tag' does not exist in this repository" >&2 + exit 1 + fi + if ! git ls-remote --tags --exit-code "$DSH_REPO" "$DSH_VERSION" >/dev/null; then + echo "upstream ref '$DSH_VERSION' not found in $DSH_REPO" >&2 + exit 1 + fi + { + echo "identity=$identity" + echo "dsh_version=$DSH_VERSION" + echo "pnpm_version=$PNPM_VERSION" + echo "release_revision=$RELEASE_REVISION" + echo "ghcr_image=$GHCR_IMAGE" + echo "gateway_image=$GATEWAY_IMAGE" + echo "tag=$tag" + } >> "$GITHUB_OUTPUT" + + - name: Verify repository pins + run: ./scripts/check-versions.sh + + tests: + name: Unit tests and lint + needs: meta + runs-on: ubuntu-latest + timeout-minutes: 15 + defaults: + run: + working-directory: examples/dsh-agents-as-a-service/gateway + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.12" + cache: pip + cache-dependency-path: | + examples/dsh-agents-as-a-service/gateway/requirements.txt + examples/dsh-agents-as-a-service/gateway/requirements-dev.txt + - name: Install dependencies + run: | + set -euo pipefail + python -m pip install --upgrade pip + pip install -r requirements.txt -r requirements-dev.txt + - name: Lint + run: | + set -euo pipefail + python -m ruff check . + python -m ruff format --check . + - name: Unit tests + run: python -m pytest -q + + build-main: + name: Publish main image + needs: [meta, tests] + uses: ./.github/workflows/_build-image.yaml + permissions: + contents: read + packages: write + id-token: write + attestations: write + with: + image: ${{ needs.meta.outputs.ghcr_image }} + version: ${{ needs.meta.outputs.identity }} + context: . + dockerfile: Dockerfile + build-args: | + DSH_VERSION=${{ needs.meta.outputs.dsh_version }} + RELEASE_REVISION=${{ needs.meta.outputs.release_revision }} + PNPM_VERSION=${{ needs.meta.outputs.pnpm_version }} + VCS_REF=${{ github.sha }} + tags: | + ${{ needs.meta.outputs.ghcr_image }}:${{ needs.meta.outputs.identity }} + ${{ needs.meta.outputs.ghcr_image }}:latest + + build-gateway: + name: Publish gateway image + needs: [meta, tests] + uses: ./.github/workflows/_build-image.yaml + permissions: + contents: read + packages: write + id-token: write + attestations: write + with: + image: ${{ needs.meta.outputs.gateway_image }} + version: ${{ needs.meta.outputs.identity }} + context: examples/dsh-agents-as-a-service/gateway + dockerfile: Dockerfile + tags: | + ${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }} + ${{ needs.meta.outputs.gateway_image }}:latest + + legacy-gateway-alias: + name: Publish legacy :gateway alias (temporary) + needs: [meta, build-gateway] + if: github.event_name != 'workflow_dispatch' || inputs.publish_legacy_gateway_alias + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - name: Login to GitHub Container Registry + uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Retag gateway as the legacy package tag + env: + SOURCE: ${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }} + LEGACY: ${{ needs.meta.outputs.ghcr_image }}:gateway + run: | + set -euo pipefail + docker buildx imagetools create -t "$LEGACY" "$SOURCE" + echo "Deprecated alias $LEGACY now points at $SOURCE" >> "$GITHUB_STEP_SUMMARY" + + publish-release: + name: Changelog and GitHub Release + needs: [meta, build-main, build-gateway] + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + + - name: Install git-cliff + env: + GIT_CLIFF_VERSION: v2.14.2 + run: | + set -euo pipefail + version="${GIT_CLIFF_VERSION#v}" + curl -fsSL -o /tmp/git-cliff.tar.gz "https://github.com/orhun/git-cliff/releases/download/$GIT_CLIFF_VERSION/git-cliff-$version-x86_64-unknown-linux-gnu.tar.gz" + tar -xzf /tmp/git-cliff.tar.gz -C /tmp + install -m 0755 "/tmp/git-cliff-$version/git-cliff" /usr/local/bin/git-cliff + git-cliff --version + + - name: Generate CHANGELOG and release notes + env: + IDENTITY: ${{ needs.meta.outputs.identity }} + run: | + set -euo pipefail + git-cliff --tag "$IDENTITY" --output CHANGELOG.md + git-cliff --tag "$IDENTITY" --latest --strip header --output /tmp/release-notes.md + echo "### Release $IDENTITY" >> "$GITHUB_STEP_SUMMARY" + head -n 40 /tmp/release-notes.md >> "$GITHUB_STEP_SUMMARY" + + - name: Create GitHub Release + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + IDENTITY: ${{ needs.meta.outputs.identity }} + run: | + set -euo pipefail + if gh release view "$IDENTITY" >/dev/null 2>&1; then + echo "release $IDENTITY already exists; skipping" >&2 + else + gh release create "$IDENTITY" --title "$IDENTITY" --notes-file /tmp/release-notes.md --verify-tag + fi + + - name: Open CHANGELOG pull request + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + IDENTITY: ${{ needs.meta.outputs.identity }} + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + if git diff --quiet -- CHANGELOG.md; then + echo "CHANGELOG.md already up to date" + exit 0 + fi + branch="chore/changelog-$IDENTITY" + git switch -c "$branch" + git add CHANGELOG.md + git commit -m "chore(changelog): refresh for $IDENTITY" + git push origin "$branch" + gh pr create \ + --base main \ + --head "$branch" \ + --title "chore(changelog): refresh for $IDENTITY" \ + --body "Generated by git-cliff for $IDENTITY. Do not edit CHANGELOG.md by hand." diff --git a/.github/workflows/validate.yaml b/.github/workflows/validate.yaml deleted file mode 100644 index f9d2520..0000000 --- a/.github/workflows/validate.yaml +++ /dev/null @@ -1,137 +0,0 @@ -name: Validate - -# Validates the Kubernetes manifests against the real agent-sandbox CRD -# schema (server-side dry-run on a throwaway kind cluster), runs the gateway -# unit tests + lint, and builds/smoke-tests the gateway image without a -# cluster. Runs on every PR and on pushes to main. -on: - pull_request: - push: - branches: [main] - -permissions: - contents: read - -jobs: - validate-manifests: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Create kind cluster - uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1 - with: - version: v0.33.0 - cluster_name: dsh-validate - - - name: Install agent-sandbox (extensions) - run: | - VERSION=v1.0.7 - kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/sandbox-with-extensions.yaml - kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s - kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s - kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s - kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s - - - name: Validate manifests (server dry-run) - run: | - set -euo pipefail - kubectl create namespace dsh-demo - for f in examples/dsh-agents-as-a-service/*.yaml; do - echo "== $f ==" - kubectl apply --dry-run=server -f "$f" - done - kubectl apply --dry-run=server -f deploy/k8s/sandbox-template.yaml - echo "all manifests validated" - - gateway-tests: - # The gateway lives in examples/dsh-agents-as-a-service/gateway/. Its - # runtime pins and dev pins are kept separate so the image only ships - # requirements.txt while CI installs both files. - name: Gateway unit tests and lint - runs-on: ubuntu-latest - defaults: - run: - working-directory: examples/dsh-agents-as-a-service/gateway - steps: - - name: Checkout - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Set up Python - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: "3.12" - cache: pip - cache-dependency-path: | - examples/dsh-agents-as-a-service/gateway/requirements.txt - examples/dsh-agents-as-a-service/gateway/requirements-dev.txt - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt -r requirements-dev.txt - - - name: Lint (ruff) - run: python -m ruff check . - - - name: Run gateway unit tests - run: python -m pytest -v - - gateway-image: - # No cluster and no real credentials: the gateway must start and answer - # /healthz with a dummy kubeconfig, which catches broken imports, missing - # runtime deps, and an unresponsive ASGI entrypoint. - name: Build and smoke-test gateway image - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Build gateway image - run: | - docker build \ - -t dsh-aaas-gateway:ci \ - examples/dsh-agents-as-a-service/gateway/ - - - name: Smoke-test /healthz without a cluster - run: | - set -euo pipefail - cat > /tmp/kubeconfig <<'EOF' - apiVersion: v1 - kind: Config - clusters: - - name: dummy - cluster: - server: https://127.0.0.1:6443 - insecure-skip-tls-verify: true - contexts: - - name: dummy - context: - cluster: dummy - user: dummy - users: - - name: dummy - user: {} - current-context: dummy - EOF - docker run -d --name gateway-smoke \ - -p 18080:8080 \ - -e NAMESPACE=dsh-demo \ - -e POOL=dsh-pool \ - -e KUBECONFIG=/tmp/kubeconfig \ - -v /tmp/kubeconfig:/tmp/kubeconfig:ro \ - dsh-aaas-gateway:ci - trap 'docker logs gateway-smoke || true; docker rm -f gateway-smoke || true' EXIT - - code="" - for _ in $(seq 1 30); do - code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:18080/healthz || true) - [ "$code" = "200" ] && break - sleep 1 - done - if [ "$code" != "200" ]; then - echo "gateway /healthz returned '$code' instead of 200" >&2 - exit 1 - fi - echo "gateway image healthy: GET /healthz -> 200" diff --git a/.gitignore b/.gitignore index 1cec806..a0e466a 100644 --- a/.gitignore +++ b/.gitignore @@ -159,3 +159,8 @@ __pycache__/ htmlcov/ .venv/ venv/ + +# Local repository tooling +.uvcache/ +coverage.xml + diff --git a/.hadolint.yaml b/.hadolint.yaml new file mode 100644 index 0000000..707bd1d --- /dev/null +++ b/.hadolint.yaml @@ -0,0 +1,6 @@ +# Container lint policy. The base images are rebuilt on every DSH release, so +# unpinned apt/pip packages are intentional inside a pinned base image. +failure-threshold: warning +ignored: + - DL3008 # apt-get packages are intentionally not version-pinned + - DL3018 # apk packages are intentionally not version-pinned diff --git a/.lychee.toml b/.lychee.toml new file mode 100644 index 0000000..57a3870 --- /dev/null +++ b/.lychee.toml @@ -0,0 +1,13 @@ +# Link checker for README/docs. Placeholders used by the examples are excluded +# on purpose; real URLs must resolve. +verbose = "info" +no_progress = true +max_retries = 2 +exclude = [ + "localhost", + "127\\.0\\.0\\.1", + "example\\.com", + "your-github-org", + "mailto:", + "ghcr\\.io/.*@sha256", +] diff --git a/.yamllint.yml b/.yamllint.yml new file mode 100644 index 0000000..2b3edf8 --- /dev/null +++ b/.yamllint.yml @@ -0,0 +1,19 @@ +extends: default + +rules: + document-start: disable + line-length: + max: 200 + level: error + truthy: + check-keys: false + comments-indentation: disable + comments: + min-spaces-from-content: 1 + +ignore: | + .venv/ + .uvcache/ + node_modules/ + # Upstream CodeQL boilerplate: long documentation comments, linted by actionlint. + .github/workflows/codeql.yml diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..c1539fd --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,140 @@ +# AGENTS.md + +Guidance for AI coding agents (and humans) working in this repository. Read it +before changing anything; it encodes the conventions the CI enforces. + +## What this repository is + +- **The root is the primary artifact.** The Dockerfile packages DeepSeek + Harness Web (dsh web) as a slim, non-root, multi-arch image. Everything else + exists to support it or to demonstrate an implementation on top of it. +- **Every implementation is a self-contained example** under examples//. + There is exactly one example for the agents-as-a-service platform; do not + split it into parallel directories. +- **versions.env is the single source of truth** for the DSH ref, pnpm, the + control-plane versions and the image names. +- **Docs describe the shipped behavior**, not intentions. If you change a + manifest, port, env var or workflow, update the docs in the same change. +- **CHANGELOG.md is generated** by git-cliff from Conventional Commits. Never + edit it by hand. + +## Golden rules + +1. Keep the Dockerfile working for both linux/amd64 and linux/arm64; it must + stay the first-class artifact. +2. Never hard-code a version, image tag or registry path. Edit versions.env and + run make set-version when bumping. +3. Never commit a real secret or usable credential. Secret-shaped YAML lives in + *.yaml.example files; keys are documented in .env.sample only. +4. Every manifest under examples/*/manifests/ must parse and pass a server-side + dry-run in CI. Keep manifests numbered (NN-*.yaml), and keep exposure + variants under manifests/exposure// documented by the single + manifests/exposure/README.md (variants are mutually exclusive). +5. Do not edit CHANGELOG.md, and do not move or delete published tags. + Releasing is documented in docs/versioning.md. +6. Keep the gateway HTTP/WebSocket contract stable unless the change is + explicitly about it; clients and examples depend on it. + +## Repository map + +| Path | What lives there | +| --- | --- | +| Dockerfile | Main DSH Web image (build + runtime stages, entrypoint) | +| versions.env | Every pinned version and image name | +| Makefile | Canonical local entrypoint | +| build.sh | Deprecated shim that forwards to make | +| docs/ | main-image.md, examples.md, gitflow.md, versioning.md | +| examples/ | One implementation per directory (contract in docs/examples.md) | +| examples/dsh-agents-as-a-service/ | The multi-tenant platform example (manifests, gateway, docs, scripts) | +| examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart | +| scripts/ | check-versions.sh, check-examples.sh, check-manifests.py, set-version.sh | +| .github/workflows/ | ci.yaml, reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml | + +## Commands + + make check # versions, example layout and manifest YAML (run before every commit) + make test # gateway unit tests (109+ tests) + make lint # ruff + shellcheck (+ yamllint/actionlint when installed) + make image # build the main DSH Web image (slow: clones and builds DSH) + make gateway # build the example gateway image (fast) + make e2e # full kind walkthrough; needs a cluster with agent-sandbox + make changelog # regenerate CHANGELOG.md with git-cliff (release only) + make release-dry-run + make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=rN + +No cluster is needed for check, test, lint, gateway or image. Only make e2e +needs kind plus the agent-sandbox CRDs (see the example README). + +## Testing expectations per change + +| Change | Minimum verification | +| --- | --- | +| Documentation only | make check; verify relative links | +| Example manifest | make check; kubectl apply --dry-run=server in CI | +| Gateway code | make test; make lint; add or update tests | +| Dockerfile / versions.env | make check; make image (amd64) and a container smoke test | +| Workflow | actionlint in CI; keep actions SHA-pinned; no hard-coded versions | +| New example | make check; all files described in docs/examples.md | + +## Versioning and releases + +- Release identity = DSH_VERSION, optionally with a repository revision suffix + for re-releases at the same DSH ref (first release: dsh-v0.2.1-rc.2-r2). +- The current identity is dsh-v0.2.1-rc.2-r2 (stored in versions.env). The + dsh-v0.2.1-rc.2 image tag already exists and tags are immutable, which is why + the first release under this scheme carries -r2. +- Channels: release tags are immutable; latest follows the newest release; + main is the development channel for both image packages. +- A broken release is fixed by publishing -r(N+1), never by re-tagging. +- The full procedure, compatibility table and rollback rules are in + docs/versioning.md. + +## Branch and PR conventions + +- Trunk-based with release branches, documented in docs/gitflow.md. +- Branch names: feat/, fix/, docs/, chore/, refactor/, test/. +- Conventional Commit subjects and PR titles (feat(scope): ..., fix(scope): ...). +- PRs are squash-merged; main stays linear and protected. +- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL, + dependency-review. +- Keep diffs scoped; update docs with the code; never mix a version bump with + unrelated changes. + +## Adding an example + +1. Read docs/examples.md and examples/README.md. +2. Create examples// with README.md, manifests/NN-*.yaml, optional + manifests/exposure// (plus README), optional components with their + own Dockerfile/README/tests, and scripts/ (executable, idempotent, + self-cleaning). +3. Publish component images under ////. +4. Add the example to examples/README.md; scripts/check-examples.sh enforces + the layout. + +## Security notes + +- AUTH_PROXY_HEADER is opt-in. Only enable it when the gateway Service is + reachable exclusively through the authenticating proxy; otherwise any client + can spoof X-Forwarded-User. +- The example gateway reads the DSH launch token from pod logs. Treat pod logs + as sensitive and keep its RBAC namespace-scoped. +- The example shares one DEEPSEEK_API_KEY across sandboxes; do not describe it + as tenant-isolated. +- TLS terminates outside the gateway; set COOKIE_SECURE=true for HTTPS and keep + the base manifest safe for local HTTP. + +## Ask before + +- Changing release workflows, tag formats or published image names. +- Editing the gateway's public routes or auth semantics. +- Adding a new top-level tool, service or dependency to the main image. +- Rewriting generated files (CHANGELOG.md) or git history. + +## Where to look first + +- [docs/main-image.md](docs/main-image.md) - container contract and tags +- [docs/examples.md](docs/examples.md) - example authoring rules +- [docs/gitflow.md](docs/gitflow.md) - branches, PRs, required checks +- [docs/versioning.md](docs/versioning.md) - versioning, changelog, releases +- [CONTRIBUTING.md](CONTRIBUTING.md) - development setup +- [SECURITY.md](SECURITY.md) - vulnerability reporting and threat notes diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..bb8e1f5 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,46 @@ +# Code of Conduct + +This project adopts the Contributor Covenant, version 2.1: +https://www.contributor-covenant.org/version/2/1/code_of_conduct/ + +## Our standards + +We are committed to a welcoming, harassment-free experience for everyone, +regardless of age, body size, visible or invisible disability, ethnicity, sex +characteristics, gender identity and expression, level of experience, +education, socio-economic status, nationality, personal appearance, race, +religion, or sexual identity and orientation. + +Examples of behavior that contributes to a positive environment: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes +- Focusing on what is best for the community and the project + +Examples of unacceptable behavior: + +- Sexualized language or imagery, and sexual attention or advances +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information without explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement + +Maintainers are responsible for clarifying and enforcing these standards and +may remove, edit, or reject comments, commits, code, issues, and other +contributions that do not align with this Code of Conduct. + +Report abusive, harassing, or otherwise unacceptable behavior to the +maintainers through a private GitHub Security Advisory or by contacting the +repository owner on GitHub. All complaints will be reviewed and investigated +promptly and fairly. Maintainers must respect the privacy and security of the +reporter. + +## Attribution + +This Code of Conduct is adapted from the Contributor Covenant, version 2.1, +available at https://www.contributor-covenant.org/version/2/1/code_of_conduct/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..a405c7e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,88 @@ +# Contributing + +Thanks for improving deepseek-kubernetes-sandbox. This repository packages +DeepSeek Harness Web as a container and ships self-contained implementation +examples around it. The root is the container; every implementation lives +under examples/. + +## Repository layout + +| Path | Purpose | +| --- | --- | +| Dockerfile | The primary artifact: the DSH Web image | +| versions.env | Single source of truth for every pinned version | +| Makefile | Canonical local entrypoint (build, test, lint, check, e2e) | +| docs/ | Repository-wide docs: main image, example contract, gitflow, versioning | +| examples/ | One self-contained implementation per directory | +| scripts/ | Repository checks (versions, example layout, manifests, version bump) | +| .github/ | CI, releases, dependency updates, templates | + +## Requirements + +- Docker (for image builds) +- Python 3.12+ (gateway tests and repository checks) +- make and git +- kubectl and kind only for the end-to-end example + +## Local workflow + + git switch -c feat/ + make check # versions, example layout, manifest YAML + make test # gateway unit tests + make lint # ruff, shellcheck (+ yamllint/actionlint when installed) + make image # build the main DSH Web image + make gateway # build the example gateway image + make e2e # full kind walkthrough (optional, slow) + +Install the gateway dev dependencies once: + + python -m pip install -r examples/dsh-agents-as-a-service/gateway/requirements-dev.txt + +## Pull requests + +- Use a Conventional Commit title, for example feat(gateway): add wake metrics + or docs(examples): clarify SSO trust boundary. PR titles are linted. +- Keep the diff scoped. Squash merge into main; history stays linear. +- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL, + and dependency-review. Branch protection is described in docs/gitflow.md. +- Update the docs that the change affects; docs are part of the change, not a + follow-up. +- Never commit a real secret. Secret-shaped YAML belongs in a *.yaml.example + file, and versions live in versions.env only. + +## Changing versions + + make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=r2 + make check + +CHANGELOG.md is generated by git-cliff from Conventional Commits. Do not edit +it by hand; the release workflow regenerates it. + +## Adding an example + +Read docs/examples.md, then add examples// with: + +- README.md (what it demonstrates, prerequisites, steps, teardown) +- manifests/NN-*.yaml (numbered, plain kubectl) +- optional manifests/exposure// with its own README.md +- optional components (a Dockerfile plus README and tests per component) +- scripts/ with executable, idempotent, self-cleaning scripts +- docs/ for deeper material + +List it in examples/README.md. scripts/check-examples.sh enforces this layout. + +## Documentation + +- Root README.md is the front door: main image first, then examples. +- docs/ holds repository-wide material; example-specific material belongs in + the example directory. + +## Branches and releases + +See docs/gitflow.md for the branch model and docs/versioning.md for the +release identity, channels and rollback rules. + +## Security + +See SECURITY.md. Report vulnerabilities privately through GitHub Security +Advisories; do not open a public issue. diff --git a/Dockerfile b/Dockerfile index 839e11d..c644fc1 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,12 +1,23 @@ # syntax=docker/dockerfile:1.7 - +# +# The primary artifact of this repository: DeepSeek Harness Web (dsh web), +# packaged as a slim, non-root, multi-arch runtime image. +# +# Build args (all pinned in versions.env, verified by scripts/check-versions.sh): +# DSH_REPO upstream git remote to package +# DSH_VERSION upstream git ref to build (tag or branch) +# RELEASE_REVISION optional repository revision label (r2, r3, ...) +# PNPM_VERSION pnpm used to build DSH +# VCS_REF this repository's commit, recorded in OCI labels +# NODE_BUILD_IMAGE node image used to build DSH +# NODE_RUNTIME_IMAGE node image used at runtime ARG NODE_BUILD_IMAGE=docker.io/node:24-trixie ARG NODE_RUNTIME_IMAGE=docker.io/node:24-trixie-slim # --------------------------------------------------------------------------- -# Build stage +# Build stage: clone DSH at the pinned ref, build, and package the CLI. # --------------------------------------------------------------------------- -FROM ${NODE_BUILD_IMAGE} AS build +FROM $NODE_BUILD_IMAGE AS build ARG DSH_REPO=https://github.com/deepseek-ai/deepseek-harness.git ARG DSH_VERSION=dsh-v0.2.1-rc.2 @@ -16,11 +27,11 @@ ENV CI=true \ NODE_OPTIONS=--max-old-space-size=4096 RUN corepack enable \ - && corepack prepare "pnpm@${PNPM_VERSION}" --activate + && corepack prepare "pnpm@$PNPM_VERSION" --activate WORKDIR /src -RUN git clone --depth 1 --branch "${DSH_VERSION}" "${DSH_REPO}" . +RUN git clone --depth 1 --branch "$DSH_VERSION" "$DSH_REPO" . RUN pnpm install --frozen-lockfile @@ -44,22 +55,30 @@ RUN cd /opt/dsh \ # --------------------------------------------------------------------------- # Runtime stage # --------------------------------------------------------------------------- -FROM ${NODE_RUNTIME_IMAGE} AS runtime +FROM $NODE_RUNTIME_IMAGE AS runtime -ARG DSH_VERSION +ARG DSH_VERSION=dsh-v0.2.1-rc.2 +ARG RELEASE_REVISION=r2 ARG PNPM_VERSION=11.28.5 +ARG VCS_REF=unknown LABEL org.opencontainers.image.title="DeepSeek Harness Web packaged by SREDevOps.org" \ - org.opencontainers.image.version="${DSH_VERSION}" \ - org.opencontainers.image.source="https://github.com/sredevopsorg/deepseek-kubernetes-sandbox" + org.opencontainers.image.description="DeepSeek Harness Web (dsh web) as a slim, non-root, Kubernetes-ready image" \ + org.opencontainers.image.source="https://github.com/sredevopsorg/deepseek-kubernetes-sandbox" \ + org.opencontainers.image.version="$DSH_VERSION" \ + org.opencontainers.image.revision="$VCS_REF" \ + org.opencontainers.image.licenses="MIT" \ + io.sredevopsorg.dsh.version="$DSH_VERSION" \ + io.sredevopsorg.release.revision="$RELEASE_REVISION" ENV NODE_ENV=production \ HOME=/home/node \ DSH_HOME=/home/node/.dsh \ DSH_PORT=3080 \ - PATH="/app/node_modules/.bin:${PATH}" + PATH="/app/node_modules/.bin:$PATH" -# git/ripgrep are useful for DSH agent tools; tini handles signals. +# git/ripgrep/jq/yq/curl/gh are deliberate agent tooling parity: a sandboxed DSH +# instance expects the usual developer CLI. tini reaps signals for a clean stop. RUN apt-get update \ && apt-get install -y --no-install-recommends \ ca-certificates \ @@ -73,9 +92,9 @@ RUN apt-get update \ gh \ && rm -rf /var/lib/apt/lists/* \ && curl -LsSf https://astral.sh/uv/install.sh | sh \ - && install -d -o node -g node -m 0755 /workspace "${DSH_HOME}" \ + && install -d -o node -g node -m 0755 /workspace "$DSH_HOME" \ && corepack enable \ - && corepack prepare "pnpm@${PNPM_VERSION}" --activate \ + && corepack prepare "pnpm@$PNPM_VERSION" --activate \ && pnpm --version COPY --from=build --chown=node:node /opt/dsh /app @@ -98,7 +117,7 @@ if [ -n "$DSH_PUBLIC_URL" ]; then set -- --public-url "$DSH_PUBLIC_URL" "$@" fi -# Additional trusted hosts (e.g. the public gateway host behind an Ingress). +# Additional trusted hosts (e.g. the public gateway host behind a proxy). # DSH Host/Origin-fences /api: without this, requests forwarded with the # browser's Host header are rejected. Accepts comma- or space-separated names; # localhost/127.0.0.1 are always trusted below. @@ -127,4 +146,4 @@ USER node EXPOSE 3080 VOLUME ["/home/node/.dsh"] STOPSIGNAL SIGTERM -ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/dsh-entrypoint"] \ No newline at end of file +ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/dsh-entrypoint"] diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..1c42cca --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Nicolás Georger + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/Makefile b/Makefile index da1d75d..50b3c11 100644 --- a/Makefile +++ b/Makefile @@ -1,37 +1,82 @@ -# Local build and test helpers. Every variable can be overridden, e.g. -# make image DSH_VERSION=dsh-v0.2.1-rc.2 IMAGE=dsh-web:dev -# make test -DSH_VERSION ?= dsh-v0.2.1-rc.2 -PNPM_VERSION ?= 11.28.5 -IMAGE ?= dsh-web:0.2.1-rc.2 -GATEWAY_IMAGE ?= ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway -GATEWAY_DIR := examples/dsh-agents-as-a-service/gateway -PYTHON ?= python3 -DSH_PORT ?= 3080 +# Canonical local entrypoint. Includes versions.env so there is exactly one +# place to bump a version (run "make check" to verify pins across the repo). +# +# make image build the main DSH Web image +# make gateway build the example gateway image +# make test gateway unit tests +# make lint ruff + shellcheck (+ yamllint/actionlint when installed) +# make check versions, example layout and manifest YAML checks +# make e2e full example walkthrough on a kind cluster +# make changelog regenerate CHANGELOG.md with git-cliff +include versions.env -.PHONY: help image gateway run test lint +RELEASE_IDENTITY := $(DSH_VERSION)$(if $(RELEASE_REVISION),-$(RELEASE_REVISION)) +IMAGE ?= $(GHCR_IMAGE):$(RELEASE_IDENTITY) +GATEWAY_LOCAL_IMAGE ?= dsh-aaas-gateway:dev +GATEWAY_DIR := examples/dsh-agents-as-a-service/gateway +EXAMPLE_DIR := examples/dsh-agents-as-a-service +PYTHON ?= $(if $(wildcard $(CURDIR)/.venv/bin/python),$(CURDIR)/.venv/bin/python,python3) +DSH_PORT ?= 3080 +VCS_REF ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown) + +.PHONY: help image run gateway test cov lint check e2e changelog release-dry-run set-version clean help: ## Show available targets - @grep -E '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) | sed 's/:.*## /\t/' + @grep -E '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) | sed 's/:.*## / /' -image: ## Build the DSH Web image +image: ## Build the main DSH Web image (no push) docker build \ --build-arg DSH_VERSION=$(DSH_VERSION) \ + --build-arg RELEASE_REVISION=$(RELEASE_REVISION) \ --build-arg PNPM_VERSION=$(PNPM_VERSION) \ + --build-arg VCS_REF=$(VCS_REF) \ -t $(IMAGE) . -gateway: ## Build the gateway image - docker build -t $(GATEWAY_IMAGE) $(GATEWAY_DIR)/ - -run: image ## Build and run the DSH Web image +run: image ## Build and run the main image (needs DEEPSEEK_API_KEY) docker run --rm -it \ -p $(DSH_PORT):3080 \ -v dsh-home:/home/node/.dsh \ -e DEEPSEEK_API_KEY \ $(IMAGE) -test: ## Run the gateway unit tests (needs requirements + requirements-dev installed) +gateway: ## Build the example gateway image locally + docker build -t $(GATEWAY_LOCAL_IMAGE) $(GATEWAY_DIR)/ + +test: ## Gateway unit tests cd $(GATEWAY_DIR) && $(PYTHON) -m pytest -q -lint: ## Lint the gateway with ruff - cd $(GATEWAY_DIR) && ruff check . +cov: ## Gateway unit tests with coverage (needs pytest-cov) + cd $(GATEWAY_DIR) && $(PYTHON) -m pytest -q --cov=. --cov-report=term-missing --cov-report=xml:coverage.xml + +lint: ## Lint Python, shell, YAML and Dockerfiles (skips missing tools locally) + cd $(GATEWAY_DIR) && $(PYTHON) -m ruff check . + cd $(GATEWAY_DIR) && $(PYTHON) -m ruff format --check . + shellcheck build.sh scripts/*.sh $(EXAMPLE_DIR)/scripts/*.sh + @command -v yamllint >/dev/null 2>&1 && yamllint . || echo "yamllint not installed; CI enforces it" + @command -v actionlint >/dev/null 2>&1 && actionlint || echo "actionlint not installed; CI enforces it" + +check: ## Verify version pins, example layout and manifest YAML + ./scripts/check-versions.sh + ./scripts/check-examples.sh + $(PYTHON) scripts/check-manifests.py + +e2e: ## Run the full example walkthrough on the current kind cluster + $(EXAMPLE_DIR)/scripts/e2e-kind.sh + +changelog: ## Regenerate CHANGELOG.md from Conventional Commits + @command -v git-cliff >/dev/null 2>&1 || { echo "install git-cliff: https://git-cliff.org" >&2; exit 1; } + git-cliff --output CHANGELOG.md + +release-dry-run: check ## Validate the release identity and preview release notes + @echo "release identity: $(RELEASE_IDENTITY)" + @echo "main image: $(GHCR_IMAGE):$(RELEASE_IDENTITY)" + @echo "gateway image: $(GATEWAY_IMAGE):$(RELEASE_IDENTITY)" + @if git rev-parse -q --verify "refs/tags/$(RELEASE_IDENTITY)" >/dev/null; then echo "ERROR: tag already exists" >&2; exit 1; fi + @command -v git-cliff >/dev/null 2>&1 && git-cliff --unreleased --tag $(RELEASE_IDENTITY) || echo "git-cliff not installed; skipping notes preview" + +set-version: ## Bump pins: make set-version DSH_VERSION=dsh-vX [RELEASE_REVISION=rN] + @test -n "$(DSH_VERSION)" || { echo "DSH_VERSION is required" >&2; exit 1; } + ./scripts/set-version.sh "$(DSH_VERSION)" "$(RELEASE_REVISION)" + +clean: ## Remove local build/test caches + rm -rf .pytest_cache .ruff_cache $(GATEWAY_DIR)/.pytest_cache $(GATEWAY_DIR)/.ruff_cache $(GATEWAY_DIR)/__pycache__ diff --git a/README.md b/README.md index d4c9bd7..69a6a74 100644 --- a/README.md +++ b/README.md @@ -1,180 +1,214 @@ # deepseek-kubernetes-sandbox -Packages [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web (`dsh web`) into -a multi-arch, Kubernetes-ready container image, and ships the -[agents-as-a-service](examples/dsh-agents-as-a-service/) platform pattern — running a personal -DSH Web instance per user on [kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) -behind a small browser-usable gateway. +**DeepSeek Harness Web, packaged as a container.** This repository's primary +artifact is a slim, non-root, multi-arch image of +[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web +(`dsh web`), ready for Docker and Kubernetes. Everything else exists to support +that image or to demonstrate an implementation on top of it. -## What's in this repo +The flagship implementation is +[`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/): the +"agents-as-a-service" platform pattern with one DSH Web instance per user, +provisioned from a warm pool on +[kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox), +suspended when idle, and fronted by a small browser-usable gateway. -| Path | Purpose | -| --- | --- | -| [`Dockerfile`](Dockerfile) | Multi-stage build that clones DSH, `pnpm pack`s the CLI, and produces a slim runtime image (`:3080`) | -| [`build.sh`](build.sh), [`Makefile`](Makefile) | Local image builds: DSH Web and gateway, plus test/lint targets | -| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | The full multi-tenant platform pattern (warm pool, per-user claims, gateway, smoke test) | -| [`examples/dsh-agents-as-a-service/gateway/`](examples/dsh-agents-as-a-service/gateway/) | Starlette ASGI gateway: signup, cookie sessions, `/u/{user}/**` HTTP + WebSocket proxy, idle sweeper | -| [`deploy/k8s/sandbox-template.yaml`](deploy/k8s/sandbox-template.yaml) | Minimal single-tenant `SandboxTemplate` | -| [`docs/architecture.md`](docs/architecture.md) | Container and sequence diagrams, design decisions, security notes | -| [`.github/workflows/`](.github/workflows/) | CI: manifests dry-run, gateway tests/lint, multi-arch image builds | - -## Published images - -Everything is published to GitHub Container Registry as -`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox`: - -| Tag | Contents | Port | -| --- | --- | --- | -| `dsh-v0.2.1-rc.2` (also `main`, `latest`) | DSH Web image | `3080` | -| `gateway` | Starlette gateway image | `8080` | - -The DSH Web tag matches the upstream DSH git tag it was built from. - -## Quick start (DSH Web image) +## Quick start ```bash docker run -d --name dsh-web \ -p 3080:3080 \ -v dsh-home:/home/node/.dsh \ -e DEEPSEEK_API_KEY \ - ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 + ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 ``` -**DSH Web is not usable at a bare `/`.** Without its launch-token cookie it answers `401`, and -the token is only printed in the startup logs: +DSH Web is **not usable at a bare `/`**. Without its launch-token cookie it +answers `401`, and the token is printed only in the startup logs: ```bash docker logs dsh-web 2>&1 | grep -m1 '?token=' # dsh web: http://localhost:3080/?token=... ``` -Open the printed `?token=...` URL once — DSH sets its auth cookie and the UI loads normally -afterwards. Hitting `/` directly (or after the cookie expires) returns `401` by design; that is -why Kubernetes probes use `/favicon.svg` (see below) and why the gateway performs this -handshake for the browser. +Open the printed `?token=...` URL once: DSH sets its auth cookie and the UI loads +normally afterwards. Hitting `/` directly (or after the cookie expires) returns +`401` by design. -> **Different host port?** If `3080` is already taken on your workstation, map another port and -> set the public URL so the printed token URL matches: `-p 18081:3080 -e -> DSH_PUBLIC_URL=http://localhost:18081`, then open `http://localhost:18081/?token=...`. +### Host port and public URL -## Quick start (Kubernetes) +If `3080` is taken locally, map another port and set the public URL so the quoted +token URL matches: ```bash -kubectl create namespace dsh-demo --dry-run=client -o yaml | kubectl apply -f - -kubectl -n dsh-demo create secret generic dsh-provider-keys \ - --from-literal=DEEPSEEK_API_KEY="sk-..." -kubectl apply -f deploy/k8s/sandbox-template.yaml - -# A template needs a warm pool and a claim to become a running Sandbox. -kubectl apply -f - <<'EOF' -apiVersion: extensions.agents.x-k8s.io/v1beta1 -kind: SandboxWarmPool -metadata: - name: dsh-pool - namespace: dsh-demo -spec: - replicas: 1 - sandboxTemplateRef: - name: dsh-web-template ---- -apiVersion: extensions.agents.x-k8s.io/v1beta1 -kind: SandboxClaim -metadata: - name: dsh - namespace: dsh-demo -spec: - warmPoolRef: - name: dsh-pool -EOF -kubectl -n dsh-demo wait sandboxclaim/dsh --for=condition=Ready --timeout=180s +docker run -d --name dsh-web -p 18081:3080 \ + -e DSH_PUBLIC_URL=http://localhost:18081 \ + -v dsh-home:/home/node/.dsh \ + -e DEEPSEEK_API_KEY \ + ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 +# then open http://localhost:18081/?token=... ``` -Then reach the instance through its launch-token URL: +### Hosts, probes and the 401 + +- **Probes must not hit `/`.** It returns `401` without the launch token, so a + readiness/liveness probe on `/` crash-loops the pod. Probe `/favicon.svg` + instead: it is unauthenticated. The shipped manifests use it. +- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and + `127.0.0.1` are always trusted; add any other public hostname to + `DSH_TRUSTED_HOSTS` (comma or space separated). A gateway or Ingress that + forwards the browser's `Host` needs its hostname listed there. +- **State lives in `/home/node/.dsh`.** Mount a volume (Docker) or PVC + (Kubernetes) to keep conversations, sessions and config across restarts. + +### Runtime environment + +| Variable | Default | Purpose | +| --- | --- | --- | +| `DEEPSEEK_API_KEY` | empty | Provider credential consumed by DSH. Inject it; never bake it into the image or commit it. | +| `DSH_PORT` | `3080` | Port DSH Web listens on inside the container. | +| `DSH_HOST` | auto-detected pod/container IP | Bind address. Leave empty in Kubernetes. | +| `DSH_PUBLIC_URL` | `http://localhost:${DSH_PORT}` | Public base URL used in generated links and the launch-token URL in the logs. | +| `DSH_TRUSTED_HOSTS` | empty | Extra trusted `Host`/`Origin` entries, comma/space separated. | +| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (mount a volume here). | + +Full contract, build args, probes, volumes and signals: +[docs/main-image.md](docs/main-image.md). + +## Published images + +Published to GitHub Container Registry. The main image is the repository image; +example components are nested packages below it: + +``` +ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox # DSH Web (main image) +ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway # example gateway +``` + +| Tag | Contents | Meaning | +| --- | --- | --- | +| `dsh-v0.2.1-rc.2-r2` | DSH Web, port `3080` | **Current release identity.** Immutable: once published it never moves. | +| `latest` | DSH Web | Most recent release; same digest as the current identity. | +| `main` | DSH Web | Development channel rebuilt from every push to `main`. Not a release; not for production. | +| `dsh-v0.2.1-rc.2` | DSH Web | **Legacy tag** published before the release-identity scheme. It is not updated. New deployments use `dsh-v0.2.1-rc.2-r2`. | + +The example gateway is published under its nested path with the same release +identity, e.g. +`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2`. + +Release identities, channels and rollback rules are explained in +[docs/versioning.md](docs/versioning.md). Production deployments should pin a +digest: ```bash -SB=$(kubectl -n dsh-demo get sandboxclaim dsh -o jsonpath='{.status.sandbox.name}') -kubectl -n dsh-demo logs "$SB" | grep -m1 '?token=' -kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080 -# open http://localhost:18081/?token=... from the log line +docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 +# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256: ``` -Two things bite first-time setups: - -- **Probes must not hit `/`.** DSH returns `401` there without the launch token, so a - readiness/liveness probe on `/` crash-loops the pod. The shipped templates use - `/favicon.svg`, which is unauthenticated; keep that path in custom templates. -- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and `127.0.0.1` are - always trusted; for any other hostname add it to the template's `DSH_TRUSTED_HOSTS` (comma or - space separated). The gateway forwards the browser's `Host`, so the public gateway hostname - must be listed there. - -## The multi-user platform pattern - -`examples/dsh-agents-as-a-service/` is a port of the upstream -[hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service) -example to DSH Web. It demonstrates the platform shape — one `SandboxTemplate`, one warm pool, -one tiny `SandboxClaim` per user, and `operatingMode` as the cost dial — plus a gateway that -turns it into a usable service. - -**Browser flow:** open the gateway landing page, sign up, and get an `HttpOnly` session cookie. -The browser then browses `http:///u//`, which proxies HTTP **and WebSocket** to -that user's DSH Web. The gateway reads the DSH launch token from the pod logs, performs the -token handshake server-side, and injects the resulting DSH cookie on every proxied request — the -browser never sees the token. Suspended instances are woken transparently on the next request; -idle ones are suspended again after `IDLE_TIMEOUT`. - -See the [example README](examples/dsh-agents-as-a-service/README.md) for the full walkthrough -and [docs/architecture.md](docs/architecture.md) for the diagrams and trade-offs. - -### Mechanism mapping: Hermes → DSH - -| Hermes example mechanism | Role | Adaptation for DSH | -| :--- | :--- | :--- | -| `SandboxTemplate` | Reusable Pod blueprint | Image `dsh-web`, port `3080`, probes on `/favicon.svg` | -| `volumeClaimTemplates` | Auto-create a PVC per user | Mount `/home/node/.dsh` (DSH state) | -| `SandboxWarmPool` | Pre-warmed instances | Keep as-is; size for expected concurrency | -| `SandboxClaim` + `warmPoolRef` | "Registration" adopts a pooled instance | Keep exactly as-is; one claim per user | -| `service: true` | Stable DNS per Sandbox | Enabled, so the gateway routes to each user's DSH Web | -| `envVarsInjectionPolicy: Disallowed` | Reject claims that would cold-start | `DEEPSEEK_API_KEY` injected via `envFrom` on the template (shared platform key) | -| `operatingMode` | Cost switch: Running / Suspended | Reused directly — suspend idle instances to save cost | - -### Key differences from Hermes - -- **Multi-arch** — the DSH image is built for amd64 and arm64 (no amd64 nodeSelector). -- **Single port** — DSH Web serves everything on `3080`; no API/dashboard port split. -- **One secret** — only `DEEPSEEK_API_KEY`; no dashboard/session/api-server credentials. -- **Host forwarding** — the gateway forwards `Host` + `X-Forwarded-*` (and keeps - `DSH_PUBLIC_URL` empty) so DSH generates reachable links; add the public hostname to - `DSH_TRUSTED_HOSTS`. -- **Browser sessions + WebSocket** — the gateway is Starlette ASGI, so it proxies WebSocket - upgrades as well as HTTP/SSE, and authenticates browsers with an `HttpOnly` cookie instead of - exposing per-user bearer tokens in URLs. - -## Build it yourself +## Build from source ```bash -./build.sh # build the DSH Web image, then run it interactively -./build.sh web # build the DSH Web image only -./build.sh gateway # build the gateway image - -make image # same as ./build.sh web -make gateway # gateway image -make test # gateway unit tests (pytest) -make lint # gateway lint (ruff) +make image # build the main DSH Web image (clones and builds upstream DSH) +make run # build it, then run it interactively (needs DEEPSEEK_API_KEY) ``` -Or directly: +The pins (`DSH_VERSION`, `RELEASE_REVISION`, `PNPM_VERSION`) come from +[versions.env](versions.env) — never hard-code them +(`make check` verifies). [`build.sh`](build.sh) is a deprecated shim that +forwards to `make`; use the Makefile targets. -```bash -docker build \ - --build-arg DSH_VERSION=dsh-v0.2.1-rc.2 \ - --build-arg PNPM_VERSION=11.28.5 \ - -t dsh-web:0.2.1-rc.2 . +## Examples -docker build -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway \ - examples/dsh-agents-as-a-service/gateway/ -``` +Self-contained implementations that consume the image. Index and contract: +[`examples/README.md`](examples/README.md) and +[docs/examples.md](docs/examples.md). -See [`Dockerfile`](Dockerfile) for build args (`DSH_REPO`, `DSH_VERSION`, `PNPM_VERSION`) and -the runtime image contract (non-root `node` user, `DSH_HOME=/home/node/.dsh`, `EXPOSE 3080`). -`.env.sample` documents the runtime variables for both images. +| Example | What it demonstrates | +| --- | --- | +| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Full multi-tenant platform: SandboxTemplate with per-user PVCs, warm pool, one claim per user, suspend-when-idle, wake-on-return, browser gateway with signup + `HttpOnly` sessions + HTTP/WebSocket proxy, Ingress or Gateway API + GitHub SSO exposure. Start with the [walkthrough](examples/dsh-agents-as-a-service/README.md) and the [architecture](examples/dsh-agents-as-a-service/docs/architecture.md). | +| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | The smallest useful setup: one `SandboxTemplate` and one `SandboxClaim`, reached through `kubectl port-forward` + the launch-token URL. Manifest: [`manifests/sandbox-template.yaml`](examples/minimal-single-tenant/manifests/sandbox-template.yaml). | + +The gateway is a component of the multi-tenant example and has its own +[README](examples/dsh-agents-as-a-service/gateway/README.md) (API, configuration, +auth/SSO, limitations). + +## Repository layout + +| Path | Purpose | +| --- | --- | +| [`Dockerfile`](Dockerfile) | Primary artifact: the multi-stage DSH Web image (port `3080`). | +| [`versions.env`](versions.env) | Single source of truth for the DSH ref, repository revision, pnpm/control-plane versions and image names. | +| [`Makefile`](Makefile), [`build.sh`](build.sh) | Canonical local entrypoint; `build.sh` is a deprecated shim. | +| [`docs/`](docs/README.md) | Repository docs: image contract, example contract, gitflow, versioning. | +| [`examples/`](examples/README.md) | One self-contained implementation per directory. | +| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Multi-tenant platform example: `manifests/`, `gateway/`, `docs/`, `scripts/`. | +| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | Single-sandbox quickstart. | +| [`scripts/`](scripts/) | Repository checks: versions, example layout, manifest YAML, version bump. | +| [`.github/`](.github/) | CI, release automation, dependency updates, PR/issue templates. | +| [`.env.sample`](.env.sample) | Documented local configuration (copy to `.env`; never commit it). | + +## Development + +The Makefile is the canonical entrypoint. No cluster is needed for `check`, +`test`, `lint`, `gateway` or `image`: + +| Target | What it does | +| --- | --- | +| `make check` | Version pins, example layout and manifest YAML checks. Run before every commit. | +| `make test` | Gateway unit tests (`pytest`). | +| `make lint` | `ruff` + `shellcheck` (+ `yamllint`/`actionlint` when installed). | +| `make image` | Build the main DSH Web image. | +| `make gateway` | Build the example gateway image locally (`dsh-aaas-gateway:dev`). | +| `make e2e` | Full kind walkthrough of the multi-tenant example (needs a cluster with agent-sandbox). | +| `make changelog` | Regenerate `CHANGELOG.md` with git-cliff (release only). | +| `make set-version` | Bump pins in `versions.env`; e.g. `make set-version DSH_VERSION= RELEASE_REVISION=rN`. | + +Local setup and the PR workflow: [CONTRIBUTING.md](CONTRIBUTING.md); branches, +required checks and releases: [docs/gitflow.md](docs/gitflow.md). + +## Versioning and releases + +The release identity is the upstream DSH ref plus an optional repository +revision: `DSH_VERSION` and `RELEASE_REVISION` in [versions.env](versions.env). +The current identity is **`dsh-v0.2.1-rc.2-r2`** — the first release under the +mirrored-DSH scheme at the same upstream DSH ref as the legacy +`dsh-v0.2.1-rc.2` tag. Tags are immutable; a broken release is fixed by +publishing `-r(N+1)`, never by re-tagging. + +`CHANGELOG.md` is generated by [git-cliff](https://git-cliff.org) from +Conventional Commits and is never edited by hand. See +[`CHANGELOG.md`](CHANGELOG.md), [docs/versioning.md](docs/versioning.md) and +[cliff.toml](cliff.toml). + +## Migrating from the previous layout + +If you are coming from an older checkout, these paths moved. Nothing else changed +semantically; the content was split into the example it belongs to. + +| Old path | New path | +| --- | --- | +| `gateway-api/dsh-demo/` (Gateway API + oauth2-proxy / GitHub SSO) | `examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/` (now `70-gateway.yaml` ... `74-tls-secret.yaml.example`) | +| `examples/dsh-agents-as-a-service/60-ingress.yaml` | `examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml` | +| `examples/dsh-agents-as-a-service/NN-*.yaml` (`00-` ... `50-`) | `examples/dsh-agents-as-a-service/manifests/NN-*.yaml` | +| `examples/dsh-agents-as-a-service/run-test-kind.sh` | `examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` | +| `deploy/k8s/sandbox-template.yaml` | `examples/minimal-single-tenant/manifests/sandbox-template.yaml` | +| `docs/architecture.md` | `examples/dsh-agents-as-a-service/docs/architecture.md` | + +The SSO walkthrough that lived in `gateway-api/dsh-demo/README.md` is merged +into the [multi-tenant example README](examples/dsh-agents-as-a-service/README.md) +(step 9) and the [exposure README](examples/dsh-agents-as-a-service/manifests/exposure/README.md). + +## Contributing + +- [CONTRIBUTING.md](CONTRIBUTING.md) — local setup, PR workflow, what a change needs. +- [AGENTS.md](AGENTS.md) — repository conventions for AI coding agents (and humans) and the CI commands. +- [SECURITY.md](SECURITY.md) — private vulnerability reporting and project threat notes. +- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — community standards. +- [docs/gitflow.md](docs/gitflow.md) — branches, protected `main`, required checks, release process. + +Do not commit secrets; secret-shaped YAML lives in `*.yaml.example` files, and +version pins change only through `versions.env` / `make set-version`. + +## License + +[MIT](LICENSE) © Nicolás Georger. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..f12987e --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,41 @@ +# Security policy + +## Supported versions + +Only the latest published release is supported. Image tags are immutable; +fixes ship as a new release (see docs/versioning.md). + +## Reporting a vulnerability + +Report vulnerabilities privately through GitHub Security Advisories: + +https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/security/advisories/new + +Do not open a public issue, and do not include live credentials in a report. +You can expect an acknowledgement within a few days and a coordinated +disclosure once a fix is available. + +## Threat notes specific to this project + +- **Launch token in pod logs.** The example gateway reads DSH's launch token + from pod logs (pods/log). Anyone with access to those logs can impersonate a + tenant. Keep the gateway's RBAC namespace-scoped and treat pod logs as + sensitive. Replacing log scraping with an upstream token API is tracked in + the example architecture document. +- **SSO header trust boundary.** AUTH_PROXY_HEADER makes the gateway trust an + identity header (for example X-Forwarded-User). Enable it only when the + gateway Service is reachable exclusively through the authenticating proxy; + otherwise any client can spoof the header. +- **Shared provider key.** The example injects one DEEPSEEK_API_KEY into every + tenant sandbox (a warm-pool constraint). Treat tenants as mutually trusting + until per-tenant credentials or an egress proxy exist. +- **TLS is external.** Terminate TLS at an Ingress or Gateway and set + COOKIE_SECURE=true. Plain HTTP outside localhost exposes session cookies. +- **No secrets in the repository.** The repository only ships empty Secret + manifests and *.yaml.example files. + +## Dependency and supply chain + +GitHub Actions are pinned by commit SHA. The CodeQL workflow scans Python and +Actions. The release workflow publishes images with SBOM and provenance +attestations. diff --git a/build.sh b/build.sh index 30572ff..7c4bcb6 100755 --- a/build.sh +++ b/build.sh @@ -1,68 +1,19 @@ #!/usr/bin/env bash -# Build the DSH Web and/or gateway images locally. +# Deprecated compatibility shim: "make" is the canonical entrypoint. # -# Usage: -# ./build.sh # build the DSH Web image, then run it interactively -# ./build.sh web # build the DSH Web image only -# ./build.sh gateway # build the gateway image (examples/.../gateway/) +# ./build.sh -> make run (build the main image, then run it) +# ./build.sh web -> make image +# ./build.sh gateway -> make gateway # -# Overridable environment: -# DSH_VERSION DSH git ref to build (default dsh-v0.2.1-rc.2) -# PNPM_VERSION pnpm version (default 11.28.5) -# IMAGE DSH Web image tag (default dsh-web:) -# DSH_PORT host port for docker run (default 3080) -# GATEWAY_IMAGE gateway image tag (default ghcr.io/...:gateway) -# RUN 0 to build without running (default 1) +# Use the Makefile targets directly for everything else (test, lint, check, e2e). set -euo pipefail cd "$(dirname "$0")" -DSH_VERSION="${DSH_VERSION:-dsh-v0.2.1-rc.2}" -PNPM_VERSION="${PNPM_VERSION:-11.28.5}" -IMAGE="${IMAGE:-dsh-web:${DSH_VERSION#dsh-v}}" -GATEWAY_IMAGE="${GATEWAY_IMAGE:-ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway}" -DSH_PORT="${DSH_PORT:-3080}" -RUN="${RUN:-1}" -GATEWAY_DIR=examples/dsh-agents-as-a-service/gateway - -build_web() { - docker build \ - --build-arg "DSH_VERSION=${DSH_VERSION}" \ - --build-arg "PNPM_VERSION=${PNPM_VERSION}" \ - -t "${IMAGE}" \ - "$@" \ - . -} - -build_gateway() { - docker build -t "${GATEWAY_IMAGE}" "$@" "${GATEWAY_DIR}/" -} - -usage() { - echo "usage: $0 [run|web|gateway] [extra docker build args...]" >&2 - exit 2 -} +echo "build.sh is deprecated; use 'make image', 'make run' or 'make gateway'." >&2 case "${1:-run}" in - run) - shift - build_web "$@" - if [ "${RUN}" = "1" ]; then - docker run --rm -it \ - -p "${DSH_PORT}:3080" \ - -v dsh-home:/home/node/.dsh \ - -e DEEPSEEK_API_KEY \ - "${IMAGE}" - fi - ;; - web) - shift - build_web "$@" - ;; - gateway) - shift - build_gateway "$@" - ;; - *) - usage - ;; + run) exec make run ;; + web) exec make image ;; + gateway) exec make gateway ;; + *) echo "usage: $0 [run|web|gateway]" >&2; exit 2 ;; esac diff --git a/cliff.toml b/cliff.toml new file mode 100644 index 0000000..f5a8a1e --- /dev/null +++ b/cliff.toml @@ -0,0 +1,46 @@ +# git-cliff configuration: release notes and CHANGELOG.md are generated from +# Conventional Commits. See docs/versioning.md for the release procedure. + +[changelog] +header = """ +# Changelog + +All notable changes to this repository are documented here. +Generated by git-cliff from Conventional Commits; do not edit this file by hand. +""" +body = """ +{% if version %}\ +## [{{ version }}] - {{ timestamp | date(format="%Y-%m-%d") }} +{% else %}\ +## [Unreleased] +{% endif %}\ +{% for group, commits in commits | group_by(attribute="group") %} +### {{ group | upper_first }} +{% for commit in commits %} +- {% if commit.scope %}**{{ commit.scope }}:** {% endif %}{{ commit.message | upper_first }} ({{ commit.id | truncate(length=7, end="") }}) +{%- endfor %} +{% endfor %}\n +""" +footer = "" +trim = true + +[git] +conventional_commits = true +filter_unconventional = false +split_commits = false +commit_parsers = [ + { message = "^feat", group = "Added" }, + { message = "^fix", group = "Fixed" }, + { message = "^perf", group = "Performance" }, + { message = "^refactor", group = "Changed" }, + { message = "^docs", group = "Documentation" }, + { message = "^build", group = "Build" }, + { message = "^ci", group = "CI" }, + { message = "^test", group = "Tests" }, + { message = "^chore", group = "Maintenance" }, + { message = "^revert", group = "Reverted" }, +] +protect_breaking_commits = false +filter_commits = false +tag_pattern = "dsh-v.*" +sort_commits = "oldest" diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..11e08d1 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,38 @@ +# Repository documentation + +Repository-wide documentation lives here. Example-specific material lives with +its example under [`examples//docs/`](../examples/) — see the +[agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md) +for the deep-dive that used to live in this directory. + +If you are new, read the [root README](../README.md) first: it is image-first and +gets a DSH Web container running before anything else. + +## Index + +| Document | Read it for | +| --- | --- | +| [main-image.md](main-image.md) | The primary artifact: build args, runtime environment, entrypoint, probes, volumes/ports/signals, non-root contract, image names, tags and digests. | +| [examples.md](examples.md) | The example contract: required layout, manifest rules, exposure variants, components and image naming, scripts, docs, and how to add a new example. | +| [gitflow.md](gitflow.md) | Branches, protected `main`, squash merges, Conventional PR titles, required CI checks, release branches, hotfixes and the one-time GitHub settings checklist. | +| [versioning.md](versioning.md) | Release identity (`DSH ref` + optional `-rN`), why the current identity is `dsh-v0.2.1-rc.2-r2`, channels, compatibility matrix, bumping pins, the release checklist, rollback, git-cliff and digest pinning. | + +## Example documentation + +| Document | Read it for | +| --- | --- | +| [examples/README.md](../examples/README.md) | The index of implementations in this repository. | +| [dsh-agents-as-a-service README](../examples/dsh-agents-as-a-service/README.md) | The multi-tenant platform walkthrough: warm pool, claims, suspend/resume, gateway, exposure and troubleshooting. | +| [dsh-agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md) | Container/sequence diagrams, ADRs, security and RBAC, failure modes and the evolution path. | +| [gateway README](../examples/dsh-agents-as-a-service/gateway/README.md) | The gateway's HTTP/WebSocket API, configuration, auth (including SSO), and limitations. | + +## Project policies + +| Document | Read it for | +| --- | --- | +| [AGENTS.md](../AGENTS.md) | Repository conventions for AI agents and humans, and the commands the CI runs. | +| [CONTRIBUTING.md](../CONTRIBUTING.md) | Local setup, the change workflow and what a pull request needs. | +| [SECURITY.md](../SECURITY.md) | Vulnerability reporting and project-specific threat notes (pod logs, SSO header trust, shared provider key, TLS). | +| [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) | Community standards. | +| [CHANGELOG.md](../CHANGELOG.md) | Generated release history (do not edit by hand). | +| [LICENSE](../LICENSE) | MIT. | diff --git a/docs/examples.md b/docs/examples.md new file mode 100644 index 0000000..6e6d132 --- /dev/null +++ b/docs/examples.md @@ -0,0 +1,149 @@ +# Example contract + +Every implementation in this repository is a **self-contained example** under +[`examples//`](../examples/). Examples demonstrate how to run the main +DSH Web image (and, when needed, auxiliary components such as the +agents-as-a-service gateway) on Kubernetes. They are reference material, not +libraries: each one is applied with plain `kubectl`, documented end to end, and +deletable in one command. + +[`examples/README.md`](../examples/README.md) is the index. If an example is not +listed there, it does not exist as far as `make check` is concerned. + +## Required layout + +``` +examples// ++-- README.md # required: what it demonstrates, prerequisites, +| # ordered steps, troubleshooting, cleanup ++-- manifests/ # required +| +-- NN-*.yaml # numbered top-level manifests, plain kubectl +| +-- exposure/ # optional exposure variants +| +-- README.md # required when exposure/ exists +| +-- /NN-*.yaml # e.g. ingress/, gateway-api-sso/ ++-- scripts/ # optional +| +-- *.sh # executable, idempotent, self-cleaning ++-- docs/ # optional deeper material (architecture, ADRs) ++-- / # optional published component + +-- Dockerfile + +-- README.md # required whenever a Dockerfile exists + +-- tests... +``` + +### Rules the CI enforces + +[scripts/check-examples.sh](../scripts/check-examples.sh) fails the build when: + +- `examples/README.md` is missing or does not mention `examples//` for every + example directory; +- an example has no `README.md` or no `manifests/` directory; +- `manifests/` contains no top-level `*.yaml` files; +- a `*.yaml` file sits at the example root instead of under `manifests/`; +- `manifests/exposure/` exists without a `manifests/exposure/README.md` describing + the variants; +- a subdirectory with a `Dockerfile` has no `README.md`; +- a file in `scripts/` is not executable. + +[scripts/check-manifests.py](../scripts/check-manifests.py) additionally parses +every `manifests/**/*.yaml` and requires `apiVersion`, `kind` and +`metadata.name`. CI also runs a server-side dry-run against the real +agent-sandbox CRDs on a throwaway kind cluster. + +Run both locally with: + +```bash +make check +``` + +## Manifests + +- **Numbered and ordered.** `NN-*.yaml` files are applied in filename order and + read as a walkthrough: `00-prereqs.yaml`, `10-sandboxtemplate.yaml`, + `20-sandboxwarmpool.yaml`, ... Keep the numbering stable; it is referenced from + the example README. +- **Plain kubectl.** No Helm, Kustomize, or custom installer. `kubectl apply -f` + against a namespace the example creates is the whole interface. +- **Safe by default.** Never ship a usable Secret. Secret-shaped manifests are + `*.yaml.example` files with obvious placeholders and an inline creation + command; real values are generated per install. +- **Cluster-specific state stays out.** Node affinity, replica counts tuned for a + cluster, and similar fields belong in a local patch, not in the shipped + manifest. +- **Placeholders are explicit.** Hostnames, GitHub orgs and TLS material use + values like `dsh.example.com` / `your-github-org` with a TODO comment. + +## Exposure variants + +An example that can be reached from outside the cluster puts each option under +`manifests/exposure//` and describes the trade-offs in +`manifests/exposure/README.md`. Variants are alternatives, not a stack: apply one. +`dsh-agents-as-a-service` currently ships: + +| Variant | Path | What it gives you | +| --- | --- | --- | +| Ingress | `manifests/exposure/ingress/` | Traefik-style Ingress + TLS; the gateway does its own session auth. | +| Gateway API + SSO | `manifests/exposure/gateway-api-sso/` | Gateway API `Gateway`/`HTTPRoute` in front of oauth2-proxy (GitHub OAuth); the gateway inherits `X-Forwarded-User` via `AUTH_PROXY_HEADER`. | + +## Components + +A component is a program the example needs beyond the main image: today only the +agents-as-a-service gateway. A component lives in its own subdirectory with: + +- `Dockerfile` — its image build; +- `README.md` — purpose, API/config reference, local development, limitations; +- tests — unit tests that run without a cluster, plus whatever else applies. + +Component images are published as **nested packages below the repository image**: + +``` +//// +``` + +so the gateway is +`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway`. +Never create an unrelated top-level package; `check-versions.sh` verifies the +nesting. + +## Scripts + +`scripts/*.sh` must be: + +- **executable** (the CI checks the mode bit); +- **idempotent** — safe to re-run, cleaning up before creating; +- **self-cleaning** — a trap removes the namespace, port-forwards and temp files, + even on failure; +- **non-interactive** — usable in CI, with environment-variable knobs instead of + prompts. + +`examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` is the reference +implementation: it runs the README walkthrough and deletes everything it created. + +## Documentation + +- The example `README.md` is the walkthrough. Keep it runnable: every command + block should work when pasted in order. +- Deeper material (architecture, sequence diagrams, ADRs) belongs in the + example's `docs/`, not in the repository-level [docs/](../docs/). +- Repository-wide docs describe the container and the repo rules; example docs + describe one implementation. + +## Adding a new example + +1. Create `examples//` with the layout above. +2. Write `README.md` first: what it demonstrates, prerequisites, numbered steps, + troubleshooting, limitations, cleanup. +3. Add `manifests/NN-*.yaml`; keep secrets as `*.yaml.example`. +4. If a component needs an image, publish it under + `////`. +5. Add an entry to [`examples/README.md`](../examples/README.md) using the literal + `examples//` path. +6. Run `make check` and, if you added a component, `make test` and `make lint`. +7. If the example is expensive to run (kind, images, cluster CRDs), ship a + `scripts/e2e-kind.sh` and wire it into CI. + +## Current examples + +| Example | Demonstrates | +| --- | --- | +| [`examples/dsh-agents-as-a-service/`](../examples/dsh-agents-as-a-service/) | The full multi-tenant platform pattern: SandboxTemplate, warm pool, per-user claims, suspend/resume, and the browser gateway. | +| [`examples/minimal-single-tenant/`](../examples/minimal-single-tenant/) | A single DSH Web sandbox: one SandboxTemplate and a claim. | diff --git a/docs/gitflow.md b/docs/gitflow.md new file mode 100644 index 0000000..e2053d2 --- /dev/null +++ b/docs/gitflow.md @@ -0,0 +1,128 @@ +# Gitflow: branches, pull requests and CI + +This repository uses **trunk-based development with release branches**. `main` is +always releasable; feature work happens on short-lived branches; releases are cut +from `main` onto a release branch and tagged with an immutable release identity. +See [versioning](versioning.md) for what the tags mean. + +## Branches + +| Branch | Branched from | Merged into | Purpose | +| --- | --- | --- | --- | +| `main` | — | — | The integration trunk. Protected, linear history, always releasable. | +| `feat/` | `main` | `main` (squash) | New capability or example. | +| `fix/` | `main` | `main` (squash) | Bug fix. | +| `docs/` | `main` | `main` (squash) | Documentation only. | +| `refactor/`, `chore/`, `test/` | `main` | `main` (squash) | Non-behavioral rework, maintenance, tests. | +| `release/` | `main` at the release commit | — | Release stabilization and tagging. Only release-blocking fixes land here. | +| `hotfix/` | The release branch (or the tagged commit) | `main` and the release branch | Urgent production fix. | + +Keep branches short-lived. Rebase on `main` rather than merging `main` into the +branch; the PR is squash-merged, so individual branch commits are never part of +`main` history. + +## Pull requests + +1. **One scoped change per PR.** Do not mix a version bump with unrelated work, + and update the docs the change touches in the same PR. +2. **Conventional PR title.** The title becomes the squash commit message, so it + is linted by the `pr-title` check: `feat(scope): ...`, `fix(scope): ...`, + `docs(scope): ...`, `build: ...`, `ci: ...`, `refactor: ...`, `test: ...`, + `chore: ...`. Breaking changes use `feat(scope)!: ...` or a + `BREAKING CHANGE:` footer. +3. **CI must be green.** All required checks below must pass before merge. +4. **Squash merge only.** `main` stays linear and protected; merged PRs are one + commit each. +5. **No direct pushes to `main`** or to a release branch. Both are protected. +6. **Review for behavior changes.** Infrastructure, workflow, release and + gateway-contract changes need a maintainer review; trivial docs changes still + need the checks. + +## Required checks + +| Check | Workflow / job | What it verifies | +| --- | --- | --- | +| `lint` | `ci.yaml` | `scripts/check-versions.sh` (version pins and markdown tokens), ruff on the gateway, shellcheck on shell scripts, yamllint/actionlint when available. | +| `python` | `ci.yaml` | Gateway unit tests (`pytest`). | +| `manifests` | `ci.yaml` | `scripts/check-examples.sh`, `scripts/check-manifests.py` and a server-side dry-run of every example manifest against the agent-sandbox CRDs. | +| `gateway-image` | `ci.yaml` | Builds the gateway image and smoke-tests `/healthz` without a cluster. | +| `pr-title` | `pr-title.yaml` | Conventional Commit PR title. | +| `CodeQL` | `codeql.yml` | Static analysis for Python and GitHub Actions. | +| `dependency-review` | `ci.yaml` | Dependency changes introduced by the PR (job `dependency-review`; runs on pull requests only). | + +The image build workflows (`_build-image.yaml`, `build-main.yaml`, +`build-gateway.yaml`) and `e2e-kind.yaml` are **not** required on every PR: the +image builds run where they are needed, and the kind end-to-end suite is reserved +for release validation and manual dispatch because it is slow. + +## Releases + +Releases are cut from `main`: + +1. Confirm `main` is green and `make check`, `make test`, `make lint` pass. +2. Bump [versions.env](../versions.env) if the DSH ref changed, using + `make set-version`; otherwise keep the same `DSH_VERSION` and bump + `RELEASE_REVISION`. +3. Create `release/` from the release commit. +4. Run `make release-dry-run` and review the generated notes. +5. Create and push the annotated `dsh-*` release tag. The `release.yaml` + workflow validates that the tag matches [versions.env](../versions.env), builds + and pushes the multi-arch images (version + latest), creates the GitHub + Release from the git-cliff notes, and opens a `chore(changelog)` PR that + refreshes `CHANGELOG.md` on `main`. +6. Merge the changelog PR and any release-branch-only fixes back to `main`. + +Full procedure: [versioning](versioning.md). + +## Hotfixes + +A released tag is **never** moved or re-tagged — tags and image identities are +immutable. To fix a broken release: + +1. Branch `hotfix/` from the release tag (or its release branch). +2. Make the minimal fix, add a test, open a PR; the same required checks apply. +3. Publish a new identity with a higher `RELEASE_REVISION` (`-r(N+1)`). Do not + reuse or delete the broken tag. +4. Merge the fix back into `main` (or cherry-pick it) so the trunk is fixed too. + +Rollback is also a re-release, never a retag: see +[versioning](versioning.md#rollback). + +## Tag protection + +All release tags match `dsh-*` and are protected: + +- only maintainers (the release workflow) may create them; +- they cannot be deleted or force-updated; +- they are annotated and point at the release commit on a release branch. + +Image tags derive from the same identity and are equally immutable; the mutable +`latest` and `main` image channels always point at a release or a trunk build +respectively. + +## One-time GitHub settings checklist + +Configure these on the repository before relying on the workflow: + +- [ ] **Branches → main**: require a pull request; require the checks `lint`, + `python`, `manifests`, `gateway-image`, `pr-title`, `CodeQL`, + `dependency-review`; require branches to be up to date; require conversation + resolution; restrict direct pushes; allow only squash merges and enable + "Automatically delete head branches". +- [ ] **Branches → release/***: same protections, plus allow the release workflow + to push tags. +- [ ] **Tags → dsh-***: restrict creation/update/deletion to maintainers; a tag + push triggers the release workflow. +- [ ] **Actions → General**: allow GitHub Actions; set workflow permissions to + read-only by default; allow `GITHUB_TOKEN` to create pull requests only + where required. +- [ ] **Actions → Runners**: make the arm64 runners (`ubuntu-24.04-arm`) available + to the public/private repository as appropriate. +- [ ] **Packages**: grant the release workflow `packages: write` (workflow + permissions do this) and protect the + `ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox` packages from deletion. +- [ ] **Security**: enable CodeQL default setup **or** keep the advanced + `codeql.yml` workflow (not both), enable Dependabot alerts and + dependency review. +- [ ] **Secrets/Variables**: none are needed for image publishing; the workflows + use `GITHUB_TOKEN`. diff --git a/docs/main-image.md b/docs/main-image.md new file mode 100644 index 0000000..f7ab861 --- /dev/null +++ b/docs/main-image.md @@ -0,0 +1,169 @@ +# The main DSH Web image + +This repository's primary artifact is a container image that packages +[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web +(`dsh web`) for Kubernetes and plain Docker. The [Dockerfile](../Dockerfile) at the +repository root builds it; every version pin it uses lives in +[versions.env](../versions.env). + +The image is image-first: it is the product, and the examples under +[`examples/`](../examples/) are self-contained implementations that consume it. + +## What the build does + +The Dockerfile is a two-stage build: + +1. **Build stage** (`node:24-trixie`) clones DSH at the pinned git ref, installs + dependencies with pnpm, runs the production build, packages the `@deepseek-ai/dsh` + CLI with `pnpm pack`, and installs its production dependencies with npm. +2. **Runtime stage** (`node:24-trixie-slim`) copies `/opt/dsh` to `/app`, adds the + CLI tools DSH agents commonly use (`git`, `ripgrep`, `jq`, `yq`, `curl`, `wget`, + `gh`, `tini`, `uv`), and starts `dsh web` through a small entrypoint. + +## Build it + +```bash +make image # canonical: builds the DSH Web image, no push +``` + +`make image` reads [versions.env](../versions.env) and passes the pins as build args. +To reproduce the equivalent by hand: + +```bash +docker build \ + --build-arg DSH_VERSION=dsh-v0.2.1-rc.2 \ + --build-arg RELEASE_REVISION=r2 \ + --build-arg PNPM_VERSION=11.28.5 \ + --build-arg VCS_REF="$(git rev-parse --short HEAD)" \ + -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 . +``` + +### Build args + +| Arg | Default / source | Purpose | +| --- | --- | --- | +| `DSH_REPO` | `https://github.com/deepseek-ai/deepseek-harness.git` | Upstream DSH repository to clone. | +| `DSH_VERSION` | `DSH_VERSION` in [versions.env](../versions.env) | Upstream DSH git tag/ref to package; recorded as `org.opencontainers.image.version`. | +| `RELEASE_REVISION` | `RELEASE_REVISION` in [versions.env](../versions.env) | Repository revision suffix (`rN`); the release identity is `DSH_VERSION-RELEASE_REVISION`, recorded in the `io.sredevopsorg.release.revision` label. See [versioning](versioning.md). | +| `PNPM_VERSION` | `PNPM_VERSION` in [versions.env](../versions.env) | pnpm version activated with corepack in both stages. | +| `VCS_REF` | `git rev-parse --short HEAD` (via `make image`) | Source revision recorded for traceability. | +| `NODE_BUILD_IMAGE` | `docker.io/node:24-trixie` | Build-stage base image. | +| `NODE_RUNTIME_IMAGE` | `docker.io/node:24-trixie-slim` | Runtime-stage base image. | + +Never hard-code a version outside [versions.env](../versions.env): +`./scripts/check-versions.sh` (and `make check`) fails when a pin drifts. + +## Runtime contract + +### Entrypoint + +The entrypoint is `tini -- /usr/local/bin/dsh-entrypoint`: + +1. `tini` runs as PID 1 and reaps children/forwards signals. +2. The shell entrypoint applies defaults, derives a bind address when `DSH_HOST` is + empty (the pod/container's first non-loopback IPv4), builds the + `dsh web` command line, and `exec`s it so signals reach the Node process. +3. `localhost` and `127.0.0.1` are always added to DSH's trusted-host list; each + `DSH_TRUSTED_HOSTS` entry is added as an extra `--trusted-host`. + +### Environment variables + +| Variable | Default | Purpose | +| --- | --- | --- | +| `DSH_PORT` | `3080` | Port DSH Web listens on inside the container. | +| `DSH_HOST` | auto-detected non-loopback IPv4 | Bind address. Leave empty in Kubernetes so the pod IP is used automatically. | +| `DSH_PUBLIC_URL` | `http://localhost:${DSH_PORT}` | Public base URL used to build links and the launch-token URL printed in the logs. Set it when the container is published on a different host port or behind a proxy. | +| `DSH_TRUSTED_HOSTS` | empty | Extra `Host`/`Origin` allow-list entries, comma- or space-separated. `localhost`/`127.0.0.1` are always trusted. Required when a proxy forwards the browser's `Host`. | +| `DEEPSEEK_API_KEY` | empty | Provider credential read by DSH. Inject it with `-e`/`envFrom`; never bake it into the image. Only model calls fail when it is missing. | +| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (conversations, sessions, config, credentials). Mount a volume here to persist it. | + +The image also sets `NODE_ENV=production`, `HOME=/home/node`, and a `PATH` that +includes `/app/node_modules/.bin`. + +### Ports, volumes and signals + +| Item | Value | Notes | +| --- | --- | --- | +| Exposed port | `3080/tcp` | `EXPOSE 3080`; one port serves the UI, assets, API, SSE and WebSocket traffic. | +| State volume | `/home/node/.dsh` | Declared with `VOLUME`; use a named volume (Docker) or PVC (Kubernetes). | +| Working directory | `/workspace` | The agent workspace; mount a volume here if agents should keep files. | +| Stop signal | `SIGTERM` | `tini` forwards it to `dsh web`; allow a graceful shutdown period. | +| User | `node`, UID/GID `1000` | The process is non-root. A PVC mounted at `DSH_HOME` must be writable by UID 1000. | + +### Health probes + +`/` is **not** a health endpoint: DSH returns `401` until a browser completes the +launch-token handshake. Probe `/favicon.svg` instead — it is unauthenticated and +served by the web layer, so it proves the HTTP server is up without triggering auth. + +```yaml +startupProbe: + httpGet: {path: /favicon.svg, port: 3080} + periodSeconds: 5 + failureThreshold: 24 +readinessProbe: + httpGet: {path: /favicon.svg, port: 3080} + periodSeconds: 5 +livenessProbe: + httpGet: {path: /favicon.svg, port: 3080} + periodSeconds: 15 +``` + +The shipped templates use exactly this shape; see +[`examples/minimal-single-tenant/manifests/sandbox-template.yaml`](../examples/minimal-single-tenant/manifests/sandbox-template.yaml). +A `startupProbe` is important because plugin activation (or the first start of a +large sandbox) can exceed a default liveness budget. + +### Launch token + +DSH prints its one-time launch URL to the logs: + +``` +dsh web: http://:${DSH_PORT}/?token=... +``` + +Open that URL once to obtain DSH's auth cookie. After that the UI works at the bare +URL; without the cookie `/` answers `401` by design. The +[agents-as-a-service gateway](../examples/dsh-agents-as-a-service/gateway/) performs +this handshake server-side so browsers never see the token. + +## Published image names and tags + +Registry layout: the main image is the repository itself, and every example +component is a nested package below it: + +``` +ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox # main DSH Web image +ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway # example component +``` + +The main image is published multi-arch (`linux/amd64`, `linux/arm64`) to +GitHub Container Registry: + +| Tag | Meaning | +| --- | --- | +| `dsh-v0.2.1-rc.2-r2` | Current release identity. **Immutable**: once published it never moves. | +| `latest` | Most recent release; points at the same digest as the current identity. | +| `main` | Development channel rebuilt from every push to `main`; not a release, not for production. | +| `dsh-v0.2.1-rc.2` | Legacy tag published before the release-identity scheme existed. Do not use it for new deployments; see the migration table in the [root README](../README.md). | + +Release identities, channels and rollback rules are defined in +[versioning](versioning.md). Tags are immutable, so production deployments should +pin a digest: + +```bash +docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 +# ... pin the digest in your deployment manifests: +# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256: +``` + +## Verifying an image + +```bash +docker inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 \ + --format '{{json .Config.Labels}}' +# org.opencontainers.image.title / .version / .source +``` + +Release builds also publish SBOM and provenance attestations from the release +workflow (see [SECURITY.md](../SECURITY.md) and [gitflow](gitflow.md)). diff --git a/docs/versioning.md b/docs/versioning.md new file mode 100644 index 0000000..2f68155 --- /dev/null +++ b/docs/versioning.md @@ -0,0 +1,163 @@ +# Versioning, releases and changelog + +[versions.env](../versions.env) is the **single source of truth** for the DSH ref, +the repository revision, the pnpm version, the control-plane component versions +and the image names. Nothing else may hard-code a version: +[scripts/check-versions.sh](../scripts/check-versions.sh) fails when a pin drifts, +and it also verifies that prose mentions only the current release identity or its +base DSH tag. + +## Release identity + +``` +release identity = DSH_VERSION + optional "-" + RELEASE_REVISION +``` + +| Piece | Meaning | +| --- | --- | +| `DSH_VERSION` | The upstream DeepSeek Harness git ref being packaged, e.g. `dsh-v0.2.1-rc.2`. It mirrors upstream's own tag, which is what "mirrored-DSH scheme" means. | +| `RELEASE_REVISION` | An optional repository revision at that DSH ref: empty for the first release, `rN` (`N >= 2`) for a repository-only re-release. | + +Examples: + +- `DSH_VERSION=dsh-v0.2.1-rc.2`, `RELEASE_REVISION=` → identity `dsh-v0.2.1-rc.2` + (first release at that upstream ref). +- `DSH_VERSION=dsh-v0.2.1-rc.2`, `RELEASE_REVISION=r2` → identity + `dsh-v0.2.1-rc.2-r2` (documentation, CI or gateway changes on top of the same + upstream DSH; no DSH upgrade). + +A DSH upgrade produces a new base identity (for example the next upstream ref +with an empty revision); a repository-only change at the same DSH ref increments +`rN`. + +## Why the current identity is `dsh-v0.2.1-rc.2-r2` + +The image tag `dsh-v0.2.1-rc.2` was already published to GHCR for an earlier +repository state. **Container tags are immutable** in this project: re-pushing a +tag would silently change what an existing deployment runs. The mirrored-DSH +scheme therefore starts its first identity at `-r2`: the plain tag is +conceptually revision 1 and is taken, so the first publishable repository +revision at the same upstream ref is `r2`. + +The identity is stored in [versions.env](../versions.env) and consumed everywhere: +the Dockerfile label, the example manifests, the Makefile and the release +workflow. The legacy `dsh-v0.2.1-rc.2` tag remains published but is not updated; +new deployments use `dsh-v0.2.1-rc.2-r2`. + +The gateway image moved from the repository-level `:gateway` tag to a nested +package (`/dsh-agents-as-a-service/gateway`). For exactly one +release the release workflow also republishes the legacy +`:gateway` tag as a deprecated alias; it is removed in the next +release, so deployments should pin the nested package. + +## Channels + +| Channel | Image tag | Meaning | +| --- | --- | --- | +| Release | `dsh-v0.2.1-rc.2-r2` | The current immutable release identity. What production should pin (preferably by digest). | +| `latest` | `latest` | Alias for the most recent release; moves only when a new release is published. | +| `main` | `main` | Development channel, rebuilt from every push to `main`. Never treat it as a release; it exists so `main` can be tested. | + +The same channels exist for every published package, including the nested +example component image. + +## Compatibility matrix + +Values from [versions.env](../versions.env) at the current identity: + +| Component | Pin | Source | +| --- | --- | --- | +| Upstream DSH source | `DSH_REPO` | [versions.env](../versions.env) | +| DSH ref | `dsh-v0.2.1-rc.2` | `DSH_VERSION` | +| Repository revision | `r2` | `RELEASE_REVISION` | +| pnpm | `11.28.5` | `PNPM_VERSION` | +| agent-sandbox (CRDs) | `v1.0.7` | `AGENT_SANDBOX_VERSION`, used by the examples and CI dry-run | +| Kubernetes Gateway API | `v1.2.1` | `GATEWAY_API_VERSION`, used by the SSO exposure variant | +| oauth2-proxy | `quay.io/oauth2-proxy/oauth2-proxy:v7.15.5` | `OAUTH2_PROXY_IMAGE` | +| Main image | `ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox` | `GHCR_IMAGE` | +| Gateway image | `ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway` | `GATEWAY_IMAGE`, nested below `GHCR_IMAGE` | + +Example component images are always nested below the main package: +`//`. `check-versions.sh` rejects an unrelated +top-level package. + +## Bumping versions + +```bash +make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=rN +make check +``` + +`set-version` updates [versions.env](../versions.env) through +[scripts/set-version.sh](../scripts/set-version.sh) and leaves every consuming +file to derive from it. Never edit the Dockerfile, manifests, Makefile or +workflows with a literal version. + +- New upstream DSH ref: change `DSH_VERSION` and clear `RELEASE_REVISION`. +- Repository-only change at the same DSH ref: keep `DSH_VERSION`, set + `RELEASE_REVISION=r(N+1)`. + +## Release checklist + +1. `main` is green and `make check`, `make test`, `make lint` pass locally. +2. `make image` and `make gateway` build; smoke-test the images. +3. Bump the pins with `make set-version` (commit that change through a PR). +4. `make release-dry-run` — verifies the identity, prints the image names, checks + the tag does not already exist, and previews the generated notes. +5. Create and push the annotated `dsh-*` release tag (or dispatch + `release.yaml` with an existing tag). The workflow validates that the tag + matches [versions.env](../versions.env). +6. The workflow builds and pushes multi-arch images for the main image and the + example components (the identity tag plus `latest`), creates the GitHub + Release from the git-cliff notes, records the published digests, and opens a + `chore(changelog)` PR that refreshes `CHANGELOG.md` on `main`. +7. Merge the changelog PR. Verify: + `docker buildx imagetools inspect :` shows both platforms, + and the GitHub Release references the tag and the digest. +8. Announce the identity and the digest; deployments pin the digest. + +## Rollback + +Tags and release identities are **immutable**, so a bad release is never re-tagged +or deleted. Fix forward: + +1. Fix the problem on a `hotfix/` branch (see [gitflow](gitflow.md)). +2. Publish a new identity with the next revision, `-r(N+1)`, at the same + `DSH_VERSION`. +3. Point deployments at the new digest; keep the broken tag for audit history. + +Emergency rollback to the previous release is also a re-deploy of a **previous +digest**, not a move of `latest`. + +## Changelog + +`CHANGELOG.md` is generated by [git-cliff](https://git-cliff.org) from +Conventional Commits, configured in [cliff.toml](../cliff.toml): + +```bash +make changelog # regenerate CHANGELOG.md locally +``` + +- Do not edit `CHANGELOG.md` by hand; the release workflow regenerates it and opens a PR to refresh it on `main`. +- Commit messages and PR titles must be Conventional Commits so they are grouped + correctly (Added / Fixed / Changed / Documentation / Build / CI / Tests / + Maintenance). +- The changelog groups commits since the previous `dsh-*` tag; release notes use + the same rendering. +- The changelog is excluded from the version-token check because it records + historical identities on purpose. + +## Digest pinning + +Release identities are mutable only in theory; deployments should still pin the +resolved digest for reproducibility and supply-chain safety: + +```bash +docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 +# then, in the deployment: +# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256: +``` + +The example manifests use the identity tag so they stay readable and upgrade with +`versions.env`; production overlays should pin the digest. Release images carry +SBOM and provenance attestations (see [SECURITY.md](../SECURITY.md)). diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..0c233bb --- /dev/null +++ b/examples/README.md @@ -0,0 +1,47 @@ +# Examples + +Every implementation in this repository is a self-contained example: one +directory, plain `kubectl`, an end-to-end README, and a cleanup command. +Examples consume the [main DSH Web image](../docs/main-image.md); they are +reference material, not libraries. + +| Example | What it demonstrates | Needs | +| --- | --- | --- | +| [`examples/dsh-agents-as-a-service/`](dsh-agents-as-a-service/) | The full multi-tenant platform pattern: one `SandboxTemplate` with per-user PVCs, a warm pool, one ~10-line `SandboxClaim` per user, suspend-when-idle / wake-on-return, and a browser gateway with signup, `HttpOnly` sessions, an HTTP + WebSocket proxy and two exposure variants (Ingress, or Gateway API + GitHub SSO). | agent-sandbox with extensions; kind for the e2e script. | +| [`examples/minimal-single-tenant/`](minimal-single-tenant/) | The smallest useful setup: a single `SandboxTemplate` and one `SandboxClaim`, no gateway — reach DSH Web with a launch-token URL through `kubectl port-forward`. | agent-sandbox with extensions. | + +## The example contract (short version) + +Full rules: [docs/examples.md](../docs/examples.md). `scripts/check-examples.sh` +enforces the layout, and `make check` runs it. + +``` +examples// ++-- README.md # walkthrough: prerequisites, steps, troubleshooting, cleanup ++-- manifests/ # NN-*.yaml, applied with plain kubectl +| +-- exposure// # optional per-variant manifests... +| +-- exposure/README.md # ...required whenever exposure/ exists ++-- scripts/ # optional executable, idempotent, self-cleaning *.sh ++-- docs/ # optional architecture/ADR material ++-- / # optional component with Dockerfile + README + tests +``` + +- Secrets never ship populated: use `*.yaml.example` files with placeholders. +- Manifests are numbered (`00-`, `10-`, ...) and read as an ordered walkthrough. +- Component images are published nested below the main image: + `////`. +- Every example must be listed above with its literal `examples//` path. +- Scripts under `scripts/` must be executable, idempotent and clean up after + themselves. + +## Running an example + +```bash +# e.g. the multi-tenant platform example +kubectl apply -f examples/dsh-agents-as-a-service/manifests/ + +# or run the whole walkthrough (kind + agent-sandbox required) and let it clean up +examples/dsh-agents-as-a-service/scripts/e2e-kind.sh +``` + +Each example README has the exact prerequisites, exposure options and teardown. diff --git a/examples/dsh-agents-as-a-service/README.md b/examples/dsh-agents-as-a-service/README.md index 81f7416..fde7fe7 100644 --- a/examples/dsh-agents-as-a-service/README.md +++ b/examples/dsh-agents-as-a-service/README.md @@ -2,20 +2,41 @@ This example distills the upstream [hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service) -platform pattern into pure agent-sandbox resources + kubectl, running **DeepSeek Harness Web -(DSH)** instead of Hermes. Every user of such a platform gets a personal, long-lived DSH Web -instance that is **provisioned in ~2 seconds from a warm pool, suspended when idle to save -cost, and woken transparently when the user returns** — with all state (conversations, -sessions, config) surviving on a per-user PVC at `/home/node/.dsh`. +platform pattern into pure agent-sandbox resources + kubectl, running **DeepSeek +Harness Web (DSH)** instead of Hermes. Every user of such a platform gets a +personal, long-lived DSH Web instance that is **provisioned in ~2 seconds from a +warm pool, suspended when idle to save cost, and woken transparently when the +user returns** — with all state (conversations, sessions, config) surviving on a +per-user PVC at `/home/node/.dsh`. -It ships in two layers: +It ships in three layers: -1. **The resource layer** (steps 1–7): pure manifests + kubectl, so every mechanism is visible. -2. **The service layer** (step 8): a small included **gateway** (`gateway/`) that makes the - platform browser-usable — a signup page/API, `HttpOnly` cookie sessions, a `/u/{user}/**` - proxy for HTTP **and WebSocket** that transparently **wakes suspended instances on - connect**, and an idle sweeper that suspends them again. The gateway performs the DSH - launch-token handshake server-side, so the browser never sees the token. +1. **The resource layer** (steps 1–7): pure manifests + kubectl, so every + mechanism is visible. +2. **The service layer** (step 8): a small included **gateway** (`gateway/`) that + makes the platform browser-usable — a signup page/API, `HttpOnly` cookie + sessions, a `/u/{user}/**` proxy for HTTP **and WebSocket** that transparently + **wakes suspended instances on connect**, and an idle sweeper that suspends + them again. The gateway performs the DSH launch-token handshake server-side, + so the browser never sees the token. +3. **The exposure layer** (step 9): choose exactly one way in — a plain Traefik + Ingress, or Gateway API + oauth2-proxy (GitHub SSO). + +All manifests live under [`manifests/`](manifests/): + +``` +manifests/ ++-- 00-prereqs.yaml # namespace + empty provider-key Secret ++-- 10-sandboxtemplate.yaml # the DSH Web blueprint ++-- 20-sandboxwarmpool.yaml # 2 pre-warmed spares ++-- 30-claim-alice.yaml # one user ++-- 40-claim-rejected.yaml # negative demo: env injection rejected ++-- 50-gateway.yaml # Deployment/RBAC/Service for the gateway ++-- exposure/ + +-- README.md # the two exposure variants, when to use which + +-- ingress/60-ingress.yaml + +-- gateway-api-sso/70-gateway.yaml ... 74-tls-secret.yaml.example +``` What it demonstrates, feature by feature: @@ -34,20 +55,21 @@ What it demonstrates, feature by feature: - **Multi-arch**: the DSH image is built for amd64 and arm64, so there is no `kubernetes.io/arch: amd64` nodeSelector. -- **Single port**: DSH Web serves everything on `3080`; the gateway proxies to one upstream - instead of splitting `/v1/*` (API) from the dashboard. -- **One shared secret**: only `DEEPSEEK_API_KEY` is injected (via `envFrom`); there is no - dashboard basic-auth, session-cookie secret, or API-server key. -- **Launch-token handshake**: DSH gates `/` behind a token it prints only in its logs. The - gateway reads it, performs the handshake, and keeps the resulting auth cookie server-side; - the browser authenticates to the gateway with its own cookie. See - [How the DSH handshake works](#how-the-dsh-handshake-works). -- **Host allow-list**: DSH rejects unknown `Host` headers. The gateway forwards the browser's - `Host`, so the public hostname must be added to `DSH_TRUSTED_HOSTS` in the template. -- **Probes target `/favicon.svg`**: `/` returns `401` without the launch token, so a probe on - `/` crash-loops the pod. `/favicon.svg` is unauthenticated, and a `startupProbe` gates - liveness/readiness so a slow cold start (plugin activation, or a resumed sandbox) is not - restarted mid-boot. +- **Single port**: DSH Web serves everything on `3080`; the gateway proxies to one + upstream instead of splitting `/v1/*` (API) from the dashboard. +- **One shared secret**: only `DEEPSEEK_API_KEY` is injected (via `envFrom`); there + is no dashboard basic-auth, session-cookie secret, or API-server key. +- **Launch-token handshake**: DSH gates `/` behind a token it prints only in its + logs. The gateway reads it, performs the handshake, and keeps the resulting + auth cookie server-side; the browser authenticates to the gateway with its own + cookie. See [How the DSH handshake works](#how-the-dsh-handshake-works). +- **Host allow-list**: DSH rejects unknown `Host` headers. The gateway forwards + the browser's `Host`, so the public hostname must be added to + `DSH_TRUSTED_HOSTS` in the template. +- **Probes target `/favicon.svg`**: `/` returns `401` without the launch token, so + a probe on `/` crash-loops the pod. `/favicon.svg` is unauthenticated, and a + `startupProbe` gates liveness/readiness so a slow cold start (plugin + activation, or a resumed sandbox) is not restarted mid-boot. ## Prerequisites @@ -55,43 +77,49 @@ What it demonstrates, feature by feature: - agent-sandbox with extensions installed: ```bash - VERSION=v1.0.7 - kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/sandbox-with-extensions.yaml + VERSION=v1.0.7 # AGENT_SANDBOX_VERSION in versions.env + kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$VERSION/sandbox-with-extensions.yaml kubectl -n agent-sandbox-system rollout status deployment --timeout=180s ``` -Or run everything below in one shot with `./run-test-kind.sh` (expects a kind cluster + the -manifest above already applied; it creates and deletes the namespace itself). + For step 9's SSO variant you additionally need the Gateway API CRDs and a + Gateway implementation (`gatewayClassName: traefik` in the manifests); see the + [exposure README](manifests/exposure/README.md). + +- Optional: run the whole resource + gateway walkthrough in one shot with + [`scripts/e2e-kind.sh`](scripts/e2e-kind.sh). It expects a kind cluster with + agent-sandbox installed, then creates and deletes the `dsh-demo` namespace + itself. It stops after the gateway layer (step 8); exposure is not applied. ## 1. Install the platform resources -First the namespace, then the shared provider key. The key reaches **every** tenant sandbox -(a warm-pool constraint: pod env is stamped before any user exists), so it is **never -committed** — generate a fresh value per install: +First the namespace, then the shared provider key. The key reaches **every** +tenant sandbox (a warm-pool constraint: pod env is stamped before any user +exists), so it is **never committed** — generate a fresh value per install: ```bash -kubectl apply -f 00-prereqs.yaml +kubectl apply -f manifests/00-prereqs.yaml kubectl -n dsh-demo create secret generic dsh-provider-keys \ --from-literal=DEEPSEEK_API_KEY="sk-..." -kubectl apply -f 10-sandboxtemplate.yaml -f 20-sandboxwarmpool.yaml +kubectl apply -f manifests/10-sandboxtemplate.yaml -f manifests/20-sandboxwarmpool.yaml ``` -Watch the two warm spares boot (first time pulls the image — subsequent operations never pay -that again; that is the point of the pool): +Watch the two warm spares boot (first time pulls the image — subsequent operations +never pay that again; that is the point of the pool): ```bash kubectl -n dsh-demo get sandboxes -l agents.x-k8s.io/warm-pool-sandbox -w ``` -Wait until both show `READY: True`. Each spare already has its PVC bound and DSH Web serving -on port `3080`. +Wait until both show `READY: True`. Each spare already has its PVC bound and DSH +Web serving on port `3080`. ## 2. "Signup": claim an instance (~2s) A user signing up is one small claim — `warmPoolRef` plus a pod label: ```bash -kubectl apply -f 30-claim-alice.yaml +kubectl apply -f manifests/30-claim-alice.yaml kubectl -n dsh-demo wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s ``` @@ -105,20 +133,21 @@ kubectl -n dsh-demo get sandboxes --show-labels Three things worth noticing: -- **The sandbox keeps its pool-generated name** (`dsh-pool-…`), not the claim's name — - adoption re-labels an existing warm sandbox rather than creating one. A platform must always - resolve user → sandbox through `claim.status.sandbox.name`. -- The `agents.x-k8s.io/warm-pool-sandbox` label is **gone** from the adopted sandbox (it left - the pool's inventory), and the pool has already started replenishing back to 2 spares. +- **The sandbox keeps its pool-generated name** (`dsh-pool-…`), not the claim's + name — adoption re-labels an existing warm sandbox rather than creating one. A + platform must always resolve user → sandbox through `claim.status.sandbox.name`. +- The `agents.x-k8s.io/warm-pool-sandbox` label is **gone** from the adopted + sandbox (it left the pool's inventory), and the pool has already started + replenishing back to 2 spares. - The pod carries the claim's `additionalPodMetadata` label (`sandbox.users.io/dsh-user=alice`), and - `kubectl -n dsh-demo get sandbox $SB -o jsonpath='{.status.serviceFQDN}'` shows the stable - DNS name the gateway proxies to. + `kubectl -n dsh-demo get sandbox $SB -o jsonpath='{.status.serviceFQDN}'` shows + the stable DNS name the gateway proxies to. ## 3. Talk to the instance -DSH Web needs its launch token on first contact — a bare `/` returns `401`. The token is in the -pod logs: +DSH Web needs its launch token on first contact — a bare `/` returns `401`. The +token is in the pod logs: ```bash kubectl -n dsh-demo logs "$SB" | grep -m1 '?token=' @@ -127,16 +156,16 @@ kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080 # open http://localhost:18081/?token=... (from the log line) ``` -> **Local port note:** DSH Web listens on `3080` **in-cluster**. If `3080` is already in use -> on your workstation (e.g. by a local DSH), forward to a different local port (here `18081`), -> never `3080`. `localhost` is always in DSH's trusted-host list, so the forwarded request is -> accepted. +> **Local port note:** DSH Web listens on `3080` **in-cluster**. If `3080` is +> already in use on your workstation (e.g. by a local DSH), forward to a +> different local port (here `18081`), never `3080`. `localhost` is always in +> DSH's trusted-host list, so the forwarded request is accepted. -> **Why the pod stays up:** this claim was applied by hand, so it carries no gateway ownership -> annotation (`aaas.example.com/managed-by`). The gateway's idle sweeper only suspends claims it -> created through `POST /users`; the resource-layer claims from these manifests are never -> touched. You can deploy the gateway (step 8) and still keep a sandbox you are port-forwarding -> to. +> **Why the pod stays up:** this claim was applied by hand, so it carries no +> gateway ownership annotation (`aaas.example.com/managed-by`). The gateway's idle +> sweeper only suspends claims it created through `POST /users`; the resource-layer +> claims from these manifests are never touched. You can deploy the gateway +> (step 8) and still keep a sandbox you are port-forwarding to. ## 4. Write state on the PVC @@ -161,20 +190,20 @@ kubectl -n dsh-demo wait sandbox "$SB" --for=condition=Ready --timeout=180s kubectl -n dsh-demo exec "$SB" -- cat /home/node/.dsh/marker.txt # "remember me" ``` -> **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`. DSH requires -> its `.credentials.yaml` to be `0600`, while the kubelet's default fsGroup policy widens every -> file to `0660` on each pod start — a resumed sandbox would crash-loop. See -> [Troubleshooting](#troubleshooting) for the one-time fix on sandboxes created before this -> setting existed. +> **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`. +> DSH requires its `.credentials.yaml` to be `0600`, while the kubelet's default +> fsGroup policy widens every file to `0660` on each pod start — a resumed sandbox +> would crash-loop. See [Troubleshooting](#troubleshooting) for the one-time fix +> on sandboxes created before this setting existed. ## 7. Injection policy rejects env-carrying claims ```bash -kubectl apply -f 40-claim-rejected.yaml +kubectl apply -f manifests/40-claim-rejected.yaml kubectl -n dsh-demo get sandboxclaim dsh-mallory \ -o jsonpath='{.status.conditions[?(@.type=="Ready")].reason}' # EnvVarsInjectionRejected — the claim never bound a sandbox -kubectl delete -f 40-claim-rejected.yaml +kubectl delete -f manifests/40-claim-rejected.yaml ``` ## 8. The gateway: signup, browser proxy, idle-suspend, wake-on-connect @@ -182,59 +211,66 @@ kubectl delete -f 40-claim-rejected.yaml ### Deploy it ```bash -# Published image (on kind, load it into the cluster; on a real cluster it is -# pulled from GHCR): -docker pull ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway -kind load docker-image ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway --name +# Published image (on kind, load it into the cluster; on a real cluster the +# kubelet pulls it from GHCR): +GW=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2 +docker pull "$GW" +kind load docker-image "$GW" --name -# ... or build it locally and tag it exactly as 50-gateway.yaml expects: -docker build -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway gateway/ -kind load docker-image ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway --name +# ... or build it locally and tag it exactly as manifests/50-gateway.yaml +# expects (make gateway builds dsh-aaas-gateway:dev): +make gateway +docker tag dsh-aaas-gateway:dev "$GW" +kind load docker-image "$GW" --name -kubectl apply -f 50-gateway.yaml +kubectl apply -f manifests/50-gateway.yaml kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s kubectl -n dsh-demo port-forward svc/dsh-gateway 18080:8080 ``` > The gateway binds `8080`; the local port-forward uses `18080` and never `3080`. > -> **Re-deploying after a new image?** The workflow publishes the mutable `:gateway` tag and the -> Deployment uses `imagePullPolicy: IfNotPresent`, so a node that already cached the tag keeps -> the old image across `kubectl apply`/`rollout restart`. Pin the digest for a reproducible -> rollout (the workflow's `Inspect image` step and `docker buildx imagetools inspect` both show -> it): +> **Image pull policy.** `manifests/50-gateway.yaml` uses +> `imagePullPolicy: IfNotPresent` with the release identity, so a locally built +> image must be loaded into the cluster under exactly that name (kind: +> `kind load`). A node that already cached a different tag is the usual cause of +> "my new build is not running"; pin the digest for a reproducible rollout: > > ```bash > kubectl -n dsh-demo set image deploy/dsh-gateway \ -> gateway=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256: +> gateway=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway@sha256: > kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s > ``` > -> **Applying over an existing install?** Run `kubectl diff -f 10-sandboxtemplate.yaml` -> first. Cluster-specific fields such as `nodeAffinity` and the warm-pool replica count are -> intentionally not in the repo manifests; preserve them with a targeted patch instead of a +> **Applying over an existing install?** Run +> `kubectl diff -f manifests/10-sandboxtemplate.yaml` first. Cluster-specific +> fields such as `nodeAffinity` and the warm-pool replica count are intentionally +> not in the repo manifests; preserve them with a targeted patch instead of a > blind apply. Moving the probes to `/favicon.svg` is the crash-loop fix. > -> The template also sets `fsGroupChangePolicy: OnRootMismatch` (see the note in step 6). -> Existing `Sandbox` objects snapshot their pod template, so a template apply does not update -> them: patch each sandbox's `spec.podTemplate.spec.securityContext.fsGroupChangePolicy` and +> The template also sets `fsGroupChangePolicy: OnRootMismatch` (see the note in +> step 6). Existing `Sandbox` objects snapshot their pod template, so a template +> apply does not update them: patch each sandbox's +> `spec.podTemplate.spec.securityContext.fsGroupChangePolicy` and > `chmod 600` its `.credentials.yaml`, or recreate it. ### Browser flow -Open . The landing page has a one-field signup form. Enter `bob`, and -the gateway: +Open . The landing page has a one-field signup form. +Enter `bob`, and the gateway: 1. creates the `SandboxClaim` over the warm pool, 2. waits for it to become `Ready`, -3. sets an `HttpOnly`, `SameSite=Lax` cookie `dsh_gw_session=:` on the browser, +3. sets an `HttpOnly`, `SameSite=Lax` cookie `dsh_gw_session=:` on + the browser, 4. redirects to `/u/bob/`, where the DSH Web UI loads through the gateway. -From then on every request under `/u/bob/` (assets, API calls, SSE, WebSocket) is proxied to -bob's sandbox. The DSH launch token and DSH's own auth cookie stay in the gateway; the browser -only holds the gateway cookie. +From then on every request under `/u/bob/` (assets, API calls, SSE, WebSocket) is +proxied to bob's sandbox. The DSH launch token and DSH's own auth cookie stay in +the gateway; the browser only holds the gateway cookie. -The same flow with `curl`, no JavaScript (the session cookie is the only credential): +The same flow with `curl`, no JavaScript (the session cookie is the only +credential): ```bash JAR=$(mktemp) @@ -252,48 +288,53 @@ curl -s -b "$JAR" -H 'Origin: http://localhost:18080' -X DELETE http://localhost ``` API clients that prefer bearer tokens can use `POST /users` with a JSON body -(`{"user":"bob"}`) and the returned one-time token, `POST /login` to exchange a token for a -cookie, and `Authorization: Bearer ` on subsequent calls. +(`{"user":"bob"}`) and the returned one-time token, `POST /login` to exchange a +token for a cookie, and `Authorization: Bearer ` on subsequent calls. ### Gateway API | Route | Purpose | | --- | --- | | `GET /healthz` | Liveness/readiness; no cluster access | -| `GET /` | Landing page: signup, token login, logout | +| `GET /` | Landing page: signup, token login, logout. With SSO it auto-provisions and connects instead. | | `POST /users` | Signup. JSON or form; `201` (JSON), `202` (waking), `303` (browser redirect), `409` if user exists; sets the session cookie | | `POST /login` | Exchange a bearer token for the session cookie | | `POST /logout` | Clear the session cookie | +| `POST /recreate` | SSO only: delete the authenticated user's claim (wiping workspace data) and provision a fresh one; `303` to `/u/{user}/`. Cookie-authenticated calls must be same-origin. | | `GET /users/{user}` | Derived state (`Ready`/`Waking`/`Suspended`) | | `DELETE /users/{user}` | Delete claim (cascades sandbox + PVC) | | `GET /u/{user}` | `308` redirect to `/u/{user}/` | | `GET /u/{user}/**` | HTTP/SSE proxy to the user's DSH Web | | `WS /u/{user}/**` | WebSocket proxy to the user's DSH Web | +The full API reference, configuration table, auth semantics and failure matrix +live in the [gateway README](gateway/README.md). + ### How the DSH handshake works -DSH does not expose an API for its launch token; it prints a URL containing `?token=…` in the -pod logs. For each `(user, pod UID, Host)` the gateway: +DSH does not expose an API for its launch token; it prints a URL containing +`?token=…` in the pod logs. For each `(user, pod UID, Host)` the gateway: -1. resolves the claim → sandbox → pod (annotation `agents.x-k8s.io/pod-name`, or the sandbox - name as pod name), -2. reads the pod's logs (last `DSH_LOG_TAIL_LINES` lines) and parses the most recent - `dsh web: …?token=…` line, +1. resolves the claim → sandbox → pod (annotation `agents.x-k8s.io/pod-name`, or + the sandbox name as pod name), +2. reads the pod's logs (last `DSH_LOG_TAIL_LINES` lines) and parses the most + recent `dsh web: …?token=…` line, 3. requests `http://:3080/?token=…` with the browser's `Host`, 4. captures DSH's `dsh-auth-*` `Set-Cookie` response and caches it, -5. injects that cookie (and strips the browser's `Cookie`/`Authorization` and any `?token=`) - on every proxied HTTP and WebSocket request, -6. refreshes the handshake once if DSH answers `401`, and drops the cached cookie when the - sandbox suspends or is deleted. +5. injects that cookie (and strips the browser's `Cookie`/`Authorization` and any + `?token=`) on every proxied HTTP and WebSocket request, +6. refreshes the handshake once if DSH answers `401`, and drops the cached cookie + when the sandbox suspends or is deleted. -This is why the gateway's RBAC includes `pods` `get`/`list` and `pods/log` `get`. It is also -the most fragile part of the design; see [Limitations](#limitations). +This is why the gateway's RBAC includes `pods` `get`/`list` and `pods/log` +`get`. It is also the most fragile part of the design; see +[Limitations](#limitations). ### DSH_TRUSTED_HOSTS -DSH rejects requests whose `Host` is not allow-listed (`localhost` and `127.0.0.1` are always -trusted). Because the gateway forwards the browser's `Host`, set the public hostname in the -template: +DSH rejects requests whose `Host` is not allow-listed (`localhost` and +`127.0.0.1` are always trusted). Because the gateway forwards the browser's +`Host`, set the public hostname in the template: ```yaml env: @@ -301,36 +342,151 @@ env: value: "dsh.example.com" # comma or space separated; empty for port-forward only ``` -The same applies to a local `docker run`: the printed token URL uses `DSH_PUBLIC_URL` if set, -otherwise `http://localhost:${DSH_PORT}`. +The same applies to a local `docker run`: the printed token URL uses +`DSH_PUBLIC_URL` if set, otherwise `http://localhost:`. ### WebSocket -The gateway is a Starlette ASGI app and proxies WebSocket upgrades with a bidirectional pump, so -the DSH web GUI is fully interactive. Requirements: +The gateway is a Starlette ASGI app and proxies WebSocket upgrades with a +bidirectional pump, so the DSH web GUI is fully interactive. Requirements: -- the ingress/load balancer in front of the gateway must pass `Upgrade`/`Connection` headers; - `60-ingress.yaml` is a starting point (Traefik handles upgrades by default), -- idle suspension is disabled for users with in-flight proxied requests or open WebSockets, so - a long-running chat stream is not cut off mid-turn, -- the upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`, and performs - the launch-token handshake under that same `pod-IP:3080` authority. DSH fences on `Host` - (only its bind address, loopback, or an allow-listed host passes) and binds its auth cookie to - the request authority (`payload.authority === authority`), while `websockets` derives `Host` - from the URI and cannot be overridden. A browser-host cookie or forwarded origin makes the - upgrade fail (`401`/`403`, surfaced as close `1011`), which the client retries forever as - `[connection] connection lost, retry #N`, -- missing/invalid gateway auth is rejected right after the upgrade with close code `4401` - (WebSocket auth has no HTTP `401` path); cross-origin or Origin-less upgrades with the cookie - are rejected the same way (CSWSH protection), so clients must send a matching `Origin`; - upstream bridging failures close with `1011`. +- the ingress/load balancer in front of the gateway must pass `Upgrade`/`Connection` + headers; [`manifests/exposure/ingress/60-ingress.yaml`](manifests/exposure/ingress/60-ingress.yaml) + is a starting point (Traefik handles upgrades by default), +- idle suspension is disabled for users with in-flight proxied requests or open + WebSockets, so a long-running chat stream is not cut off mid-turn, +- the upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`, + and performs the launch-token handshake under that same `pod-IP:3080` authority. + DSH fences on `Host` (only its bind address, loopback, or an allow-listed host + passes) and binds its auth cookie to the request authority + (`payload.authority === authority`), while `websockets` derives `Host` from the + URI and cannot be overridden. A browser-host cookie or forwarded origin makes + the upgrade fail (`401`/`403`, surfaced as close `1011`), which the client + retries forever as `[connection] connection lost, retry #N`, +- missing/invalid gateway auth is rejected right after the upgrade with close code + `4401` (WebSocket auth has no HTTP `401` path); cross-origin or Origin-less + upgrades with the cookie are rejected the same way (CSWSH protection), so + clients must send a matching `Origin`; upstream bridging failures close with + `1011`. ### Path routing -Each user's DSH Web is served under `/u//`. DSH injects `` and uses -relative asset/API URLs, so the UI works under a path prefix without a per-user hostname or -wildcard DNS. The gateway also rewrites absolute-path `Location` headers into `/u//…` and -redirects `/u/` to the trailing-slash form with `308` so relative URLs resolve correctly. +Each user's DSH Web is served under `/u//`. DSH injects `` +and uses relative asset/API URLs, so the UI works under a path prefix without a +per-user hostname or wildcard DNS. The gateway also rewrites absolute-path +`Location` headers into `/u//…` and redirects `/u/` to the +trailing-slash form with `308` so relative URLs resolve correctly. + +## 9. Expose it: choose exactly one variant + +The base manifests expose nothing outside the cluster (`dsh-gateway` is a +ClusterIP Service on port `8080`). Pick **one** variant — they are alternatives, +not a stack. Details, commands and the switch procedure are in the +[exposure README](manifests/exposure/README.md). + +> Apply the base resources first, then the variant **by path**. Do not use +> `kubectl apply -R`: it would apply both variants and create two paths to the +> same gateway, which is unsafe once SSO is enabled. + +### Option A — plain Ingress (gateway's own session auth) + +```bash +# Replace the dsh.example.com host and TLS Secret (or drop the tls: block for +# plain HTTP), then: +kubectl apply -f manifests/exposure/ingress/ +``` + +1. Set `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml` to the same + public host (the gateway forwards the browser's `Host`) and re-apply the + template. +2. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS: + `kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`. + (The base manifest keeps `false` so the plain-HTTP port-forward demo works.) +3. Verify: `curl -sI https:///healthz` returns `200`; sign up in a browser + at `https:///`. + +Keep the router free of buffering/compression middlewares: they break WebSocket +upgrades and streaming. + +### Option B — Gateway API + GitHub SSO (oauth2-proxy) + +`manifests/exposure/gateway-api-sso/` puts a TLS `Gateway` + `HTTPRoute` in +front of **oauth2-proxy**, which authenticates the browser against GitHub +(restricted to a configured organization) and forwards to the gateway with +`X-Forwarded-User` set. The gateway trusts that header only when +`AUTH_PROXY_HEADER` is set (SSO is **opt-in**). + +Prerequisites: + +- Gateway API CRDs and a Gateway implementation; the manifests assume + `gatewayClassName: traefik` (the CRD version is pinned as + `GATEWAY_API_VERSION` in `versions.env`). +- A GitHub OAuth App whose homepage/callback match the public FQDN + (`https:///` and `https:///oauth2/callback`). +- The oauth2-proxy credentials Secret (template: + `73-oauth2-proxy-secret.yaml.example`) and a TLS Secret for the Gateway + listener (template: `74-tls-secret.yaml.example`). + +Steps: + +```bash +# 1. Real Secrets; the committed *.yaml.example files are templates only. +kubectl -n dsh-demo create secret generic oauth2-proxy \ + --from-literal=client-id='' \ + --from-literal=client-secret='' \ + --from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')" +kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key + +# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72). + +# 3. Enable SSO on the gateway and trust the public host in the template. +kubectl -n dsh-demo set env deploy/dsh-gateway \ + AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true +# ... and set DSH_TRUSTED_HOSTS=dsh-demo.example.com in +# manifests/10-sandboxtemplate.yaml, then re-apply the template. + +# 4. Apply the variant. +kubectl apply -f manifests/exposure/gateway-api-sso/ +``` + +Verification: + +```bash +kubectl get gateway dsh-gateway -n dsh-demo -o yaml # Programmed=True +kubectl get httproute -n dsh-demo -o wide # backend oauth2-proxy:4180, Accepted +kubectl get deploy oauth2-proxy -n dsh-demo # 1/1 Ready +kubectl get endpoints oauth2-proxy -n dsh-demo # backend present +kubectl -n dsh-demo logs deploy/oauth2-proxy --tail=50 # no config errors +``` + +Browser check: open `https:///` — you are redirected to GitHub, and after +signing in as a member of the allowed organization the gateway auto-provisions +your sandbox (the first page can show `Provisioning`) and connects you. A +non-member is denied before reaching the gateway. GitHub logins are lowercase +DNS-1123 labels, so they map 1:1 onto the gateway's `user` validation. The SSO +landing page also offers **Recreate sandbox** (`POST /recreate`), which deletes +the claim — wiping the workspace PVC — and provisions a fresh one. + +Sign-out: the gateway `/logout` clears `dsh_gw_session`; oauth2-proxy's session +can be cleared at `/oauth2/sign_out`. + +> **SSO trust boundary — read this before enabling it.** After SSO is on, the +> `dsh-gateway` Service must be reachable **only** through oauth2-proxy (it is +> ClusterIP). Anyone who can reach `dsh-gateway:8080` directly can send +> `X-Forwarded-User: ` and be treated as that user. Never apply the +> Ingress variant (or any other route to the Service) while SSO is enabled, and +> do not expose the gateway Service with `kubectl port-forward` on a shared +> machine while `AUTH_PROXY_HEADER` is set. To turn SSO off, remove the variable +> (`kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER-`) and +> delete the SSO variant; the gateway falls back to bearer-token/session auth. + +### Switching variants + +Delete the variant you had before applying the other one +(`kubectl delete -f manifests/exposure/ingress/`), revert +`AUTH_PROXY_HEADER`/`COOKIE_SECURE`, and verify with +`kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to the +gateway remains. ## Troubleshooting @@ -346,36 +502,65 @@ redirects `/u/` to the trailing-slash form with `308` so relative URLs res | Console spams `[connection] connection lost, retry #N` | The upstream WS upgrade is being rejected, surfaced as close `1011`. The gateway dials the sandbox **pod IP**, forwards no `Host`/`Origin`, and handshakes under the same `pod-IP:3080` authority DSH binds its cookie to; an older gateway image, or a sandbox with an empty `status.podIPs`, hits this loop | | `manifest.webmanifest` returns `401` in the console | Browsers fetch the Web App Manifest without credentials, so the gateway's session cookie is absent and the request fails closed. The UI is unaffected; only the PWA install prompt is lost | | Signup is slow / `202` then `Waking` | Warm pool exhausted (`kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox`); scale the pool | -| `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `50-gateway.yaml`) | -| Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Upgrade the gateway to a build that honors the marker, or resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` | -| Resume crash-loops: `credentials-local: … is readable beyond its owner (mode 660)` | The pod template lacks `fsGroupChangePolicy: OnRootMismatch`, so the kubelet's default fsGroup policy rewrote `.credentials.yaml` to `0660`; DSH requires `0600` and refuses to start. New installs get the policy from the template. For an existing sandbox: suspend it, run a throwaway pod on `dsh-home-$SB` to `chmod 600 /home/node/.dsh/.credentials.yaml`, patch the sandbox's `podTemplate` with the policy, then resume. | +| `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `manifests/50-gateway.yaml`) | +| Gateway pod `ErrImagePull` / `ImagePullBackOff` (especially on kind) | The manifest uses `imagePullPolicy: IfNotPresent` with the release identity. Build with `make gateway`, retag to exactly the manifest's image, `kind load docker-image `, or set the Deployment image to your local tag / pinned digest. On a real cluster, make sure the GHCR package is readable (public or a pull secret) | +| Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` | +| Resume crash-loops: `credentials-local: … is readable beyond its owner (mode 660)` | The pod template lacks `fsGroupChangePolicy: OnRootMismatch`, so the kubelet's default fsGroup policy rewrote `.credentials.yaml` to `0660`; DSH requires `0600` and refuses to start. New installs get the policy from the template. For an existing sandbox: suspend it, run a throwaway pod on `dsh-home-$SB` to `chmod 600 /home/node/.dsh/.credentials.yaml`, patch the sandbox's `podTemplate` with the policy, then resume | +| SSO: GitHub login succeeds but the gateway asks for a token / returns `401` to `/recreate` | `AUTH_PROXY_HEADER` is not set on `deploy/dsh-gateway` (SSO is opt-in). Set `AUTH_PROXY_HEADER=X-Forwarded-User` and confirm the request actually reaches the gateway through oauth2-proxy | +| SSO: gateway treats a caller as another user / bad `X-Forwarded-User` accepted | Trust boundary violation: something other than oauth2-proxy can reach `dsh-gateway:8080`. Remove the second path (Ingress/Gateway/port-forward), keep the Service ClusterIP, and only re-enable SSO once the proxy is the sole entry point. Never put a trusted identity header in front of a service that is directly reachable | +| SSO: oauth2-proxy `cookie_secret must be 16, 24, or 32 bytes ... but is 44 bytes` | The cookie secret was generated as standard base64. Use URL-safe unpadded base64 (see `73-oauth2-proxy-secret.yaml.example`) | +| SSO: redirect loop (`ERR_TOO_MANY_REDIRECTS`) | oauth2-proxy `--redirect-url` / callback does not match the public FQDN, or TLS is not terminating at the Gateway listener; fix the OAuth App URLs and the manifest placeholders | +| SSO: DSH rejects the public host behind the proxy | Add the public hostname to `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml`; the gateway forwards the browser's `Host` | ## Limitations Deliberately **not** production code: -- **Single replica.** Idle clocks, in-flight tracking, and the handshake cache are in memory. - Running two gateway replicas would double-suspend or lose wake tracking; scale up only after - moving that state to a shared store. -- **Log scraping.** The launch token is recovered from pod logs. Log rotation, a restarted pod, - or a changed DSH log format can break the handshake until the next `401` refresh. -- **Shared provider key.** `DEEPSEEK_API_KEY` is shared by all tenants via the template. Real - multi-tenancy needs per-tenant credentials or an egress proxy. -- **No TLS in the gateway.** Terminate TLS at an Ingress/reverse proxy and set +- **Single replica.** Idle clocks, in-flight tracking, and the handshake cache + are in memory. Running two gateway replicas would double-suspend or lose wake + tracking; scale up only after moving that state to a shared store. +- **Log scraping.** The launch token is recovered from pod logs. Log rotation, a + restarted pod, or a changed DSH log format can break the handshake until the + next `401` refresh. +- **Shared provider key.** `DEEPSEEK_API_KEY` is shared by all tenants via the + template. Real multi-tenancy needs per-tenant credentials or an egress proxy. +- **No TLS in the gateway.** Terminate TLS at an Ingress or Gateway and set `COOKIE_SECURE=true`. -- **In-memory user directory.** Users are claims + annotations; there is no database, password - reset, or admin API. Tokens are stored only as SHA-256 hashes in claim annotations. -- **Bearer tokens in URLs.** Avoid putting the signup token in a URL/log; the browser flow uses - the `HttpOnly` cookie instead. +- **SSO trust is hop-based.** `AUTH_PROXY_HEADER` trusts whoever sets the header; + the only guarantee is that the gateway Service is not directly reachable. + There is no signature, mTLS, or NetworkPolicy enforcing that in this example. +- **OAuth org is the only authorization.** Any GitHub user in the configured + organization gets an auto-provisioned sandbox; there is no per-user allow-list, + quota, or admin API. +- **In-memory user directory.** Users are claims + annotations; there is no + database, password reset, or admin API. Tokens are stored only as SHA-256 + hashes in claim annotations. +- **Bearer tokens in URLs.** Avoid putting the signup token in a URL/log; the + browser flow uses the `HttpOnly` cookie instead. ## Cost model -The same economics as the upstream pattern: suspend-when-idle plus warm pools turn an -always-on instance into a sub-second cold-start pause per return-after-absence, at a fraction -of the cost. +The same economics as the upstream pattern: suspend-when-idle plus warm pools turn +an always-on instance into a sub-second cold-start pause per return-after-absence, +at a fraction of the cost. ## Cleanup ```bash +# Delete an exposure variant first if you applied one (optional; namespace +# deletion below removes everything): +kubectl delete -f manifests/exposure/ingress/ # or gateway-api-sso/ +# Then the whole example: kubectl delete namespace dsh-demo ``` + +If you created the oauth2-proxy/TLS Secrets or the GitHub OAuth App for step 9, +delete the Secrets with the namespace and revoke the OAuth App in GitHub. + +## More + +- [Gateway README](gateway/README.md) — API, configuration, auth/SSO, failure + semantics, limitations. +- [Architecture](docs/architecture.md) — container/sequence diagrams, ADRs, + security and RBAC, failure modes, evolution path. +- [Exposure README](manifests/exposure/README.md) — the two variants in detail. diff --git a/docs/architecture.md b/examples/dsh-agents-as-a-service/docs/architecture.md similarity index 90% rename from docs/architecture.md rename to examples/dsh-agents-as-a-service/docs/architecture.md index 654af9b..1da55ce 100644 --- a/docs/architecture.md +++ b/examples/dsh-agents-as-a-service/docs/architecture.md @@ -1,7 +1,7 @@ # Architecture -This document describes the browser-usable **agents-as-a-service** platform in -[`examples/dsh-agents-as-a-service/`](../examples/dsh-agents-as-a-service/): how the gateway, +This document describes the browser-usable **agents-as-a-service** platform in this example +([README](../README.md), [manifests](../manifests/), [gateway](../gateway/)): how the gateway, Kubernetes, and DSH Web fit together, and why the design looks the way it does. Scope: one Kubernetes namespace (`dsh-demo`), one shared DSH Web image, one gateway deployment, @@ -49,6 +49,24 @@ Responsibilities: | **DSH Web pod** | One `dsh web` process per user, state on the per-user PVC, listening on `3080`. | | **Warm pool** | Keeps `N` Ready sandboxes so signup is a label flip instead of a cold start. | +## Exposure variants + +The base manifests deploy the gateway as a ClusterIP Service (`dsh-gateway:8080`) and expose +nothing outside the cluster. Two **mutually exclusive** variants live under +[`manifests/exposure/`](../manifests/exposure/): + +| Variant | Entry point | Authentication | Notes | +| --- | --- | --- | --- | +| [Ingress](../manifests/exposure/ingress/60-ingress.yaml) | Traefik `Ingress` + optional TLS | the gateway's own signup/session cookie | simplest; no external IdP; set `COOKIE_SECURE=true` with HTTPS | +| [Gateway API + SSO](../manifests/exposure/gateway-api-sso/70-gateway.yaml) | Gateway API `Gateway`/`HTTPRoute` terminating TLS | GitHub OAuth via [oauth2-proxy](../manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml), identity forwarded as `X-Forwarded-User` | requires `AUTH_PROXY_HEADER` on the gateway; the trust boundary applies | + +They are alternatives, not a stack: applying both creates a second path to the same gateway +and defeats SSO. When SSO is enabled, the Service must stay ClusterIP and be reachable only +through oauth2-proxy — otherwise any client can send `X-Forwarded-User` itself and be treated +as that user. The gateway ignores the header unless `AUTH_PROXY_HEADER` is set. See the +[exposure README](../manifests/exposure/README.md) and the +[gateway README](../gateway/README.md#sso-opt-in). + ## Flows ### Signup (browser) @@ -247,7 +265,7 @@ sequenceDiagram ## Security and RBAC -The gateway runs with a namespace-scoped ServiceAccount (`50-gateway.yaml`): +The gateway runs with a namespace-scoped ServiceAccount (`manifests/50-gateway.yaml`): | API group | Resource | Verbs | Why | | --- | --- | --- | --- | diff --git a/examples/dsh-agents-as-a-service/gateway/.dockerignore b/examples/dsh-agents-as-a-service/gateway/.dockerignore new file mode 100644 index 0000000..8b20043 --- /dev/null +++ b/examples/dsh-agents-as-a-service/gateway/.dockerignore @@ -0,0 +1,11 @@ +# Runtime image only needs the app and its pinned runtime requirements. +__pycache__/ +*.py[cod] +.pytest_cache/ +.ruff_cache/ +.venv/ +test_gateway.py +requirements-dev.txt +pyproject.toml +README.md +coverage.xml diff --git a/examples/dsh-agents-as-a-service/gateway/README.md b/examples/dsh-agents-as-a-service/gateway/README.md index 625e360..2ba1287 100644 --- a/examples/dsh-agents-as-a-service/gateway/README.md +++ b/examples/dsh-agents-as-a-service/gateway/README.md @@ -4,7 +4,7 @@ A single-file, browser-usable control + data plane for [DeepSeek Harness Web (DSH)](https://github.com/deepseek-ai/deepseek-harness) sandboxes running behind the `agent-sandbox` CRDs. -It does three things: +It does four things: 1. **Users** — creates a `SandboxClaim` from a warm pool, mints a bearer token (only its SHA-256 is stored as a claim annotation), reports derived state, @@ -15,10 +15,15 @@ It does three things: 3. **Reverse proxy** — proxies HTTP (including SSE) and WebSocket traffic to the sandbox's DSH Web server on `:3080`, including DSH's own cookie handshake. +4. **Optional SSO** — when `AUTH_PROXY_HEADER` is set, trusts the identity + header an authenticating reverse proxy (for example oauth2-proxy) forwards, + auto-provisions that user, and exposes `POST /recreate` for a clean restart. -DSH is pinned to `dsh-v0.2.1-rc.2` in the example manifests. It binds `:3080`, -serves `/`, gates `/api` behind a `dsh-auth-*` cookie and requires a WebSocket -at `/api/remote.mux`. +The DSH version pin lives in [`versions.env`](../../../versions.env) +(`DSH_VERSION` plus the optional repository `RELEASE_REVISION`); the example +manifests and published images use the release identity built from it (currently +`dsh-v0.2.1-rc.2-r2`). DSH binds `:3080`, serves `/`, gates `/api` behind a +`dsh-auth-*` cookie and requires a WebSocket at `/api/remote.mux`. ## Flow @@ -58,10 +63,11 @@ token. A failed handshake returns `503` with `Retry-After: 10`. | Method | Path | Auth | Purpose | | --- | --- | --- | --- | | `GET` | `/healthz` | none | Liveness/readiness. Never touches the cluster. | -| `GET` | `/` | none | Landing page with signup + token login forms. | +| `GET` | `/` | none | Landing page with signup + token login forms. With SSO it auto-provisions the proxy-identified user and redirects to their workspace. | | `POST` | `/users` | none | Signup. JSON body `{"user": ""}` or form field `user`. JSON returns `201` when Ready, `202` when accepted but still Waking; form returns `303` to `/u//`. Both set the session cookie. `409` if the user exists, `400` if invalid, `413` if the body is too large. | | `POST` | `/login` | none | Form (`user`, `token`) returns `303` to `/u//`; JSON returns `200`. Sets the session cookie. `401` on a bad token, `404` for an unknown user. | | `POST` | `/logout` | none | Clears the session cookie and `303`s to `/`. | +| `POST` | `/recreate` | trusted proxy header | **SSO only.** Deletes the authenticated user's claim (cascading sandbox + PVC, so workspace data is wiped), waits out finalizers, provisions a fresh claim and `303`s to `/u//`. `401` without a valid identity header; cookie-authenticated calls must be same-origin. | | `GET` | `/users/{user}` | bearer or cookie | Derived state: `Provisioning`, `Waking`, `Ready`, `Suspending`, `Suspended`. | | `DELETE` | `/users/{user}` | bearer or cookie | Delete the claim (sandbox + PVC cascade). Cookie-auth mutations require a same-origin `Origin`/`Referer`. | | `GET` | `/u/{user}` | — | `308` redirect to `/u/{user}/`. | @@ -86,6 +92,37 @@ token. A failed handshake returns `503` with `Retry-After: 10`. absolute paths are rewritten under `/u//`, and any `token` query parameter is stripped. +### SSO (opt-in) + +By default the gateway is the only authenticator: users sign up with a token or +the session cookie, and the gateway ignores all identity headers. Set +`AUTH_PROXY_HEADER` to enable SSO behind an authenticating reverse proxy such as +oauth2-proxy: + +```bash +kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER=X-Forwarded-User +``` + +When set, the gateway treats a request carrying that header as authenticated as +that user, provided the value is a valid DNS-1123 label; the header is never +trusted when the variable is unset. The landing page then auto-provisions the +user (creates their claim idempotently, mints a rotated session token) and +redirects to `/u//`, and `POST /recreate` becomes available for a clean +restart. The base manifests leave it unset on purpose; the +[SSO exposure variant](../manifests/exposure/gateway-api-sso/) documents how to +enable it on the Deployment (`kubectl set env`). + +> **Trust boundary — this is the security-critical part.** A trusted identity +> header is only safe when **nothing else can reach the gateway**. The gateway +> Service must stay ClusterIP and be reachable exclusively through the +> authenticating proxy; if a client can reach `dsh-gateway:8080` directly (a +> second Ingress/Gateway/Route, a port-forward on a shared host, a broad +> NetworkPolicy), it can send `X-Forwarded-User` itself and impersonate any user. +> Treat `AUTH_PROXY_HEADER` as "the proxy already authenticated this request", +> not as an authentication mechanism of its own. Disable SSO by removing the +> variable: the header is then ignored and bearer-token/session auth applies +> again. + ### Proxy failure semantics | Situation | Result | @@ -123,6 +160,7 @@ can also be injected into `create_app` for tests. | `MAX_BODY_MB` | `32` | Maximum buffered request body (and `413` threshold). | | `HTTP_TIMEOUT` | `300` | Upstream connect/read timeout in seconds. Proxied streams (SSE, long responses) keep the connect/write budget but have **no read timeout**, so an idle event stream is not cut every five minutes. | | `SWEEP_INTERVAL` | `15` | Seconds between idle sweeps. | +| `AUTH_PROXY_HEADER` | unset | Opt-in SSO: name of the trusted identity header set by an authenticating reverse proxy (e.g. `X-Forwarded-User`). Unset disables SSO and the header is never trusted. Enable only when the Service is reachable exclusively through that proxy (see [SSO](#sso-opt-in)). | ### Required RBAC @@ -147,12 +185,21 @@ python -m ruff check . uvicorn gateway:create_app --factory --host 0.0.0.0 --port 8080 ``` -Container image: +Container image, from the repository root: ```bash -docker build -t dsh-gateway examples/dsh-agents-as-a-service/gateway +make gateway # builds dsh-aaas-gateway:dev from this directory ``` +The equivalent direct build is +`docker build -t dsh-aaas-gateway:dev examples/dsh-agents-as-a-service/gateway`. +For kind, retag it to the image reference in +`examples/dsh-agents-as-a-service/manifests/50-gateway.yaml` and `kind load` it. +Published images are nested below the main image as +`/dsh-agents-as-a-service/gateway:`, where the identity +comes from [`versions.env`](../../../versions.env) — see +[`docs/versioning.md`](../../../docs/versioning.md). + The image runs as UID `10000`, sets `PYTHONDONTWRITEBYTECODE=1` and `HOME=/tmp` (read-only-rootfs compatible), listens on `8080`, and disables uvicorn's access log so bearer tokens in URLs are not logged. @@ -182,3 +229,8 @@ uvicorn's access log so bearer tokens in URLs are not logged. re-runs the handshake once and replays the request. * **Only DSH on `:3080` is proxied.** There is no dashboard/API-port split and no platform API key injection. +* **SSO trusts a header.** With `AUTH_PROXY_HEADER` set, any client that can + reach the gateway Service directly can spoof the identity header. The gateway + cannot enforce the trust boundary itself: keep the Service ClusterIP and + reachable only through the authenticating proxy. There is no mTLS or signed + identity assertion. diff --git a/examples/dsh-agents-as-a-service/gateway/gateway.py b/examples/dsh-agents-as-a-service/gateway/gateway.py index 71b5dba..27b6ba8 100644 --- a/examples/dsh-agents-as-a-service/gateway/gateway.py +++ b/examples/dsh-agents-as-a-service/gateway/gateway.py @@ -195,9 +195,7 @@ def derive_state(sandbox: dict | None) -> str: def token_matches(claim: dict, token: str | None) -> bool: """Timing-safe check of a user token against the stored SHA-256.""" - want = (claim.get("metadata", {}).get("annotations") or {}).get( - TOKEN_ANNOTATION, "" - ) + want = (claim.get("metadata", {}).get("annotations") or {}).get(TOKEN_ANNOTATION, "") if not token or not want: return False got = hashlib.sha256(token.encode()).hexdigest() @@ -518,9 +516,7 @@ class Kubernetes: async def get_pod(self, name: str) -> dict | None: try: - pod = await self._call( - "core", "read_namespaced_pod", name, self.namespace - ) + pod = await self._call("core", "read_namespaced_pod", name, self.namespace) except ApiException as exc: if exc.status == 404: return None @@ -530,9 +526,7 @@ class Kubernetes: # so callers can keep using ``pod.get(...)``. return k8s_client.ApiClient().sanitize_for_serialization(pod) - async def read_pod_logs( - self, name: str, container: str, tail_lines: int - ) -> str: + async def read_pod_logs(self, name: str, container: str, tail_lines: int) -> str: result = await self._call( "core", "read_namespaced_pod_log", @@ -616,9 +610,7 @@ class Gateway: return False # a session for another user never authorizes this one return token_matches(claim, token) - async def _sso_session( - self, remote: str, headers, cookies, response: Response - ) -> dict | None: + async def _sso_session(self, remote: str, headers, cookies, response: Response) -> dict | None: """Provision ``remote`` idempotently and mint/refresh a session token. The authenticating proxy has already verified ``remote``, so it is @@ -682,9 +674,7 @@ class Gateway: while True: async with self.state.lock: if not self.state.maintenance.get(user): - self.state.in_flight[user] = ( - self.state.in_flight.get(user, 0) + 1 - ) + self.state.in_flight[user] = self.state.in_flight.get(user, 0) + 1 self.state.last_activity[user] = time.time() return True if time.monotonic() > deadline: @@ -693,9 +683,7 @@ class Gateway: async def _release(self, user: str) -> None: async with self.state.lock: - self.state.in_flight[user] = max( - self.state.in_flight.get(user, 1) - 1, 0 - ) + self.state.in_flight[user] = max(self.state.in_flight.get(user, 1) - 1, 0) self.state.last_activity[user] = time.time() # -- sandbox / handshake helpers -------------------------------------- @@ -724,9 +712,7 @@ class Gateway: # Wake-on-connect: safe in ANY state, including mid-suspension. self._clear_handshake(user) try: - await self.k8s.patch_operating_mode( - sandbox["metadata"]["name"], "Running" - ) + await self.k8s.patch_operating_mode(sandbox["metadata"]["name"], "Running") except Exception as exc: raise SandboxUnavailable("could not wake the sandbox") from exc sandbox = await self.wait_ready(user, self.settings.wake_timeout) @@ -740,10 +726,7 @@ class Gateway: def _pod_name(self, sandbox: dict) -> str: annotations = sandbox.get("metadata", {}).get("annotations") or {} - return ( - annotations.get(POD_NAME_ANNOTATION) - or sandbox["metadata"]["name"] - ) + return annotations.get(POD_NAME_ANNOTATION) or sandbox["metadata"]["name"] @staticmethod def _pod_ip(sandbox: dict) -> str | None: @@ -787,16 +770,11 @@ class Gateway: self.settings.dsh_log_tail_lines, ) except ApiException as exc: - raise HandshakeError( - f"could not read logs for {pod_name}: {exc}" - ) from exc + raise HandshakeError(f"could not read logs for {pod_name}: {exc}") from exc launch_token = parse_dsh_launch_token(logs) if not launch_token: raise HandshakeError("DSH launch token not found in pod logs") - url = ( - f"http://{fqdn}:{self.settings.dsh_port}/" - f"?token={quote(launch_token, safe='')}" - ) + url = f"http://{fqdn}:{self.settings.dsh_port}/?token={quote(launch_token, safe='')}" headers = { "Host": host, "X-Forwarded-Host": host, @@ -812,9 +790,7 @@ class Gateway: except httpx.HTTPError as exc: raise HandshakeError(f"DSH handshake failed: {exc}") from exc if response.status_code >= 400: - raise HandshakeError( - f"DSH handshake returned {response.status_code}" - ) + raise HandshakeError(f"DSH handshake returned {response.status_code}") cookie = extract_dsh_cookies(response.headers.get_list("set-cookie")) if not cookie: raise HandshakeError("DSH handshake set no dsh-auth cookie") @@ -856,7 +832,7 @@ class Gateway: else: status = html.escape(state) confirm = ( - 'onsubmit="return confirm(\'Delete this sandbox and start fresh? ' + "onsubmit=\"return confirm('Delete this sandbox and start fresh? " "Your workspace data will be wiped.')\"" ) body = ( @@ -887,9 +863,7 @@ class Gateway: # session cookie minted by _sso_session; the real page is rendered # once the sandbox state is known. response = HTMLResponse() - claim = await self._sso_session( - remote, request.headers, request.cookies, response - ) + claim = await self._sso_session(remote, request.headers, request.cookies, response) sandbox = await self._sandbox_of(claim) if claim else None page = HTMLResponse(self._sso_page(remote, derive_state(sandbox))) if "set-cookie" in response.headers: @@ -953,14 +927,10 @@ against the JSON API. The browser session is an HttpOnly cookie.

@staticmethod def _wants_form(request: Request) -> bool: ctype = request.headers.get("content-type", "").lower() - return ctype.startswith( - ("application/x-www-form-urlencoded", "multipart/form-data") - ) + return ctype.startswith(("application/x-www-form-urlencoded", "multipart/form-data")) async def _read_payload(self, request: Request) -> dict: - if request.headers.get("content-type", "").lower().startswith( - "application/json" - ): + if request.headers.get("content-type", "").lower().startswith("application/json"): data = await self._read_body(request) try: payload = json.loads(data) @@ -997,9 +967,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

return self._error(413, "request body too large") user = payload.get("user", "") if not valid_user(user): - return self._error( - 400, "body must be {'user': ''}" - ) + return self._error(400, "body must be {'user': ''}") if await self.k8s.get_claim(user) is not None: return self._error(409, "user exists") token = secrets.token_urlsafe(32) @@ -1059,9 +1027,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

if claim is None: return self._error(404, "not found") token, cookie_user = self.credentials(request.headers, request.cookies) - if not self._authorized( - claim, token, cookie_user, user, self.remote_user(request.headers) - ): + if not self._authorized(claim, token, cookie_user, user, self.remote_user(request.headers)): return self._error(401, "unauthorized") sandbox = await self._sandbox_of(claim) return JSONResponse( @@ -1080,9 +1046,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

if claim is None: return self._error(404, "not found") token, cookie_user = self.credentials(request.headers, request.cookies) - if not self._authorized( - claim, token, cookie_user, user, self.remote_user(request.headers) - ): + if not self._authorized(claim, token, cookie_user, user, self.remote_user(request.headers)): return self._error(401, "unauthorized") if self._mutation_from_cookie(request.headers, "DELETE", cookie_user): return self._error(403, "cross-origin request rejected") @@ -1091,9 +1055,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

async with self.state.lock: self.state.last_activity.pop(user, None) self.state.in_flight.pop(user, None) - return JSONResponse( - {"user": user, "note": "claim deleted; sandbox and PVC cascade"} - ) + return JSONResponse({"user": user, "note": "claim deleted; sandbox and PVC cascade"}) async def recreate(self, request: Request) -> Response: """Delete and re-provision the SSO user's sandbox for a clean start. @@ -1122,9 +1084,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

break await asyncio.sleep(1) response = RedirectResponse(f"/u/{remote}/", status_code=303) - await self._sso_session( - remote, request.headers, request.cookies, response - ) + await self._sso_session(remote, request.headers, request.cookies, response) return response # -- HTTP proxy -------------------------------------------------------- @@ -1141,9 +1101,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

return RedirectResponse("/", status_code=303) return self._error(404, "unknown user") token, cookie_user = self.credentials(request.headers, request.cookies) - if remote == user and not ( - cookie_user == user and token_matches(claim, token) - ): + if remote == user and not (cookie_user == user and token_matches(claim, token)): # SSO user without a valid session cookie: bootstrap, then continue. return RedirectResponse("/", status_code=303) if not self._authorized(claim, token, cookie_user, user, remote): @@ -1152,31 +1110,21 @@ against the JSON API. The browser session is an HttpOnly cookie.

return self._error(403, "cross-origin request rejected") if not await self._acquire(user): - return self._error( - 503, "gateway busy, retry shortly", {"Retry-After": "5"} - ) + return self._error(503, "gateway busy, retry shortly", {"Retry-After": "5"}) streaming = False try: sandbox, fqdn = await self._resolve_sandbox(user, claim) host = request.headers.get("host", "") - cookie, _, was_cached = await self.dsh_cookie( - user, sandbox, fqdn, host - ) + cookie, _, was_cached = await self.dsh_cookie(user, sandbox, fqdn, host) body = await self._read_body(request) - upstream = await self._send_upstream( - request, user, fqdn, cookie, body - ) + upstream = await self._send_upstream(request, user, fqdn, cookie, body) if upstream.status_code == 401 and was_cached: # The cached DSH cookie expired (pod restart, logout, ...): # handshake once more and replay the (already buffered) request. await upstream.aclose() - cookie, _, _ = await self.dsh_cookie( - user, sandbox, fqdn, host, force=True - ) - upstream = await self._send_upstream( - request, user, fqdn, cookie, body - ) + cookie, _, _ = await self.dsh_cookie(user, sandbox, fqdn, host, force=True) + upstream = await self._send_upstream(request, user, fqdn, cookie, body) response = StreamingResponse( self._relay(user, upstream), @@ -1216,8 +1164,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

headers = { key: value for key, value in request.headers.items() - if key.lower() not in OWNED_REQUEST_HEADERS - and key.lower() != "content-length" + if key.lower() not in OWNED_REQUEST_HEADERS and key.lower() != "content-length" } host = request.headers.get("host", "") headers["host"] = host @@ -1229,9 +1176,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

else: headers["x-forwarded-for"] = "" port = host.rsplit(":", 1)[1] if ":" in host else "" - headers["x-forwarded-port"] = port or ( - "443" if request.url.scheme == "https" else "80" - ) + headers["x-forwarded-port"] = port or ("443" if request.url.scheme == "https" else "80") last_error: Exception | None = None for attempt in range(self.settings.upstream_attempts): @@ -1341,9 +1286,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

if ":" in upstream_host and not upstream_host.startswith("["): upstream_host = f"[{upstream_host}]" # IPv6 literal authority = f"{upstream_host}:{self.settings.dsh_port}" - cookie, _, _ = await self.dsh_cookie( - user, sandbox, fqdn, authority - ) + cookie, _, _ = await self.dsh_cookie(user, sandbox, fqdn, authority) url = f"ws://{authority}/{path}" query = filtered_query(websocket.url.query) if query: @@ -1362,9 +1305,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

headers["x-forwarded-for"] = websocket.client.host subprotocols = [ part.strip() - for part in websocket.headers.get( - "sec-websocket-protocol", "" - ).split(",") + for part in websocket.headers.get("sec-websocket-protocol", "").split(",") if part.strip() ] upstream_cm = self.ws_connect( @@ -1380,16 +1321,12 @@ against the JSON API. The browser session is an HttpOnly cookie.

return await self._ws_reject(websocket, 1011) try: - await websocket.accept( - subprotocol=getattr(upstream, "subprotocol", None) - ) + await websocket.accept(subprotocol=getattr(upstream, "subprotocol", None)) except RuntimeError: pass await self._pump(websocket, upstream) with contextlib.suppress(Exception): - await websocket.close( - code=self._upstream_close_code(upstream) - ) + await websocket.close(code=self._upstream_close_code(upstream)) finally: if upstream_cm is not None: with contextlib.suppress(Exception): @@ -1429,9 +1366,7 @@ against the JSON API. The browser session is an HttpOnly cookie.

asyncio.create_task(client_to_upstream()), asyncio.create_task(upstream_to_client()), ] - done, pending = await asyncio.wait( - tasks, return_when=asyncio.FIRST_COMPLETED - ) + done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) for task in pending: task.cancel() await asyncio.gather(*done, *pending, return_exceptions=True) @@ -1458,17 +1393,13 @@ against the JSON API. The browser session is an HttpOnly cookie.

continue if self.state.in_flight.get(user, 0) > 0: continue # requests in flight => the user is active - idle = time.time() - self.state.last_activity.setdefault( - user, time.time() - ) + idle = time.time() - self.state.last_activity.setdefault(user, time.time()) if idle < self.settings.idle_timeout: continue self.state.maintenance[user] = True try: print(f"suspending {user} (idle {int(idle)}s)", flush=True) - await self.k8s.patch_operating_mode( - sandbox["metadata"]["name"], "Suspended" - ) + await self.k8s.patch_operating_mode(sandbox["metadata"]["name"], "Suspended") self._clear_handshake(user) finally: async with self.state.lock: @@ -1519,12 +1450,8 @@ def create_app( k8s = k8s if k8s is not None else Kubernetes(settings.namespace) owns_http_client = http_client is None if http_client is None: - http_client = httpx.AsyncClient( - follow_redirects=False, timeout=settings.http_timeout - ) - gateway = Gateway( - settings, k8s, http_client, ws_connect or websockets.connect - ) + http_client = httpx.AsyncClient(follow_redirects=False, timeout=settings.http_timeout) + gateway = Gateway(settings, k8s, http_client, ws_connect or websockets.connect) routes = [ Route("/healthz", gateway.healthz), Route("/", gateway.landing), diff --git a/examples/dsh-agents-as-a-service/gateway/requirements-dev.txt b/examples/dsh-agents-as-a-service/gateway/requirements-dev.txt index 5c51b9d..a3d3260 100644 --- a/examples/dsh-agents-as-a-service/gateway/requirements-dev.txt +++ b/examples/dsh-agents-as-a-service/gateway/requirements-dev.txt @@ -1,4 +1,5 @@ -r requirements.txt pytest==9.1.1 pytest-asyncio==1.4.0 +pytest-cov==7.1.0 ruff==0.17.1 diff --git a/examples/dsh-agents-as-a-service/gateway/test_gateway.py b/examples/dsh-agents-as-a-service/gateway/test_gateway.py index f3db972..e10f643 100644 --- a/examples/dsh-agents-as-a-service/gateway/test_gateway.py +++ b/examples/dsh-agents-as-a-service/gateway/test_gateway.py @@ -45,9 +45,7 @@ FQDN = "dsh-alice-sbx.test-ns.svc.cluster.local" def dsh_response(status=200, content=b"", headers=None): """A MockTransport response that is a real stream (``aiter_raw`` works).""" - return httpx.Response( - status, headers=headers, stream=httpx.ByteStream(content) - ) + return httpx.Response(status, headers=headers, stream=httpx.ByteStream(content)) # --- fakes ----------------------------------------------------------------- @@ -99,9 +97,7 @@ class FakeK8s: self.token_patches.append((user, token_hash)) claim = self.claims.get(user) if claim is not None: - annotations = claim.setdefault("metadata", {}).setdefault( - "annotations", {} - ) + annotations = claim.setdefault("metadata", {}).setdefault("annotations", {}) annotations["aaas.example.com/token-sha256"] = token_hash async def get_pod(self, name): @@ -118,9 +114,7 @@ class FakeUpstream: self.requests = [] self.handshake_requests = [] self.handshake_status = 303 - self.handshake_cookies = [ - "dsh-auth-a=one; Path=/; HttpOnly; SameSite=Strict" - ] + self.handshake_cookies = ["dsh-auth-a=one; Path=/; HttpOnly; SameSite=Strict"] self.proxy_handler = None self.proxy_requests = [] @@ -140,9 +134,7 @@ class FakeUpstream: self.proxy_requests.append(request) if self.proxy_handler is not None: return self.proxy_handler(request) - return dsh_response( - 200, b"hello from DSH", {"content-type": "text/plain"} - ) + return dsh_response(200, b"hello from DSH", {"content-type": "text/plain"}) class FakeWS: @@ -207,9 +199,7 @@ def add_user(k8s, user="alice", token=TOKEN, ready=True, pod_name=None, suffix=" "metadata": { "name": claim_name(user), "annotations": { - "aaas.example.com/token-sha256": hashlib.sha256( - token.encode() - ).hexdigest() + "aaas.example.com/token-sha256": hashlib.sha256(token.encode()).hexdigest() }, }, "status": {"sandbox": {"name": sandbox_name}}, @@ -224,10 +214,7 @@ def add_user(k8s, user="alice", token=TOKEN, ready=True, pod_name=None, suffix=" }, } k8s.pods[pod_name] = {"metadata": {"name": pod_name, "uid": POD_UID}} - k8s.logs[pod_name] = ( - "booting\n" - f"dsh web: http://10.0.0.7:3080/?token={TOKEN}-launch\n" - ) + k8s.logs[pod_name] = f"booting\ndsh web: http://10.0.0.7:3080/?token={TOKEN}-launch\n" return sandbox_name, pod_name @@ -243,9 +230,7 @@ def env(): ) k8s = FakeK8s() upstream = FakeUpstream() - http_client = httpx.AsyncClient( - transport=httpx.MockTransport(upstream), timeout=5 - ) + http_client = httpx.AsyncClient(transport=httpx.MockTransport(upstream), timeout=5) ws_connect = FakeWSConnect() app = create_app( settings=settings, @@ -275,9 +260,7 @@ def asgi_client(env, cookies=None): def stale_handshake(env): - env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = ( - "dsh-auth-a=stale" - ) + env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = "dsh-auth-a=stale" # --- pure helpers ---------------------------------------------------------- @@ -419,9 +402,7 @@ def test_token_matches_is_timing_safe_hash_check(): claim = { "metadata": { "annotations": { - "aaas.example.com/token-sha256": hashlib.sha256( - TOKEN.encode() - ).hexdigest() + "aaas.example.com/token-sha256": hashlib.sha256(TOKEN.encode()).hexdigest() } } } @@ -486,14 +467,9 @@ async def test_kubernetes_create_claim_body(monkeypatch): body = args[4] assert body["metadata"]["name"] == "dsh-alice" assert body["metadata"]["annotations"]["aaas.example.com/token-sha256"] == "hash" - assert ( - body["metadata"]["annotations"]["aaas.example.com/managed-by"] - == "dsh-gateway" - ) + assert body["metadata"]["annotations"]["aaas.example.com/managed-by"] == "dsh-gateway" assert body["spec"]["warmPoolRef"]["name"] == "test-pool" - assert body["spec"]["additionalPodMetadata"]["labels"][ - "sandbox.users.io/dsh-user" - ] == "alice" + assert body["spec"]["additionalPodMetadata"]["labels"]["sandbox.users.io/dsh-user"] == "alice" @pytest.mark.asyncio @@ -625,9 +601,7 @@ async def test_landing_page_has_signup_and_login_forms(env): @pytest.mark.asyncio async def test_landing_page_shows_session_when_cookie_present(env): - async with asgi_client( - env, cookies={"dsh_gw_session": f"alice:{TOKEN}"} - ) as client: + async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: response = await client.get("/") assert "Signed in as" in response.text assert "/u/alice/" in response.text @@ -645,9 +619,7 @@ async def test_workspace_redirect_adds_trailing_slash(env): @pytest.mark.asyncio -@pytest.mark.parametrize( - "bad_user", ["Alice", "-alice", "alice-", "", "a_b", 123, None, "a" * 60] -) +@pytest.mark.parametrize("bad_user", ["Alice", "-alice", "alice-", "", "a_b", 123, None, "a" * 60]) async def test_signup_rejects_invalid_dns1123_label(env, bad_user): async with asgi_client(env) as client: response = await client.post("/users", json={"user": bad_user}) @@ -781,9 +753,7 @@ async def test_login_json_and_failures(env): @pytest.mark.asyncio async def test_logout_clears_cookie(env): - async with asgi_client( - env, cookies={"dsh_gw_session": f"alice:{TOKEN}"} - ) as client: + async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: response = await client.post("/logout") assert response.status_code == 303 assert response.headers["location"] == "/" @@ -798,12 +768,8 @@ async def test_logout_clears_cookie(env): async def test_get_user_requires_valid_bearer_or_cookie(env): add_user(env.k8s) async with asgi_client(env) as client: - forbidden = await client.get( - "/users/alice", headers={"Authorization": "Bearer wrong"} - ) - bearer = await client.get( - "/users/alice", headers={"Authorization": f"Bearer {TOKEN}"} - ) + forbidden = await client.get("/users/alice", headers={"Authorization": "Bearer wrong"}) + bearer = await client.get("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}) cookie = await client.get( "/users/alice", headers={"Cookie": f"dsh_gw_session=alice:{TOKEN}"}, @@ -839,9 +805,7 @@ async def test_invalid_user_in_path_is_404(env, path): async def test_delete_user_requires_auth(env): add_user(env.k8s) async with asgi_client(env) as client: - response = await client.delete( - "/users/alice", headers={"Authorization": "Bearer wrong"} - ) + response = await client.delete("/users/alice", headers={"Authorization": "Bearer wrong"}) assert response.status_code == 401 assert env.k8s.deleted == [] @@ -853,12 +817,8 @@ async def test_delete_with_cookie_requires_same_origin(env): cookies = {"dsh_gw_session": f"alice:{TOKEN}"} async with asgi_client(env, cookies=cookies) as client: missing = await client.delete("/users/alice") - cross = await client.delete( - "/users/alice", headers={"Origin": "http://evil.test"} - ) - good = await client.delete( - "/users/alice", headers={"Origin": "http://gateway.test"} - ) + cross = await client.delete("/users/alice", headers={"Origin": "http://evil.test"}) + good = await client.delete("/users/alice", headers={"Origin": "http://gateway.test"}) assert missing.status_code == 403 assert cross.status_code == 403 assert good.status_code == 200 @@ -870,9 +830,7 @@ async def test_delete_with_cookie_requires_same_origin(env): async def test_delete_with_bearer_does_not_need_origin(env): add_user(env.k8s) async with asgi_client(env) as client: - response = await client.delete( - "/users/alice", headers={"Authorization": f"Bearer {TOKEN}"} - ) + response = await client.delete("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}) assert response.status_code == 200 @@ -881,9 +839,7 @@ async def test_delete_missing_claim_is_404(env): add_user(env.k8s) env.k8s.claims.clear() async with asgi_client(env) as client: - response = await client.delete( - "/users/alice", headers={"Authorization": f"Bearer {TOKEN}"} - ) + response = await client.delete("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}) assert response.status_code == 404 @@ -958,9 +914,9 @@ async def test_missing_cookie_in_handshake_is_503(env): @pytest.mark.asyncio async def test_pod_annotation_overrides_sandbox_name(env): sandbox_name, pod_name = add_user(env.k8s) - env.k8s.sandboxes[sandbox_name]["metadata"]["annotations"][ - "agents.x-k8s.io/pod-name" - ] = "custom-pod" + env.k8s.sandboxes[sandbox_name]["metadata"]["annotations"]["agents.x-k8s.io/pod-name"] = ( + "custom-pod" + ) env.k8s.pods["custom-pod"] = {"metadata": {"uid": "other-uid"}} env.k8s.logs["custom-pod"] = env.k8s.logs[pod_name] async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: @@ -1082,9 +1038,7 @@ async def test_proxy_upstream_connection_error_is_502(env): @pytest.mark.asyncio async def test_proxy_body_limit_returns_413(env): add_user(env.k8s) - env.gateway.settings = Settings( - **{**env.settings.__dict__, "max_body_bytes": 4} - ) + env.gateway.settings = Settings(**{**env.settings.__dict__, "max_body_bytes": 4}) async with asgi_client(env) as client: response = await client.post( "/u/alice/api/echo", @@ -1194,9 +1148,7 @@ async def test_proxied_requests_use_the_stream_timeout(env, monkeypatch): "server": ("gateway.test", 80), "path_params": {"user": "alice", "path": "events"}, } - response = await env.gateway._send_upstream( - Request(scope), "alice", FQDN, "dsh-auth-a=1", b"" - ) + response = await env.gateway._send_upstream(Request(scope), "alice", FQDN, "dsh-auth-a=1", b"") await response.aclose() assert captured["timeout"].read is None @@ -1359,9 +1311,7 @@ async def test_sweeper_skips_claim_it_does_not_own(env): assert "alice" not in env.gateway.state.last_activity # ...and does not suspend it even when it looks long-idle. - env.gateway.state.last_activity["alice"] = ( - time.time() - env.settings.idle_timeout - 10 - ) + env.gateway.state.last_activity["alice"] = time.time() - env.settings.idle_timeout - 10 await env.gateway.sweep_once() assert env.k8s.patches == [] @@ -1370,9 +1320,7 @@ async def test_sweeper_skips_claim_it_does_not_own(env): async def test_sweeper_suspends_idle_ready_sandbox(env): env.k8s.claims["alice"] = ready_claim() env.k8s.sandboxes["dsh-alice-sbx"] = ready_sandbox() - env.gateway.state.last_activity["alice"] = ( - time.time() - env.settings.idle_timeout - 10 - ) + env.gateway.state.last_activity["alice"] = time.time() - env.settings.idle_timeout - 10 env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = "dsh-auth-a=x" await env.gateway.sweep_once() @@ -1478,9 +1426,7 @@ async def test_sso_landing_new_user_shows_provisioning(env): async def test_sso_landing_ready_connects_and_offers_recreate_only(env): enable_sso(env) add_user(env.k8s) - async with asgi_client( - env, cookies={"dsh_gw_session": f"alice:{TOKEN}"} - ) as client: + async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: response = await client.get("/", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 200 assert "Signed in as alice" in response.text @@ -1512,9 +1458,7 @@ async def test_recreate_deletes_and_reprovisions(env): enable_sso(env) add_user(env.k8s) async with asgi_client(env) as client: - response = await client.post( - "/recreate", headers={"X-Forwarded-User": "alice"} - ) + response = await client.post("/recreate", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 303 assert response.headers["location"] == "/u/alice/" assert env.k8s.deleted == ["alice"] @@ -1526,9 +1470,7 @@ async def test_recreate_deletes_and_reprovisions(env): async def test_recreate_requires_sso(env): add_user(env.k8s) async with asgi_client(env) as client: - response = await client.post( - "/recreate", headers={"X-Forwarded-User": "alice"} - ) + response = await client.post("/recreate", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 401 assert env.k8s.deleted == [] @@ -1537,9 +1479,7 @@ async def test_recreate_requires_sso(env): async def test_recreate_rejects_cross_origin(env): enable_sso(env) add_user(env.k8s) - async with asgi_client( - env, cookies={"dsh_gw_session": f"alice:{TOKEN}"} - ) as client: + async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: response = await client.post( "/recreate", headers={"X-Forwarded-User": "alice", "Origin": "http://evil.test"}, @@ -1553,9 +1493,7 @@ async def test_sso_get_user_authorizes_via_header(env): enable_sso(env) add_user(env.k8s) async with asgi_client(env) as client: - response = await client.get( - "/users/alice", headers={"X-Forwarded-User": "alice"} - ) + response = await client.get("/users/alice", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 200 assert response.json()["user"] == "alice" @@ -1565,9 +1503,7 @@ async def test_sso_proxy_redirects_to_bootstrap_without_session(env): enable_sso(env) add_user(env.k8s) async with asgi_client(env) as client: - response = await client.get( - "/u/alice/", headers={"X-Forwarded-User": "alice"} - ) + response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 303 assert response.headers["location"] == "/" @@ -1576,9 +1512,7 @@ async def test_sso_proxy_redirects_to_bootstrap_without_session(env): async def test_sso_proxy_redirects_to_bootstrap_when_claim_missing(env): enable_sso(env) async with asgi_client(env) as client: - response = await client.get( - "/u/alice/", headers={"X-Forwarded-User": "alice"} - ) + response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 303 assert response.headers["location"] == "/" @@ -1587,7 +1521,5 @@ async def test_sso_proxy_redirects_to_bootstrap_when_claim_missing(env): async def test_spoofed_header_ignored_when_sso_disabled(env): add_user(env.k8s) async with asgi_client(env) as client: - response = await client.get( - "/u/alice/", headers={"X-Forwarded-User": "alice"} - ) + response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"}) assert response.status_code == 401 diff --git a/examples/dsh-agents-as-a-service/00-prereqs.yaml b/examples/dsh-agents-as-a-service/manifests/00-prereqs.yaml similarity index 95% rename from examples/dsh-agents-as-a-service/00-prereqs.yaml rename to examples/dsh-agents-as-a-service/manifests/00-prereqs.yaml index 7d82f77..141d6ca 100644 --- a/examples/dsh-agents-as-a-service/00-prereqs.yaml +++ b/examples/dsh-agents-as-a-service/manifests/00-prereqs.yaml @@ -5,7 +5,7 @@ # on the SandboxTemplate, stamped before any user exists — a warm-pool # constraint), so a real key committed here would be shared by every install # of this example. Generate a fresh value per install (README step 1; -# run-test-kind.sh uses a throwaway): +# scripts/e2e-kind.sh uses a throwaway): # # kubectl -n dsh-demo create secret generic dsh-provider-keys \ # --from-literal=DEEPSEEK_API_KEY="sk-..." diff --git a/examples/dsh-agents-as-a-service/10-sandboxtemplate.yaml b/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml similarity index 88% rename from examples/dsh-agents-as-a-service/10-sandboxtemplate.yaml rename to examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml index e44ac78..6cf52ff 100644 --- a/examples/dsh-agents-as-a-service/10-sandboxtemplate.yaml +++ b/examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml @@ -2,6 +2,9 @@ # exists, so per-user secrets cannot live in pod env; the shared provider key # is injected via envFrom, and claims that would bypass the warm pool are # rejected outright (see 40-claim-rejected.yaml). +# +# A stripped-down single-tenant variant of this template (no warm pool, no +# policy enforcement) lives in examples/minimal-single-tenant/. apiVersion: extensions.agents.x-k8s.io/v1beta1 kind: SandboxTemplate metadata: @@ -18,8 +21,8 @@ spec: # The gateway routes each user to their DSH Web instance by this name. service: true - # No gateway pod scopes ingress in this standalone demo, so the - # NetworkPolicy is left unmanaged. A production platform would use + # The example gateway does not manage NetworkPolicy, so it is left + # unmanaged here. A production platform would use # networkPolicyManagement: Managed and restrict ingress to its gateway on # :3080 — with the shared provider key, that NetworkPolicy is the per-user # isolation boundary. @@ -41,11 +44,12 @@ spec: # with 0660 and crash-loops before the gateway can proxy to it. # OnRootMismatch skips that walk once the volume root already carries # the fsGroup ownership/mode applied on the first mount. - # Keep in sync with deploy/k8s/sandbox-template.yaml. + # Keep in sync with the minimal variant: + # examples/minimal-single-tenant/manifests/sandbox-template.yaml. fsGroupChangePolicy: OnRootMismatch containers: - name: dsh-web - image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 + image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 imagePullPolicy: IfNotPresent ports: - name: web @@ -61,8 +65,10 @@ spec: value: "" # DSH Host/Origin-fences /api. The gateway forwards the browser's # Host, so DSH must trust the public gateway host. Empty is right - # for port-forward/localhost; for the optional Ingress set this to - # the same host as 60-ingress.yaml (e.g. dsh.example.com). + # for port-forward/localhost; for either exposure variant set this + # to the public host (e.g. dsh.example.com for + # manifests/exposure/ingress/60-ingress.yaml, or + # dsh-demo.example.com for the gateway-api-sso/ HTTPRoute). - name: DSH_TRUSTED_HOSTS value: "" # Probes target /favicon.svg: DSH gates / behind its launch-token diff --git a/examples/dsh-agents-as-a-service/20-sandboxwarmpool.yaml b/examples/dsh-agents-as-a-service/manifests/20-sandboxwarmpool.yaml similarity index 100% rename from examples/dsh-agents-as-a-service/20-sandboxwarmpool.yaml rename to examples/dsh-agents-as-a-service/manifests/20-sandboxwarmpool.yaml diff --git a/examples/dsh-agents-as-a-service/30-claim-alice.yaml b/examples/dsh-agents-as-a-service/manifests/30-claim-alice.yaml similarity index 100% rename from examples/dsh-agents-as-a-service/30-claim-alice.yaml rename to examples/dsh-agents-as-a-service/manifests/30-claim-alice.yaml diff --git a/examples/dsh-agents-as-a-service/40-claim-rejected.yaml b/examples/dsh-agents-as-a-service/manifests/40-claim-rejected.yaml similarity index 100% rename from examples/dsh-agents-as-a-service/40-claim-rejected.yaml rename to examples/dsh-agents-as-a-service/manifests/40-claim-rejected.yaml diff --git a/examples/dsh-agents-as-a-service/50-gateway.yaml b/examples/dsh-agents-as-a-service/manifests/50-gateway.yaml similarity index 71% rename from examples/dsh-agents-as-a-service/50-gateway.yaml rename to examples/dsh-agents-as-a-service/manifests/50-gateway.yaml index a60ab11..e56b4a4 100644 --- a/examples/dsh-agents-as-a-service/50-gateway.yaml +++ b/examples/dsh-agents-as-a-service/manifests/50-gateway.yaml @@ -6,11 +6,19 @@ # POST /users, POST /login, ... session cookie auth # /u//... HTTP + WebSocket proxy to the user's DSH # -# Image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway -# For local kind development you can build and load a local tag instead, then -# change `image` below to it: -# docker build -t dsh-aaas-gateway:demo gateway/ -# kind load docker-image dsh-aaas-gateway:demo +# Image: the release image published for this example, pinned by tag. +# imagePullPolicy: IfNotPresent lets a local kind build loaded under the same +# tag be used without a registry round-trip (scripts/e2e-kind.sh builds +# dsh-aaas-gateway:e2e and overrides this image). +# +# SSO IS OPT-IN: this manifest runs the gateway with its own bearer/session +# auth only, so AUTH_PROXY_HEADER is intentionally NOT set (see the env +# block). To put GitHub OAuth in front of it, use +# manifests/exposure/gateway-api-sso/. +# +# This Service is ClusterIP and is the only intended entry point: expose it +# through exactly ONE of the documented variants in manifests/exposure/ +# (ingress/ or gateway-api-sso/), never both and never directly. apiVersion: v1 kind: ServiceAccount metadata: @@ -84,8 +92,8 @@ spec: type: RuntimeDefault containers: - name: gateway - image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway - imagePullPolicy: Always + image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2 + imagePullPolicy: IfNotPresent securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true @@ -105,6 +113,10 @@ spec: # gateway created are swept: a claim applied by hand from the # resource-layer manifests carries no aaas.example.com/managed-by # annotation and is never suspended. + # + # These are the demo values (360/240). scripts/e2e-kind.sh + # overrides both to 60/120 with `kubectl set env` so the smoke + # test does not wait six minutes for the sweeper. - name: IDLE_TIMEOUT value: "360" - name: WAKE_TIMEOUT @@ -113,19 +125,24 @@ spec: # the DSH launch token. - name: DSH_CONTAINER value: dsh-web - # Cookie dsh_gw_session is HttpOnly + SameSite=Lax. Set to "true" - # when this Service is reached over HTTPS (e.g. the optional - # 60-ingress.yaml with TLS); "false" keeps the plain-HTTP demo - # usable via port-forward. + # Cookie dsh_gw_session is HttpOnly + SameSite=Lax. "false" keeps + # the plain-HTTP demo usable through port-forward: a Secure cookie + # is never sent over HTTP, so login would silently fail. + # HTTPS deployments (the ingress/ variant with TLS, or + # gateway-api-sso/, which terminates TLS at the Gateway) MUST set + # this to "true". - name: COOKIE_SECURE - value: "true" - # Trusted reverse-proxy identity header for SSO. oauth2-proxy sets - # X-Forwarded-User to the authenticated GitHub login on every request; - # the gateway trusts it (and auto-provisions the user) only when this - # is set. Remove/unset to disable SSO and fall back to bearer/session - # auth only. - - name: AUTH_PROXY_HEADER - value: X-Forwarded-User + value: "false" + # SSO IS OPT-IN AND DISABLED HERE. With AUTH_PROXY_HEADER unset the + # gateway trusts only its own bearer/session auth and ignores every + # identity header. Enable it ONLY behind + # manifests/exposure/gateway-api-sso/, where oauth2-proxy is the + # sole route to this Service: set it to X-Forwarded-User, keep the + # Service ClusterIP and reachable only through the proxy, and + # terminate TLS in front of the gateway. Otherwise anyone who can + # reach :8080 can impersonate any user by sending that header. + # - name: AUTH_PROXY_HEADER + # value: X-Forwarded-User # Reject proxied request bodies larger than this (413). - name: MAX_BODY_MB value: "32" diff --git a/examples/dsh-agents-as-a-service/manifests/exposure/README.md b/examples/dsh-agents-as-a-service/manifests/exposure/README.md new file mode 100644 index 0000000..42b5bb3 --- /dev/null +++ b/examples/dsh-agents-as-a-service/manifests/exposure/README.md @@ -0,0 +1,135 @@ +# Exposing the gateway + +The base manifests deploy the gateway as a ClusterIP Service +(`dsh-gateway:8080`) and expose nothing outside the cluster. WebSocket +upgrades on `/u//**` are part of the contract, so the entry point must +forward `Upgrade`/`Connection` headers (Traefik does by default). + +There are two **mutually exclusive** exposure variants. Pick exactly one: + +| Variant | Entry point | Authentication | TLS | +| --- | --- | --- | --- | +| [`ingress/60-ingress.yaml`](ingress/60-ingress.yaml) | Traefik `Ingress` | the gateway's own signup/session cookie | optional, recommended | +| [`gateway-api-sso/`](gateway-api-sso/72-oauth2-proxy.yaml) | Gateway API `Gateway` + `HTTPRoute` | GitHub OAuth via oauth2-proxy, then the gateway cookie | required (the Gateway listener terminates it) | + +Applying both gives the gateway two entry points, and the SSO variant is only +safe when the gateway is reachable **exclusively** through oauth2-proxy. Do not +mix them. + +## Applying a variant + +Run these commands from the example root +(`examples/dsh-agents-as-a-service/`), the same working directory the example +README uses. + +`kubectl apply -f manifests/` is **not recursive**: it applies only the +top-level numbered files and does not descend into `exposure/`. Apply the base +resources first, then the chosen variant explicitly: + +```bash +# Base resources: namespace, template, warm pool, claims, gateway. +kubectl apply -f manifests/ + +# Plain Ingress exposure: +kubectl apply -f manifests/exposure/ingress/ + +# ... or SSO exposure (create the two Secrets and replace the example.com +# placeholders first — see below): +kubectl apply -f manifests/exposure/gateway-api-sso/ +``` + +**Never use `kubectl apply -R`** (recursive): it would apply both variants at +once and hand out a second, unauthenticated path to the same gateway. Apply the +variant directory on purpose, or the individual files by name. + +`kubectl` only reads `.yaml`, `.yml` and `.json` files from a directory, so +the `*.yaml.example` secret templates are skipped automatically; create the real +Secrets imperatively (commands below) before applying the directory. + +## `ingress/`: plain Ingress, no SSO + +`60-ingress.yaml` routes a host through the cluster's Traefik +(`ingressClassName: traefik`; k3s ships one) to the gateway Service. + +1. Replace the `dsh.example.com` host and the `dsh-example-tls` Secret name + with real values, or delete the `tls:` block to serve plain HTTP. +2. Set `DSH_TRUSTED_HOSTS` in the SandboxTemplate to the same public host: + the gateway forwards the browser's `Host`, and DSH rejects unknown hosts on + `/api` (see `../10-sandboxtemplate.yaml`). +3. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS: + `kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`. The + base manifest keeps `false` so the plain-HTTP port-forward demo works; a + `Secure` cookie is never sent over HTTP, which silently breaks login. +4. Do not attach buffering or compression middlewares to this router: they + break WebSocket upgrades and streaming. + +Verify: `curl -sI https:///healthz` returns `200`. + +## `gateway-api-sso/`: TLS + GitHub OAuth + +`70-gateway.yaml` defines the TLS `Gateway`, `71-httproute.yaml` sends every +request to oauth2-proxy, and `72-oauth2-proxy.yaml` authenticates the browser +against GitHub and proxies to the gateway with `X-Forwarded-User` set. + +Prerequisites: + +- Gateway API CRDs and a Gateway implementation installed; + `gatewayClassName: traefik` matches the Traefik Gateway provider. The CRD + version is pinned in `versions.env` (`GATEWAY_API_VERSION`). +- The OAuth App credentials and a cookie secret, from + `73-oauth2-proxy-secret.yaml.example`. +- A TLS certificate/key for the public host, from + `74-tls-secret.yaml.example`. +- oauth2-proxy is pinned in the manifest to + `quay.io/oauth2-proxy/oauth2-proxy:v7.15.5` (`versions.env`: + `OAUTH2_PROXY_IMAGE`). + +Steps: + +```bash +# 1. Real Secrets (the committed files are *.yaml.example templates). +kubectl -n dsh-demo create secret generic oauth2-proxy \ + --from-literal=client-id='' \ + --from-literal=client-secret='' \ + --from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')" +kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key + +# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72) with +# the real public host and allowed GitHub organization. + +# 3. Enable SSO on the gateway and trust the public host in the template. +kubectl -n dsh-demo set env deploy/dsh-gateway \ + AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true +# then set DSH_TRUSTED_HOSTS=dsh-demo.example.com in the SandboxTemplate +# (see ../10-sandboxtemplate.yaml) and re-apply it. + +# 4. Apply the variant. +kubectl apply -f manifests/exposure/gateway-api-sso/ +``` + +### SSO trust boundary + +`AUTH_PROXY_HEADER` is opt-in and deliberately unset in the base gateway +manifest: the gateway ignores identity headers until it is set. Enabling SSO +takes **both** sides: + +1. `AUTH_PROXY_HEADER=X-Forwarded-User` on `deploy/dsh-gateway` (step 3 + above). Without it, oauth2-proxy's login has no effect on the gateway and + users are still asked for the gateway's own token. +2. The gateway Service must be reachable **only through oauth2-proxy**: keep it + ClusterIP, route all browser traffic through the `HTTPRoute` → + oauth2-proxy → `dsh-gateway` chain, and never apply `ingress/` (or any + other path to the Service) while SSO is enabled. A client that can reach + `dsh-gateway:8080` directly can set `X-Forwarded-User` itself and + impersonate any user. + +Also terminate TLS in front of the gateway (the Gateway listener does) and +leave `COOKIE_SECURE=true` so the session cookie never travels in clear text. + +## Switching variants + +Delete the variant you had before applying the other one, e.g. +`kubectl delete -f manifests/exposure/ingress/`, and revert +`AUTH_PROXY_HEADER`/`COOKIE_SECURE` on the gateway. Verify with +`kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to +the gateway remains. diff --git a/gateway-api/dsh-demo/gateway.yaml b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/70-gateway.yaml similarity index 54% rename from gateway-api/dsh-demo/gateway.yaml rename to examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/70-gateway.yaml index ac684ed..7439b17 100644 --- a/gateway-api/dsh-demo/gateway.yaml +++ b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/70-gateway.yaml @@ -1,3 +1,8 @@ +# Browser entry point (TLS) for the gateway-api-sso variant: terminates TLS +# for dsh-demo.example.com and routes through 71-httproute.yaml to +# oauth2-proxy, which authenticates the browser before the gateway. +# Mutually exclusive with ../../ingress/ (see ../../README.md). The +# "dsh-demo-tls" Secret comes from 74-tls-secret.yaml.example. apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: diff --git a/gateway-api/dsh-demo/httproute.yaml b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/71-httproute.yaml similarity index 79% rename from gateway-api/dsh-demo/httproute.yaml rename to examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/71-httproute.yaml index b03931d..1793e6a 100644 --- a/gateway-api/dsh-demo/httproute.yaml +++ b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/71-httproute.yaml @@ -17,5 +17,7 @@ spec: - backendRefs: # Route through oauth2-proxy (GitHub OAuth) first; it authenticates the # browser and forwards to dsh-gateway:8080 with X-Forwarded-User set. + # The gateway only trusts that header when AUTH_PROXY_HEADER is set on + # its Deployment — see 72-oauth2-proxy.yaml and ../README.md. - name: oauth2-proxy port: 4180 diff --git a/gateway-api/dsh-demo/oauth2-proxy.yaml b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml similarity index 76% rename from gateway-api/dsh-demo/oauth2-proxy.yaml rename to examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml index 152d720..52dfa8b 100644 --- a/gateway-api/dsh-demo/oauth2-proxy.yaml +++ b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml @@ -4,10 +4,25 @@ # (restricted to a configured organization), then forwards the request to # the dsh-gateway Service with the authenticated identity in X-Forwarded-User # (plus X-Auth-Request-* via --set-xauthrequest). The gateway trusts that -# header (AUTH_PROXY_HEADER) to inherit the username (SSO). +# header only when AUTH_PROXY_HEADER is set on its Deployment. # -# Credentials come from the "oauth2-proxy" Secret (see -# oauth2-proxy-secret.yaml.example); this manifest never carries secrets. +# ENABLING SSO TAKES BOTH SIDES (see ../../../exposure/README.md): +# 1. Set AUTH_PROXY_HEADER=X-Forwarded-User on deploy/dsh-gateway, otherwise +# the gateway ignores the header and the login is cosmetic: +# kubectl -n dsh-demo set env deploy/dsh-gateway \ +# AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true +# 2. Make dsh-gateway reachable ONLY through this proxy: keep the Service +# ClusterIP, route browser traffic through 71-httproute.yaml, and do not +# apply ../ingress/ (or anything else that reaches the gateway). A +# client that can reach dsh-gateway:8080 can spoof X-Forwarded-User and +# impersonate any user. +# TLS terminates at the Gateway listener (70-gateway.yaml); keep +# --cookie-secure=true and COOKIE_SECURE=true on the gateway. +# +# The image is pinned to quay.io/oauth2-proxy/oauth2-proxy:v7.15.5 +# (versions.env: OAUTH2_PROXY_IMAGE). Credentials come from the +# "oauth2-proxy" Secret (see 73-oauth2-proxy-secret.yaml.example); this +# manifest never carries secrets. apiVersion: apps/v1 kind: Deployment metadata: diff --git a/gateway-api/dsh-demo/oauth2-proxy-secret.yaml.example b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/73-oauth2-proxy-secret.yaml.example similarity index 100% rename from gateway-api/dsh-demo/oauth2-proxy-secret.yaml.example rename to examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/73-oauth2-proxy-secret.yaml.example diff --git a/gateway-api/dsh-demo/tls-secret.yaml.example b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/74-tls-secret.yaml.example similarity index 91% rename from gateway-api/dsh-demo/tls-secret.yaml.example rename to examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/74-tls-secret.yaml.example index 0114c5d..ae8ffa5 100644 --- a/gateway-api/dsh-demo/tls-secret.yaml.example +++ b/examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/74-tls-secret.yaml.example @@ -2,7 +2,7 @@ # # Copy to tls-secret.yaml and populate with real base64-encoded PEM material, # then apply. The secret must live in the dsh-demo namespace and be named -# "dsh-demo-tls" (referenced by gateway.yaml). +# "dsh-demo-tls" (referenced by 70-gateway.yaml). # # kubectl -n dsh-demo create secret tls dsh-demo-tls \ # --cert=tls.crt --key=tls.key --dry-run=client -o yaml > tls-secret.yaml diff --git a/examples/dsh-agents-as-a-service/60-ingress.yaml b/examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml similarity index 59% rename from examples/dsh-agents-as-a-service/60-ingress.yaml rename to examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml index c7300db..974f354 100644 --- a/examples/dsh-agents-as-a-service/60-ingress.yaml +++ b/examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml @@ -1,15 +1,21 @@ # OPTIONAL browser entry point: expose the gateway through Traefik. # -# Requires 50-gateway.yaml (Service dsh-gateway:8080) and a Traefik install -# with an IngressClass named "traefik" (k3s ships one). Replace the placeholder -# host below with a name that resolves to your ingress address, and either -# supply a real TLS Secret or delete the `tls:` block for plain HTTP. +# This is the "no SSO" exposure variant and is MUTUALLY EXCLUSIVE with +# ../gateway-api-sso/ — apply exactly one of them (see ../README.md). +# +# Requires the base manifests (Service dsh-gateway:8080 from +# manifests/50-gateway.yaml) and a Traefik install with an IngressClass named +# "traefik" (k3s ships one). Replace the placeholder host below with a name +# that resolves to your ingress address, and either supply a real TLS Secret or +# delete the `tls:` block for plain HTTP. # # Two things must match this host: -# 1. SandboxTemplate (10-sandboxtemplate.yaml): set DSH_TRUSTED_HOSTS to -# this host so DSH accepts the forwarded Host/Origin on /api. -# 2. Gateway Deployment (50-gateway.yaml): set COOKIE_SECURE to "true" -# once this Ingress serves HTTPS, so the session cookie is Secure. +# 1. SandboxTemplate (manifests/10-sandboxtemplate.yaml): set +# DSH_TRUSTED_HOSTS to this host so DSH accepts the forwarded Host/Origin +# on /api. +# 2. Gateway Deployment (manifests/50-gateway.yaml): set COOKIE_SECURE to +# "true" once this Ingress serves HTTPS, so the session cookie is Secure. +# The base manifest sets "false" for the plain-HTTP port-forward demo. # # WebSocket support is REQUIRED: the browser talks to /u//... and DSH # uses WebSockets for live session traffic. Traefik upgrades WS/SSE diff --git a/examples/dsh-agents-as-a-service/run-test-kind.sh b/examples/dsh-agents-as-a-service/scripts/e2e-kind.sh similarity index 81% rename from examples/dsh-agents-as-a-service/run-test-kind.sh rename to examples/dsh-agents-as-a-service/scripts/e2e-kind.sh index c81979f..2aaf1e4 100755 --- a/examples/dsh-agents-as-a-service/run-test-kind.sh +++ b/examples/dsh-agents-as-a-service/scripts/e2e-kind.sh @@ -1,13 +1,37 @@ #!/usr/bin/env bash -# Smoke test for the dsh-agents-as-a-service example: runs the full README -# walkthrough non-interactively against the current kubectl context. -# Expects agent-sandbox (with extensions) already installed. +# End-to-end smoke test for the dsh-agents-as-a-service example: runs the +# README walkthrough non-interactively against the current kubectl context. +# +# Prerequisites: +# - A kind cluster whose name matches the current kubectl context, or set +# KIND_CLUSTER=. +# - agent-sandbox with extensions installed (API v1beta1). The release +# manifest is pinned by AGENT_SANDBOX_VERSION in versions.env: +# . ../../../versions.env +# kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${AGENT_SANDBOX_VERSION}/sandbox-with-extensions.yaml +# - docker, to build the gateway image (skipped with SKIP_GATEWAY=1). +# +# Environment: +# SKIP_GATEWAY=1 run only the resource-layer checks (no gateway build/load, +# no signup/proxy/WebSocket/idle/wake checks) +# KIND_CLUSTER kind cluster name for "kind load docker-image" +# (default: derived from the current kubectl context) +# GW_LOCAL_IMAGE local gateway image tag +# (default: dsh-aaas-gateway:e2e) +# +# The gateway is built from the working tree (gateway/) and loaded into kind, +# so this tests the example as checked out, not the published image. +# +# The script is destructive only inside the dsh-demo namespace: it creates the +# namespace (and a throwaway provider key) and DELETES it on exit via the +# cleanup trap. # # NOTE: the only local port bound is the gateway port-forward on 18080. # The DSH Web container port (3080) is in-cluster only and is never bound # on the local host. set -euo pipefail -cd "$(dirname "$0")" +# Example root: paths below are manifests/... and gateway/... from here. +cd "$(dirname "$0")/.." NS=dsh-demo pass() { echo "PASS: $1"; } @@ -20,12 +44,12 @@ cleanup() { trap cleanup EXIT echo "=== 1. install prereqs (throwaway key) + template + warm pool" -kubectl apply -f 00-prereqs.yaml +kubectl apply -f manifests/00-prereqs.yaml # The provider key is never committed; generate a throwaway value. kubectl -n "$NS" delete secret dsh-provider-keys --ignore-not-found >/dev/null kubectl -n "$NS" create secret generic dsh-provider-keys \ --from-literal=DEEPSEEK_API_KEY="sk-test-$(openssl rand -hex 16)" -kubectl apply -f 10-sandboxtemplate.yaml -f 20-sandboxwarmpool.yaml +kubectl apply -f manifests/10-sandboxtemplate.yaml -f manifests/20-sandboxwarmpool.yaml for _ in $(seq 1 120); do READY=$(kubectl -n "$NS" get sandboxes -l agents.x-k8s.io/warm-pool-sandbox \ -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' | grep -c True || true) @@ -37,7 +61,7 @@ pass "warm pool: 2 spares Ready" echo "=== 2. signup via claim (warm adoption)" T0=$(date +%s) -kubectl apply -f 30-claim-alice.yaml +kubectl apply -f manifests/30-claim-alice.yaml kubectl -n "$NS" wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s ELAPSED=$(( $(date +%s) - T0 )) SB=$(kubectl -n "$NS" get sandboxclaim dsh-alice -o jsonpath='{.status.sandbox.name}') @@ -65,7 +89,7 @@ MARKER=$(kubectl -n "$NS" exec "$POD" -- cat /home/node/.dsh/marker.txt) pass "resumed: state survived on the reattached PVC" echo "=== 6. injection policy rejects env-carrying claims" -kubectl apply -f 40-claim-rejected.yaml +kubectl apply -f manifests/40-claim-rejected.yaml # Wait for the POSITIVE rejection signal, not merely "not bound yet". REASON="" for _ in $(seq 1 12); do @@ -78,19 +102,25 @@ done || fail "claim was not rejected (Ready reason='$REASON')" BOUND=$(kubectl -n "$NS" get sandboxclaim dsh-mallory -o jsonpath='{.status.sandbox.name}' 2>/dev/null || true) [ -z "$BOUND" ] || fail "env-carrying claim unexpectedly bound sandbox '$BOUND'" -kubectl delete -f 40-claim-rejected.yaml >/dev/null +kubectl delete -f manifests/40-claim-rejected.yaml >/dev/null pass "claim rejected with EnvVarsInjectionRejected and never bound" echo "=== 7. gateway: browser signup, proxy, idle-suspend, wake-on-connect" if command -v docker >/dev/null && command -v kind >/dev/null; then KIND_CLUSTER="${KIND_CLUSTER:-$(kubectl config current-context | sed 's/^kind-//')}" - # Tag exactly what 50-gateway.yaml references so the smoke test exercises - # the shipped manifest unchanged (imagePullPolicy: IfNotPresent uses the - # image loaded into kind). - GW_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway - docker build -q -t "$GW_IMAGE" gateway/ >/dev/null - kind load docker-image "$GW_IMAGE" --name "$KIND_CLUSTER" >/dev/null - kubectl apply -f 50-gateway.yaml + GW_LOCAL_IMAGE="${GW_LOCAL_IMAGE:-dsh-aaas-gateway:e2e}" + # Test the working tree, not the published image: build the gateway from + # gateway/, load it into kind, then point the Deployment at that tag. The + # shipped manifest pins the release image; imagePullPolicy: IfNotPresent + # makes the node use the loaded one. + docker build -q -t "$GW_LOCAL_IMAGE" gateway/ >/dev/null + kind load docker-image "$GW_LOCAL_IMAGE" --name "$KIND_CLUSTER" >/dev/null + kubectl apply -f manifests/50-gateway.yaml + kubectl -n "$NS" set image deploy/dsh-gateway gateway="$GW_LOCAL_IMAGE" + # Override the demo timeouts from manifests/50-gateway.yaml (360/240) with the + # gateway's working-tree defaults, so idle-suspend and wake-on-connect are + # exercised in seconds instead of minutes. + kubectl -n "$NS" set env deploy/dsh-gateway IDLE_TIMEOUT=60 WAKE_TIMEOUT=120 kubectl -n "$NS" rollout status deploy/dsh-gateway --timeout=120s # Local port-forward uses 18080 (NOT 3080, which stays in-cluster). kubectl -n "$NS" port-forward svc/dsh-gateway 18080:8080 >/dev/null 2>&1 & diff --git a/examples/minimal-single-tenant/README.md b/examples/minimal-single-tenant/README.md new file mode 100644 index 0000000..dec87ac --- /dev/null +++ b/examples/minimal-single-tenant/README.md @@ -0,0 +1,127 @@ +# Minimal single-tenant DSH Web + +The smallest Kubernetes deployment of DeepSeek Harness (DSH) Web: one +`SandboxTemplate`, a one-replica warm pool and one claim. It is the "just one +DSH Web instance" quickstart. + +For the full platform — warm pool sizing, per-user claims, a browser-facing +gateway with cookie sessions, idle-suspend and wake-on-connect — see +[`examples/dsh-agents-as-a-service/`](../dsh-agents-as-a-service/README.md). +This example deliberately has no gateway: you reach DSH through a +`kubectl port-forward` and its launch-token URL. + +## Prerequisites + +- A Kubernetes cluster with a default StorageClass (kind and GKE both work). +- The agent-sandbox controller with extensions (API `v1beta1`) installed. + The version is pinned as `AGENT_SANDBOX_VERSION` in + [`versions.env`](../../versions.env): + + ```bash + # from the repository root: + . versions.env + kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${AGENT_SANDBOX_VERSION}/sandbox-with-extensions.yaml" + kubectl -n agent-sandbox-system rollout status deployment --timeout=180s + ``` + +- A DeepSeek API key for real model calls. The pod starts without one; only + model requests fail. + +## Quickstart + +Run these from this example's directory (`examples/minimal-single-tenant/`). + +```bash +# 1. Namespace + shared provider key. The key is injected into the sandbox via +# envFrom and is never committed. +kubectl create namespace dsh-demo +kubectl -n dsh-demo create secret generic dsh-provider-keys \ + --from-literal=DEEPSEEK_API_KEY="sk-..." + +# 2. The template itself (image pinned in the manifest). +kubectl apply -f manifests/sandbox-template.yaml + +# 3. A template only describes pods; a warm pool turns it into running +# Sandboxes and a claim adopts one. One replica is enough here. +kubectl apply -f - <<'EOF' +apiVersion: extensions.agents.x-k8s.io/v1beta1 +kind: SandboxWarmPool +metadata: + name: dsh-pool + namespace: dsh-demo +spec: + replicas: 1 + sandboxTemplateRef: + name: dsh-web-template +--- +apiVersion: extensions.agents.x-k8s.io/v1beta1 +kind: SandboxClaim +metadata: + name: dsh + namespace: dsh-demo +spec: + warmPoolRef: + name: dsh-pool +EOF + +kubectl -n dsh-demo wait sandboxclaim/dsh --for=condition=Ready --timeout=180s +``` + +## Talk to the instance + +DSH gates `/` behind a launch token it prints only in its startup logs. Open +the printed URL once; DSH then sets its own auth cookie and the UI loads +normally. + +```bash +SB=$(kubectl -n dsh-demo get sandboxclaim dsh -o jsonpath='{.status.sandbox.name}') +kubectl -n dsh-demo logs "$SB" | grep -m1 '?token=' +# dsh web: http://:3080/?token=... + +kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080 +# open http://localhost:18081/?token=... from the log line +``` + +> **Port note:** DSH Web listens on `3080` in-cluster. If `3080` is already in +> use on your workstation, forward to another local port (here `18081`), never +> `3080`. `localhost` is always in DSH's trusted-host list, so the forwarded +> request is accepted. + +## Probes and trusted hosts + +Two settings in `manifests/sandbox-template.yaml` are easy to get wrong: + +- **Probe path.** `/` returns `401` without the launch-token cookie, so a probe + there crash-loops the pod. The template probes `/favicon.svg` (served + unauthenticated) and gates liveness/readiness behind a `startupProbe`, which + gives a cold start (plugin activation, or the first resume) up to 120s before + a restart. +- **Host allow-list.** DSH rejects unknown `Host` headers. `localhost` and + `127.0.0.1` are always trusted; if you later front the sandbox with an + Ingress or proxy, set `DSH_TRUSTED_HOSTS` to that public host (comma or space + separated). The full example explains both exposure variants in + [`manifests/exposure/README.md`](../dsh-agents-as-a-service/manifests/exposure/README.md). + +The template also sets `fsGroupChangePolicy: OnRootMismatch`. DSH's +credentials plugin requires `.credentials.yaml` to be `0600`, while the +kubelet's default fsGroup policy rewrites it to `0660` on every pod start — +without this policy a recreated or resumed sandbox crash-loops. The rationale +is spelled out in +[`10-sandboxtemplate.yaml`](../dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml). + +## Suspend, resume and clean up + +```bash +# Suspend: the pod is deleted, the PVC (state) and Service are kept. +kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Suspended"}}' + +# Resume: a new pod reattaches the same PVC; conversations survive. +kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}' +kubectl -n dsh-demo wait sandbox "$SB" --for=condition=Ready --timeout=180s + +# Delete everything this quickstart created. +kubectl -n dsh-demo delete sandboxclaim dsh +kubectl delete namespace dsh-demo +``` + +Deleting the claim cascades to the sandbox and its PVC. diff --git a/deploy/k8s/sandbox-template.yaml b/examples/minimal-single-tenant/manifests/sandbox-template.yaml similarity index 78% rename from deploy/k8s/sandbox-template.yaml rename to examples/minimal-single-tenant/manifests/sandbox-template.yaml index 4051dfd..b055d6b 100644 --- a/deploy/k8s/sandbox-template.yaml +++ b/examples/minimal-single-tenant/manifests/sandbox-template.yaml @@ -4,8 +4,9 @@ # multi-tenant platform (warm pool, per-user claims, gateway, idle-suspend), # see examples/dsh-agents-as-a-service/. # -# Prerequisites (create these first, or apply 00-prereqs.yaml from the -# examples directory): +# Claims reference this template directly (spec.sandboxTemplateRef): there is +# no warm pool and no per-user platform here. See the README for the full +# quickstart; in short the prerequisites are: # - Namespace dsh-demo. # - Secret dsh-provider-keys in dsh-demo with key DEEPSEEK_API_KEY # (injected via envFrom; the pod starts without it, only model calls fail). @@ -26,16 +27,15 @@ spec: fsGroup: 1000 # DSH's credentials plugin refuses to start when .credentials.yaml has # any group/other permission bit (it creates the file 0600). The - # default kubelet fsGroup policy ("Always") recursively ORs 0660 into - # every file each time the pod starts, so a recreated/resumed sandbox - # comes back with 0660 and crash-loops. OnRootMismatch skips that walk - # once the volume root already has the fsGroup ownership/mode from the - # first mount. Keep in sync with - # examples/dsh-agents-as-a-service/10-sandboxtemplate.yaml. + # default kubelet fsGroup policy ("Always") rewrites it to 0660 on every + # pod start, so a recreated or resumed sandbox crash-loops; OnRootMismatch + # avoids that walk once the volume root already carries the ownership. + # Rationale in full at + # examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml. fsGroupChangePolicy: OnRootMismatch containers: - name: dsh-web - image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 + image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 imagePullPolicy: IfNotPresent ports: - name: web @@ -50,8 +50,10 @@ spec: value: "" # DSH Host/Origin-fences /api. Empty trusts only localhost, which # is right for `kubectl port-forward`. Set this to the public host - # (e.g. dsh.example.com) if you expose DSH behind an Ingress, so - # requests forwarded with the browser's Host are accepted. + # (e.g. dsh.example.com) if you front DSH with a proxy or Ingress, + # so requests forwarded with the browser's Host are accepted. The + # full example documents both exposure variants in + # examples/dsh-agents-as-a-service/manifests/exposure/README.md. - name: DSH_TRUSTED_HOSTS value: "" volumeMounts: diff --git a/gateway-api/dsh-demo/README.md b/gateway-api/dsh-demo/README.md deleted file mode 100644 index 2519000..0000000 --- a/gateway-api/dsh-demo/README.md +++ /dev/null @@ -1,91 +0,0 @@ -# Gateway API — external access for `dsh-demo/dsh-gateway` (GitHub SSO) - -Exposes the `dsh-gateway` service (`dsh-demo`, port `8080`) outside the cluster -using the Kubernetes Gateway API (Traefik Gateway controller), fronted by -**oauth2-proxy** so access requires a GitHub login restricted to a configured -GitHub organization. The gateway inherits the authenticated GitHub username via -the `X-Forwarded-User` header (SSO). - -## Request flow - -``` -Browser ──▶ Gateway dsh-gateway (HTTPS, TLS-terminate) - ──▶ HTTPRoute - ──▶ oauth2-proxy:4180 (GitHub OAuth, org-restricted) - ──▶ dsh-gateway:8080 (X-Forwarded-User = GitHub login → SSO) -``` - -## Resources - -| File | Kind | Purpose | -|---|---|---| -| `gateway.yaml` | `Gateway` | HTTPS listener with TLS termination | -| `httproute.yaml` | `HTTPRoute` | Routes the hostname → `oauth2-proxy:4180` | -| `oauth2-proxy.yaml` | `Deployment`/`Service` | GitHub OAuth reverse proxy | -| `oauth2-proxy-secret.yaml.example` | `Secret` | Example OAuth credentials (not applied as-is) | -| `tls-secret.yaml.example` | `Secret` | Example TLS cert secret (not applied as-is) | - -## Prerequisites - -1. **GitHub OAuth App** — create one at - `https://github.com/settings/developers` with: - - Homepage URL: `https://dsh-demo.example.com` - - Authorization callback URL: `https://dsh-demo.example.com/oauth2/callback` - (replace `dsh-demo.example.com` with the actual public FQDN). Note the client - ID and client secret. -2. **oauth2-proxy Secret** — `kubectl -n dsh-demo create secret generic oauth2-proxy` - with `client-id`, `client-secret`, and a `cookie-secret` (URL-safe, unpadded - base64 of 32 bytes: `python3 -c 'import secrets; print(secrets.token_urlsafe(32))'`). - See `oauth2-proxy-secret.yaml.example`. -3. **TLS secret** — create `dsh-demo-tls` in `dsh-demo` (see `tls-secret.yaml.example`). -4. **Gateway app env** — the `dsh-gateway` Deployment must run the SSO-capable - image with `AUTH_PROXY_HEADER=X-Forwarded-User` and `COOKIE_SECURE=true` - (see `examples/dsh-agents-as-a-service/50-gateway.yaml`). -5. **Ingress and DNS** — configure the upstream ingress and public DNS for the - hostname. - -## Apply - -```bash -kubectl diff -f gateway-api/dsh-demo/ -kubectl apply -f gateway-api/dsh-demo/ --dry-run=server -kubectl apply -f gateway-api/dsh-demo/ -``` - -> ⚠️ A write-enabled kubeconfig is required to create Gateways, HTTPRoutes, -> Deployments, and Secrets. - -## Verify - -```bash -kubectl get gateway dsh-gateway -n dsh-demo -o yaml # expect Programmed=True -kubectl get httproute -n dsh-demo -o wide # backend oauth2-proxy:4180, Accepted -kubectl get deploy oauth2-proxy -n dsh-demo # 1/1 Ready -kubectl get endpoints oauth2-proxy -n dsh-demo # backend present -kubectl -n dsh-demo logs deploy/oauth2-proxy --tail=50 # no config errors -``` - -Browser check: open `https:///` — you should be redirected -to GitHub, and after signing in as a member of the allowed organization your -sandbox opens with the session showing your GitHub login. A non-member is denied -before reaching the gateway. - -## Rollback - -```bash -kubectl delete -f gateway-api/dsh-demo/ -``` - -## Notes - -- **SSO is opt-in and additive.** The gateway honors `X-Forwarded-User` only when - `AUTH_PROXY_HEADER` is set; unset it to restore bearer-token/session-only auth. -- **Trust boundary.** After this change the `dsh-gateway` Service must be reachable - **only** through oauth2-proxy (it is a ClusterIP Service). A second Route/Ingress - that exposes `dsh-gateway:8080` directly would let clients spoof the header. -- **GitHub login → DSH user.** GitHub usernames are lowercase alphanumerics and - hyphens, so they map 1:1 onto the gateway's dns-1123 `user` validation. -- **Sign-out.** The gateway `/logout` clears `dsh_gw_session`; oauth2-proxy's own - session can be cleared at `/oauth2/sign_out` if desired. -- No `ReferenceGrant` is required: the Gateway, HTTPRoute, oauth2-proxy, and the - backend Service are all in `dsh-demo`. diff --git a/scripts/check-examples.sh b/scripts/check-examples.sh new file mode 100755 index 0000000..4cb2941 --- /dev/null +++ b/scripts/check-examples.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Enforce the example contract documented in docs/examples.md. +# Run by "make check" locally and by the CI lint job. +set -euo pipefail +cd "$(dirname "$0")/.." + +index=examples/README.md +fail=0 +err() { echo "check-examples: $*" >&2; fail=1; } + +[ -f "$index" ] || err "missing $index" + +for dir in examples/*/; do + name="$(basename "$dir")" + [ "$name" = "_template" ] && continue + + [ -f "${dir}README.md" ] || err "$name: missing README.md" + [ -d "${dir}manifests" ] || err "$name: missing manifests/" + + if [ -d "${dir}manifests" ]; then + count=$(find "${dir}manifests" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ') + [ "$count" -gt 0 ] || err "$name: manifests/ has no top-level *.yaml files" + fi + + stray=$(find "$dir" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ') + [ "$stray" -eq 0 ] || err "$name: manifests belong under manifests/, not at the example root" + + if [ -d "${dir}manifests/exposure" ] && [ ! -f "${dir}manifests/exposure/README.md" ]; then + err "$name: manifests/exposure/ requires a README.md describing the variants" + fi + + for component in "$dir"*/; do + [ -d "$component" ] || continue + case "$(basename "$component")" in + manifests|scripts|docs) continue ;; + esac + if [ -f "${component}Dockerfile" ] && [ ! -f "${component}README.md" ]; then + err "$name/$(basename "$component"): component with a Dockerfile needs a README.md" + fi + done + + if ! grep -Fq "examples/$name/" "$index"; then + err "$name: not listed in examples/README.md" + fi + + for script in "$dir"scripts/*.sh; do + [ -e "$script" ] || continue + [ -x "$script" ] || err "$script: must be executable" + done +done + +if [ "$fail" -ne 0 ]; then + echo "check-examples: FAILED" >&2 + exit 1 +fi +echo "check-examples: ok" diff --git a/scripts/check-manifests.py b/scripts/check-manifests.py new file mode 100644 index 0000000..5d0557b --- /dev/null +++ b/scripts/check-manifests.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python3 +"""Parse every example manifest and assert the minimal Kubernetes shape.""" +from __future__ import annotations + +import sys +from pathlib import Path + +try: + import yaml +except ModuleNotFoundError: # pragma: no cover - environment guard + sys.exit("check-manifests: PyYAML is required (python -m pip install pyyaml)") + +ROOT = Path(__file__).resolve().parent.parent +FILES = sorted(ROOT.glob("examples/*/manifests/**/*.yaml")) + +if not FILES: + sys.exit("check-manifests: no manifests found under examples/*/manifests/") + +errors: list[str] = [] +documents = 0 +for path in FILES: + relative = path.relative_to(ROOT) + try: + loaded = list(yaml.safe_load_all(path.read_text(encoding="utf-8"))) + except yaml.YAMLError as exc: + errors.append(f"{relative}: invalid YAML: {exc}") + continue + for position, document in enumerate(loaded, start=1): + if document is None: + continue + documents += 1 + if not isinstance(document, dict): + errors.append(f"{relative} document {position}: not a mapping") + continue + missing = [key for key in ("apiVersion", "kind", "metadata") if key not in document] + if missing: + errors.append(f"{relative} document {position}: missing {', '.join(missing)}") + continue + metadata = document["metadata"] + if not isinstance(metadata, dict) or not metadata.get("name"): + errors.append(f"{relative} document {position}: metadata.name is required") + +if errors: + print("\n".join(errors), file=sys.stderr) + sys.exit(1) + +print(f"check-manifests: ok ({len(FILES)} files, {documents} documents)") diff --git a/scripts/check-versions.sh b/scripts/check-versions.sh new file mode 100755 index 0000000..6e124b3 --- /dev/null +++ b/scripts/check-versions.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# Verify that every version pin in the repository agrees with versions.env. +# Run by "make check" locally and by the CI lint job. +set -euo pipefail +cd "$(dirname "$0")/.." + +# shellcheck source=/dev/null +. ./versions.env + +IDENTITY="$DSH_VERSION" +if [ -n "${RELEASE_REVISION:-}" ]; then + IDENTITY="$DSH_VERSION-$RELEASE_REVISION" +fi + +fail=0 +err() { echo "check-versions: $*" >&2; fail=1; } +expect() { # + if ! grep -Fq -- "$2" "$1"; then + err "$1: expected to contain: $2" + fi +} + +# Main Dockerfile build args +expect Dockerfile "ARG DSH_VERSION=$DSH_VERSION" +expect Dockerfile "ARG PNPM_VERSION=$PNPM_VERSION" +expect Dockerfile "ARG RELEASE_REVISION=$RELEASE_REVISION" + +# Local configuration mirrored for humans +expect .env.sample "DSH_VERSION=$DSH_VERSION" +expect .env.sample "PNPM_VERSION=$PNPM_VERSION" +expect .env.sample "RELEASE_REVISION=$RELEASE_REVISION" + +# Image references in the example manifests +for f in \ + examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml \ + examples/minimal-single-tenant/manifests/sandbox-template.yaml; do + expect "$f" "image: $GHCR_IMAGE:$IDENTITY" +done +expect examples/dsh-agents-as-a-service/manifests/50-gateway.yaml "image: $GATEWAY_IMAGE:$IDENTITY" + +# The gateway image must live below the repository image (one package per +# example component, no unrelated top-level packages). +case "$GATEWAY_IMAGE" in + "$GHCR_IMAGE"/*) ;; + *) err "GATEWAY_IMAGE ($GATEWAY_IMAGE) must be nested below GHCR_IMAGE ($GHCR_IMAGE)" ;; +esac + +# Build tooling must read versions.env instead of hard-coding a version. +# (scripts/ is excluded: set-version.sh documents usage examples in comments.) +while IFS= read -r hit; do + [ -n "$hit" ] || continue + err "hard-coded version outside versions.env: $hit" +done < <(grep -rnE 'dsh-v[0-9]' .github/workflows Makefile 2>/dev/null || true) + +# Prose may mention versions, but only the current identity or its base tag. +while IFS= read -r token; do + [ -n "$token" ] || continue + case "$token" in + "$IDENTITY"|"$DSH_VERSION") ;; + *) err "stale version reference in docs: $token" ;; + esac +done < <(grep -rhoE 'dsh-v[0-9][A-Za-z0-9._-]*' \ + --include='*.md' --exclude-dir=.git --exclude-dir=.venv --exclude-dir=.uvcache \ + --exclude=CHANGELOG.md . | sort -u) + +if [ "$fail" -ne 0 ]; then + echo "check-versions: FAILED" >&2 + exit 1 +fi +echo "check-versions: ok ($IDENTITY)" diff --git a/scripts/set-version.sh b/scripts/set-version.sh new file mode 100755 index 0000000..7912322 --- /dev/null +++ b/scripts/set-version.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# Rewrite every version pin in one pass, then leave the tree ready for +# "make check". +# +# ./scripts/set-version.sh dsh-v0.2.2-rc.1 # new DSH ref, no revision +# ./scripts/set-version.sh dsh-v0.2.1-rc.2 r3 # same ref, new repo revision +set -euo pipefail +cd "$(dirname "$0")/.." + +new_version="${1:-}" +new_revision="${2:-}" + +case "$new_version" in + dsh-v[0-9]*) ;; + *) echo "usage: $0 [rN]" >&2; exit 2 ;; +esac + +if [ -n "$new_revision" ]; then + case "$new_revision" in + r[2-9]|r[1-9][0-9]*) ;; + *) echo "revision must look like r2, r3, ..." >&2; exit 2 ;; + esac +fi + +# shellcheck source=/dev/null +old_version="$(. ./versions.env; printf '%s' "$DSH_VERSION")" +# shellcheck source=/dev/null +old_revision="$(. ./versions.env; printf '%s' "${RELEASE_REVISION:-}")" + +old_identity="$old_version" +[ -z "$old_revision" ] || old_identity="$old_version-$old_revision" +new_identity="$new_version" +[ -z "$new_revision" ] || new_identity="$new_version-$new_revision" + +if [ "$old_identity" = "$new_identity" ]; then + echo "set-version: already at $new_identity" + exit 0 +fi + +# Replace the identity first (it contains the base version), then the base +# version itself. CHANGELOG.md is generated from tags, so it is left alone. +while IFS= read -r file; do + [ -f "$file" ] || continue + case "$file" in + .env.sample|Dockerfile|Makefile|*.md|*.yaml|*.yml|*.sh|*.py|*.toml|*.env) ;; + *) continue ;; + esac + sed -i "s/$old_identity/$new_identity/g; s/$old_version/$new_version/g" "$file" +done < <(git ls-files) + +# versions.env is authoritative: write both keys explicitly so an empty +# revision is stored as an empty value. +sed -i "s/^DSH_VERSION=.*/DSH_VERSION=$new_version/" versions.env +sed -i "s/^RELEASE_REVISION=.*/RELEASE_REVISION=$new_revision/" versions.env + +echo "set-version: $old_identity -> $new_identity" +echo "set-version: run 'make check' and review the diff" diff --git a/versions.env b/versions.env new file mode 100644 index 0000000..637baf6 --- /dev/null +++ b/versions.env @@ -0,0 +1,26 @@ +# Single source of truth for every pinned version and image name in this +# repository. Consumed by the Makefile, build.sh, scripts/*.sh and the GitHub +# Actions workflows. Never hard-code these values elsewhere: run "make check" +# (scripts/check-versions.sh) to verify every pin. + +# Upstream DeepSeek Harness source, and the ref this repository packages. +DSH_REPO=https://github.com/deepseek-ai/deepseek-harness.git +DSH_VERSION=dsh-v0.2.1-rc.2 +PNPM_VERSION=11.28.5 + +# Release identity = DSH_VERSION plus an optional repository revision. +# An empty RELEASE_REVISION is the first release at this DSH ref; rN (N >= 2) +# marks a repository-only re-release at the same DSH ref. The dsh-v0.2.1-rc.2 +# image tag is already published on GHCR and tags are immutable, so the first +# release under the current scheme is dsh-v0.2.1-rc.2-r2. +RELEASE_REVISION=r2 + +# Control-plane versions used by the examples and their manifests. +AGENT_SANDBOX_VERSION=v1.0.7 +GATEWAY_API_VERSION=v1.2.1 +OAUTH2_PROXY_IMAGE=quay.io/oauth2-proxy/oauth2-proxy:v7.15.5 + +# Registry layout: the main image is the repository itself, and every example +# component is a nested package below it (////). +GHCR_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox +GATEWAY_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway