1
0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00

feat(repo)!: repo reorganization (#16)

* feat(repo)!: restructure around the dsh-harness image and one example per implementation

- the root Dockerfile is the primary artifact; versions.env is the single source of truth
- merge gateway-api/dsh-demo into examples/dsh-agents-as-a-service (manifests/, exposure variants)
- move the single-tenant template into examples/minimal-single-tenant
- publish the gateway as a nested package under the repository image
- mirror the DSH ref for release identities (first release dsh-v0.2.1-rc.2-r2)
- CI: lint/python/manifests/gateway-image, reusable multi-arch build, tag-driven release, weekly e2e
- docs: README, docs/{main-image,examples,gitflow,versioning}, example READMEs, AGENTS.md, governance
- SSO is opt-in again; gateway pull policy and cookie defaults match the documented demo

BREAKING CHANGE: example manifest paths, the gateway image name, and the removal
of deploy/ and gateway-api/ affect existing deployments; see the migration table
in README.md.

* docs(changelog): generate initial changelog from history

* fix(scripts): make set-version update Dockerfile and sample revision pins

* build(hadolint): document intentional lint exceptions for the main image

---------

Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
This commit is contained in:
Nicolás Georger
2026-10-25 08:37:43 -03:00
committed by GitHub
parent 53e55fc851
commit 3b70f179ac
69 changed files with 3774 additions and 1643 deletions

View File

@@ -1,149 +1,6 @@
# Logs # The main image build clones DeepSeek Harness from git and copies nothing
logs # from this repository, so the build context is intentionally reduced to the
*.log # Dockerfile alone. This keeps builds fast and prevents the rest of the repo
npm-debug.log* # (examples, docs, git history) from being sent to the builder.
yarn-debug.log* *
yarn-error.log* !Dockerfile
lerna-debug.log*
# Diagnostic reports (https://nodejs.org/api/report.html)
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
# Runtime data
pids
*.pid
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
*.lcov
# nyc test coverage
.nyc_output
# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Bower dependency directory (https://bower.io/)
bower_components
# node-waf configuration
.lock-wscript
# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release
# Dependency directories
node_modules/
jspm_packages/
# Snowpack dependency directory (https://snowpack.dev/)
web_modules/
# TypeScript cache
*.tsbuildinfo
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Optional stylelint cache
.stylelintcache
# Optional REPL history
.node_repl_history
# Output of 'npm pack'
*.tgz
# Yarn Integrity file
.yarn-integrity
# dotenv environment variable files
.env
.env.*
!.env.example
# parcel-bundler cache (https://parceljs.org/)
.cache
.parcel-cache
# Next.js build output
.next
out
# Nuxt.js build / generate output
.nuxt
dist
.output
# Gatsby files
.cache/
# Comment in the public line in if your project uses Gatsby and not Next.js
# https://nextjs.org/blog/next-9-1#public-directory-support
# public
# vuepress build output
.vuepress/dist
# vuepress v2.x temp directory
.temp
# Sveltekit cache directory
.svelte-kit/
# vitepress build output
**/.vitepress/dist
# vitepress cache directory
**/.vitepress/cache
# Docusaurus cache and generated files
.docusaurus
# Serverless directories
.serverless/
# FuseBox cache
.fusebox/
# DynamoDB Local files
.dynamodb/
# Firebase cache directory
.firebase/
# TernJS port file
.tern-port
# Stores Visual Studio Code versions used for testing Visual Studio Code extensions
.vscode-test
# pnpm
.pnpm-store
# yarn v3
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
# Vite files
vite.config.js.timestamp-*
vite.config.ts.timestamp-*
.vite/
# Astro generated types
.astro/
# Turbo Repo
.turbo

15
.editorconfig Normal file
View File

@@ -0,0 +1,15 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = space
indent_size = 2
[*.py]
indent_size = 4
[Makefile]
indent_style = tab

View File

@@ -1,29 +1,35 @@
# Copy to .env and fill in. Never commit a real key. # Copy to .env and fill in. Never commit a real key.
#
# The version values below are documentation only: versions.env is the single
# source of truth, and scripts/check-versions.sh fails if they drift.
# --- DSH Web image (Dockerfile / build.sh) --- # --- Main DSH Web image (root Dockerfile) ---
DEEPSEEK_API_KEY= DEEPSEEK_API_KEY=
DSH_VERSION=dsh-v0.2.1-rc.2 DSH_VERSION=dsh-v0.2.1-rc.2
RELEASE_REVISION=r2
PNPM_VERSION=11.28.5 PNPM_VERSION=11.28.5
# DSH Web port inside the container, and the local port published by # Container port, and the host port published by "make run".
# build.sh / docker run. The gateway also uses it as the upstream port.
DSH_PORT=3080 DSH_PORT=3080
# Comma/space separated extra Host allow-list entries for DSH (for an # Comma/space separated extra Host allow-list entries for DSH (for a proxy or
# Ingress/public hostname). localhost and 127.0.0.1 are always trusted. # public hostname). localhost and 127.0.0.1 are always trusted.
DSH_TRUSTED_HOSTS= DSH_TRUSTED_HOSTS=
# --- Gateway (examples/dsh-agents-as-a-service) --- # --- agents-as-a-service example ---
GATEWAY_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway
NAMESPACE=dsh-demo NAMESPACE=dsh-demo
POOL=dsh-pool POOL=dsh-pool
# Seconds a sandbox may stay idle before the sweeper suspends it. # Seconds a sandbox may stay idle before the sweeper suspends it, and the max
IDLE_TIMEOUT=60 # time a proxied request waits for a suspended sandbox to become Ready.
# Seconds a waking request may wait for the sandbox to become Ready. IDLE_TIMEOUT=360
WAKE_TIMEOUT=120 WAKE_TIMEOUT=240
DSH_CONTAINER=dsh-web DSH_CONTAINER=dsh-web
DSH_LOG_TAIL_LINES=2000 DSH_LOG_TAIL_LINES=2000
COOKIE_NAME=dsh_gw_session COOKIE_NAME=dsh_gw_session
# Set to true only behind HTTPS; secure cookies are not sent over plain HTTP. # Set to true only behind HTTPS; secure cookies are not sent over plain HTTP
# (localhost counts as a secure context in browsers).
COOKIE_SECURE=false COOKIE_SECURE=false
COOKIE_MAX_AGE=604800 COOKIE_MAX_AGE=604800
MAX_BODY_MB=32 MAX_BODY_MB=32
HTTP_TIMEOUT=300 HTTP_TIMEOUT=300
# SSO is opt-in. Set this only when the gateway Service is reachable
# exclusively through oauth2-proxy (see the example exposure docs).
# AUTH_PROXY_HEADER=X-Forwarded-User

42
.github/ISSUE_TEMPLATE/bug_report.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: Bug report
description: Report a reproducible problem with the image, an example or the tooling.
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Do not include API keys, tokens or other secrets. Report security
issues privately via SECURITY.md instead.
- type: textarea
id: what-happened
attributes:
label: What happened
description: The observed behavior and the expected behavior.
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Reproduction steps
description: Commands, manifests and versions. Redact secrets.
placeholder: |
make image
docker run ...
validations:
required: true
- type: input
id: image-tag
attributes:
label: Image or release
description: The pinned tag or digest, for example dsh-v0.2.1-rc.2-r2.
- type: input
id: cluster
attributes:
label: Environment
description: Kubernetes distribution/version, kind version, agent-sandbox version.
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste the smallest useful excerpt. Remove tokens and API keys.
render: shell

8
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/security/advisories/new
about: Report security issues privately; do not open a public issue.
- name: Documentation
url: https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/tree/main/docs
about: Read the repository docs before opening an issue.

View File

@@ -0,0 +1,35 @@
name: Feature request
description: Propose a new example, image capability or workflow improvement.
labels: ["enhancement"]
body:
- type: textarea
id: problem
attributes:
label: Problem or use case
description: What are you trying to do, and what blocks you today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
description: Describe the shape you have in mind. For a new example, say which implementation it demonstrates.
validations:
required: true
- type: dropdown
id: area
attributes:
label: Area
options:
- Main DSH Web image
- Existing example
- New example
- CI / release automation
- Documentation
- Other
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered

28
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,28 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
groups:
actions:
patterns: ["*"]
- package-ecosystem: pip
directory: /examples/dsh-agents-as-a-service/gateway
schedule:
interval: weekly
groups:
gateway-python:
patterns: ["*"]
- package-ecosystem: docker
directory: /
schedule:
interval: weekly
- package-ecosystem: docker
directory: /examples/dsh-agents-as-a-service/gateway
schedule:
interval: weekly

31
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,31 @@
## Summary
<!-- What changes, and why? Link the issue: Closes #123 -->
## Type of change
- [ ] feat: new capability or example
- [ ] fix: bug fix
- [ ] docs: documentation only
- [ ] refactor: no behavior change
- [ ] build / ci: tooling, images, workflows
- [ ] chore: maintenance
## Breaking change
- [ ] Yes, and the migration is documented in the description
- [ ] No
## Checklist
- [ ] make check passes (versions, example layout, manifest YAML)
- [ ] make test passes (gateway unit tests)
- [ ] make lint passes (ruff, shellcheck; yamllint/actionlint where installed)
- [ ] Docs updated for the change (root README, docs/, example README)
- [ ] No secrets or usable credentials committed
- [ ] CHANGELOG.md was not edited by hand (generated by git-cliff)
- [ ] Version pins changed only through versions.env / make set-version
## Testing
<!-- Commands run, clusters used, results. Mention if kind e2e was skipped. -->

160
.github/workflows/_build-image.yaml vendored Normal file
View File

@@ -0,0 +1,160 @@
# Reusable multi-arch image build: one workflow, called by build-main,
# build-gateway and release. Builds natively per platform (no QEMU), pushes by
# digest, then merges the manifest list with SBOM and provenance attestations.
name: Reusable multi-arch image build
on:
workflow_call:
inputs:
image:
description: Fully qualified image name without a tag
required: true
type: string
version:
description: Version recorded in the OCI labels
required: true
type: string
context:
description: Docker build context
required: true
type: string
dockerfile:
description: Dockerfile path, relative to the context
required: false
default: Dockerfile
type: string
build-args:
description: Newline-separated build args
required: false
default: ""
type: string
tags:
description: Newline-separated fully qualified tags to publish
required: true
type: string
permissions:
contents: read
packages: write
id-token: write
attestations: write
jobs:
build:
name: Build ${{ matrix.platform }}
runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || 'ubuntu-24.04-arm' }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm64
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare environment
env:
IMAGE: ${{ inputs.image }}
run: |
set -euo pipefail
pair=$(printf '%s' "${{ matrix.platform }}" | tr '/' '-')
echo "PLATFORM_PAIR=$pair" >> "$GITHUB_ENV"
echo "CACHE_SCOPE=$(printf '%s' "$IMAGE" | tr '/' '-')" >> "$GITHUB_ENV"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest
id: build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: ${{ inputs.context }}
file: ${{ inputs.context }}/${{ inputs.dockerfile }}
platforms: ${{ matrix.platform }}
build-args: ${{ inputs.build-args }}
labels: |
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.version=${{ inputs.version }}
org.opencontainers.image.revision=${{ github.sha }}
outputs: type=image,name=${{ inputs.image }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true
sbom: true
provenance: mode=max
cache-from: type=gha,scope=${{ env.CACHE_SCOPE }}-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ env.CACHE_SCOPE }}-${{ matrix.platform }}
- name: Export digest
run: |
set -euo pipefail
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
name: Merge manifest list
needs: build
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Download digests
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create and push manifest list
working-directory: /tmp/digests
env:
IMAGE: ${{ inputs.image }}
TAGS: ${{ inputs.tags }}
run: |
set -euo pipefail
args=()
while IFS= read -r tag; do
if [ -n "$tag" ]; then
args+=("-t" "$tag")
fi
done <<< "$TAGS"
refs=()
for digest in *; do
refs+=("$IMAGE@sha256:$digest")
done
docker buildx imagetools create "${args[@]}" "${refs[@]}"
- name: Inspect image
env:
IMAGE: ${{ inputs.image }}
TAGS: ${{ inputs.tags }}
run: |
set -euo pipefail
first=$(printf '%s\n' "$TAGS" | head -n1)
docker buildx imagetools inspect "$first"
echo "### Published ${{ inputs.image }}" >> "$GITHUB_STEP_SUMMARY"
printf '%s\n' "$TAGS" | sed 's/^/- /' >> "$GITHUB_STEP_SUMMARY"

69
.github/workflows/build-gateway.yaml vendored Normal file
View File

@@ -0,0 +1,69 @@
# Development channel for the example gateway image: a merge that touches the
# gateway publishes ghcr.io/<owner>/<repo>/dsh-agents-as-a-service/gateway:main.
name: Build gateway image
on:
workflow_dispatch:
push:
branches: [main]
paths:
- examples/dsh-agents-as-a-service/gateway/**
- versions.env
- .github/workflows/build-gateway.yaml
- .github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
concurrency:
group: build-gateway-${{ github.ref }}
cancel-in-progress: true
jobs:
meta:
name: Read versions
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
identity: ${{ steps.meta.outputs.identity }}
gateway_image: ${{ steps.meta.outputs.gateway_image }}
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Read versions.env
id: meta
shell: bash
run: |
set -euo pipefail
while IFS= read -r line; do
case "$line" in ''|'#'*) continue ;; esac
echo "$line" >> "$GITHUB_ENV"
done < versions.env
identity="$DSH_VERSION"
[ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION"
{
echo "identity=$identity"
echo "gateway_image=$GATEWAY_IMAGE"
} >> "$GITHUB_OUTPUT"
build:
name: Build and push gateway image
needs: meta
uses: ./.github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
with:
image: ${{ needs.meta.outputs.gateway_image }}
version: ${{ needs.meta.outputs.identity }}
context: examples/dsh-agents-as-a-service/gateway
dockerfile: Dockerfile
tags: |
${{ needs.meta.outputs.gateway_image }}:main

83
.github/workflows/build-main.yaml vendored Normal file
View File

@@ -0,0 +1,83 @@
# Development channel for the main DSH Web image: every merge to main that
# touches the image definition publishes ghcr.io/<owner>/<repo>:main.
# Releases are published by release.yaml with immutable version tags.
name: Build main image
on:
workflow_dispatch:
push:
branches: [main]
paths:
- Dockerfile
- versions.env
- .github/workflows/build-main.yaml
- .github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
concurrency:
group: build-main-${{ github.ref }}
cancel-in-progress: true
jobs:
meta:
name: Read versions
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
identity: ${{ steps.meta.outputs.identity }}
dsh_version: ${{ steps.meta.outputs.dsh_version }}
pnpm_version: ${{ steps.meta.outputs.pnpm_version }}
release_revision: ${{ steps.meta.outputs.release_revision }}
ghcr_image: ${{ steps.meta.outputs.ghcr_image }}
gateway_image: ${{ steps.meta.outputs.gateway_image }}
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Read versions.env
id: meta
shell: bash
run: |
set -euo pipefail
while IFS= read -r line; do
case "$line" in ''|'#'*) continue ;; esac
echo "$line" >> "$GITHUB_ENV"
done < versions.env
identity="$DSH_VERSION"
[ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION"
{
echo "identity=$identity"
echo "dsh_version=$DSH_VERSION"
echo "pnpm_version=$PNPM_VERSION"
echo "release_revision=$RELEASE_REVISION"
echo "ghcr_image=$GHCR_IMAGE"
echo "gateway_image=$GATEWAY_IMAGE"
} >> "$GITHUB_OUTPUT"
build:
name: Build and push main image
needs: meta
uses: ./.github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
with:
image: ${{ needs.meta.outputs.ghcr_image }}
version: ${{ needs.meta.outputs.identity }}
context: .
dockerfile: Dockerfile
build-args: |
DSH_VERSION=${{ needs.meta.outputs.dsh_version }}
RELEASE_REVISION=${{ needs.meta.outputs.release_revision }}
PNPM_VERSION=${{ needs.meta.outputs.pnpm_version }}
VCS_REF=${{ github.sha }}
tags: |
${{ needs.meta.outputs.ghcr_image }}:main

257
.github/workflows/ci.yaml vendored Normal file
View File

@@ -0,0 +1,257 @@
# Continuous integration: lint, unit tests, manifest dry-run, gateway image
# smoke test and dependency review. Job names are the required branch-protection
# checks documented in docs/gitflow.md.
name: CI
on:
pull_request:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
name: lint
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install Python linters
run: |
set -euo pipefail
python -m pip install --upgrade pip
pip install "yamllint==1.38.0" "pyyaml==6.0.3"
- name: Install actionlint
env:
ACTIONLINT_VERSION: v1.7.12
run: |
set -euo pipefail
version="${ACTIONLINT_VERSION#v}"
curl -fsSL -o /tmp/actionlint.tar.gz "https://github.com/rhysd/actionlint/releases/download/$ACTIONLINT_VERSION/actionlint_${version}_linux_amd64.tar.gz"
tar -xzf /tmp/actionlint.tar.gz -C /tmp actionlint
install -m 0755 /tmp/actionlint /usr/local/bin/actionlint
actionlint -version
- name: Install hadolint
env:
HADOLINT_VERSION: v2.12.0
run: |
set -euo pipefail
curl -fsSL -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/$HADOLINT_VERSION/hadolint-Linux-x86_64"
install -m 0755 /tmp/hadolint /usr/local/bin/hadolint
hadolint --version
- name: Repository checks
run: |
set -euo pipefail
./scripts/check-versions.sh
./scripts/check-examples.sh
python scripts/check-manifests.py
- name: Shellcheck
run: shellcheck -x build.sh scripts/*.sh examples/*/scripts/*.sh
- name: Actionlint
run: actionlint -color
- name: Yamllint
run: yamllint .
- name: Hadolint
run: |
set -euo pipefail
hadolint Dockerfile
hadolint examples/dsh-agents-as-a-service/gateway/Dockerfile
- name: Documentation links
uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2
with:
args: >-
--offline --no-progress --verbose
./README.md ./CONTRIBUTING.md ./SECURITY.md ./AGENTS.md ./docs ./examples
python:
name: python
runs-on: ubuntu-latest
timeout-minutes: 20
defaults:
run:
working-directory: examples/dsh-agents-as-a-service/gateway
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: |
examples/dsh-agents-as-a-service/gateway/requirements.txt
examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
- name: Install dependencies
run: |
set -euo pipefail
python -m pip install --upgrade pip
pip install -r requirements.txt -r requirements-dev.txt
- name: Lint
run: |
set -euo pipefail
python -m ruff check .
python -m ruff format --check .
- name: Unit tests with coverage
run: >-
python -m pytest -q
--cov=. --cov-report=xml:coverage.xml --cov-report=term
--junitxml=junit.xml
- name: Coverage summary
run: |
set -euo pipefail
{
echo "### Gateway test coverage"
python -m coverage report
} >> "$GITHUB_STEP_SUMMARY"
- name: Upload test artefacts
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: gateway-test-results
path: |
examples/dsh-agents-as-a-service/gateway/junit.xml
examples/dsh-agents-as-a-service/gateway/coverage.xml
if-no-files-found: warn
retention-days: 7
manifests:
name: manifests
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Create kind cluster
uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1
with:
version: v0.33.0
cluster_name: dsh-ci
- name: Read versions.env
run: |
set -euo pipefail
while IFS= read -r line; do
case "$line" in ''|'#'*) continue ;; esac
echo "$line" >> "$GITHUB_ENV"
done < versions.env
- name: Install agent-sandbox and Gateway API CRDs
run: |
set -euo pipefail
kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml"
kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s
kubectl apply -f "https://github.com/kubernetes-sigs/gateway-api/releases/download/$GATEWAY_API_VERSION/standard-install.yaml"
kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/httproutes.gateway.networking.k8s.io --timeout=180s
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install check dependencies
run: pip install "pyyaml==6.0.3"
- name: Validate manifest shape
run: python scripts/check-manifests.py
- name: Validate manifests (server dry-run)
run: |
set -euo pipefail
kubectl create namespace dsh-demo
shopt -s globstar nullglob
count=0
for file in examples/*/manifests/**/*.yaml; do
case "$file" in *.yaml.example) continue ;; esac
echo "== $file =="
kubectl apply --dry-run=server -f "$file"
count=$((count + 1))
done
if [ "$count" -eq 0 ]; then
echo "no manifests were validated" >&2
exit 1
fi
echo "validated $count manifests"
gateway-image:
name: gateway-image
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Build gateway image
run: docker build -t dsh-aaas-gateway:ci examples/dsh-agents-as-a-service/gateway/
- name: Smoke-test /healthz without a cluster
run: |
set -euo pipefail
cat > /tmp/kubeconfig <<'EOF'
apiVersion: v1
kind: Config
clusters:
- name: dummy
cluster:
server: https://127.0.0.1:6443
insecure-skip-tls-verify: true
contexts:
- name: dummy
context:
cluster: dummy
user: dummy
users:
- name: dummy
user: {}
current-context: dummy
EOF
docker run -d --name gateway-smoke \
-p 18080:8080 \
-e NAMESPACE=dsh-demo \
-e POOL=dsh-pool \
-e KUBECONFIG=/tmp/kubeconfig \
-v /tmp/kubeconfig:/tmp/kubeconfig:ro \
dsh-aaas-gateway:ci
trap 'docker logs gateway-smoke || true; docker rm -f gateway-smoke || true' EXIT
code=""
for _ in $(seq 1 30); do
code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:18080/healthz || true)
[ "$code" = "200" ] && break
sleep 1
done
if [ "$code" != "200" ]; then
echo "gateway /healthz returned '$code' instead of 200" >&2
exit 1
fi
echo "gateway image healthy: GET /healthz -> 200"
dependency-review:
name: dependency-review
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Dependency review
uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4

View File

@@ -57,7 +57,7 @@ jobs:
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v7 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# Add any setup steps before running the `github/codeql-action/init` action. # Add any setup steps before running the `github/codeql-action/init` action.
# This includes steps like installing compilers or runtimes (`actions/setup-node` # This includes steps like installing compilers or runtimes (`actions/setup-node`
@@ -67,7 +67,7 @@ jobs:
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@v4 uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }} build-mode: ${{ matrix.build-mode }}
@@ -96,6 +96,6 @@ jobs:
exit 1 exit 1
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4 uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with: with:
category: "/language:${{matrix.language}}" category: "/language:${{matrix.language}}"

78
.github/workflows/e2e-kind.yaml vendored Normal file
View File

@@ -0,0 +1,78 @@
# Weekly end-to-end run of the agents-as-a-service example against a real kind
# cluster with agent-sandbox installed. Scheduled (not PR-blocking) so upstream
# controller churn does not block merges; also available on demand.
name: E2E (kind)
on:
schedule:
- cron: "0 6 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: e2e-kind
cancel-in-progress: false
jobs:
e2e:
name: Example walkthrough on kind
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Create kind cluster
uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1
with:
version: v0.33.0
cluster_name: dsh-e2e
- name: Read versions.env
run: |
set -euo pipefail
while IFS= read -r line; do
case "$line" in ''|'#'*) continue ;; esac
echo "$line" >> "$GITHUB_ENV"
done < versions.env
- name: Install agent-sandbox
run: |
set -euo pipefail
kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml"
kubectl -n agent-sandbox-system rollout status deployment --timeout=180s
kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s
- name: Set up Python (WebSocket checks)
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install WebSocket client
run: pip install "websockets==17.3"
- name: Run the example walkthrough
env:
KIND_CLUSTER: dsh-e2e
run: examples/dsh-agents-as-a-service/scripts/e2e-kind.sh
- name: Collect cluster state
if: always()
run: |
set -euo pipefail
{
echo "== resources =="
kubectl get all,pvc -n dsh-demo -o wide 2>&1 || true
echo "== events =="
kubectl get events -n dsh-demo --sort-by=.lastTimestamp 2>&1 || true
echo "== gateway logs =="
kubectl -n dsh-demo logs deploy/dsh-gateway --tail=200 2>&1 || true
} > /tmp/e2e-cluster.txt
- name: Upload cluster state
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-cluster-state
path: /tmp/e2e-cluster.txt
if-no-files-found: warn
retention-days: 7

View File

@@ -1,277 +0,0 @@
# This workflow builds the multi-arch gateway image (Starlette ASGI, served by
# uvicorn on :8080) and pushes it to GHCR as
# ghcr.io/<owner>/<repo>:gateway. The DSH Web image is built by multi-build.yaml.
# It uses Docker Buildx with native AMD64 and ARM64 runners, without QEMU.
name: Gateway Build multi arch Docker Image with separate Github Runners
on:
workflow_dispatch:
push:
branches:
- main
paths:
- 'examples/dsh-agents-as-a-service/gateway/**'
- '.github/workflows/multi-build-gateway.yaml'
env:
# ghcr.io/<owner>/<repo>; this workflow always publishes the `:gateway` tag.
GHCR_IMAGE: ghcr.io/${{ github.repository }}
permissions:
# Global permissions for the workflow, which can be overridden at the job level
contents: read
concurrency:
# This concurrency group ensures that only one job in the group runs at a time.
# If a new job is triggered, the previous one will be canceled.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# The build job builds the Docker image for each platform specified in the matrix.
build:
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm64
# The matrix includes two platforms: linux/amd64 and linux/arm64.
# The build job will run for each platform in the matrix.
permissions:
# Permissions for the build job, which can be overridden at the step level
# The permissions are set to allow the job to write to the GitHub Container Registry (GHCR) and read from the repository.
attestations: write
actions: read
checks: write
contents: write
deployments: none
id-token: write
issues: read
discussions: read
packages: write
pages: none
pull-requests: read
repository-projects: read
security-events: read
statuses: read
runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' }}
# The job runs on different runners based on the platform.
# For linux/amd64, it runs on the latest Ubuntu runner.
# For linux/arm64, it runs on an Ubuntu 24.04 ARM runner.
# The runner is selected based on the platform specified in the matrix.
name: Build Docker image for ${{ matrix.platform }}
steps:
-
name: Prepare environment for current platform
# This step sets up the environment for the current platform being built.
# It replaces the '/' character in the platform name with '-' and sets it as an environment variable.
# This is useful for naming artifacts and other resources that cannot contain '/'.
# The environment variable PLATFORMS_PAIR will be used later in the workflow.
id: prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# This step checks out the code from the repository.
# It uses the actions/checkout action to clone the repository into the runner's workspace.
- name: Docker meta default
# This step generates metadata for the Docker image.
# It uses the docker/metadata-action to create metadata based on the repository information.
# The metadata includes information such as the image name, tags, and labels.
# The metadata will be used later in the workflow to build and push the Docker image.
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.GHCR_IMAGE }}
tags: |
type=raw,value=gateway
- name: Set up Docker Context for Buildx
# This step sets up a Docker context for Buildx.
# It creates a new context named "builders" that will be used for building the Docker image.
# The context allows Buildx to use the Docker daemon for building images.
id: buildx-context
run: |
docker context create builders
- name: Set up Docker Buildx
# This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit.
# It uses the docker/setup-buildx-action to configure Buildx with the specified context and platforms.
# The platforms are specified in the matrix and will be used for building the Docker image.
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
with:
endpoint: builders
platforms: ${{ matrix.platform }}
- name: Login to GitHub Container Registry
# This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action.
# It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication.
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest
# This step builds and pushes the Docker image using Buildx.
# It uses the docker/build-push-action to build the image with the specified context and platforms.
# The image is built with the labels and annotations generated in the previous steps.
# The outputs are configured to push the image by digest, which allows for better caching and versioning.
# The cache-from and cache-to options are used to enable caching for the build process.
# The cache is stored in GitHub Actions cache and is scoped to the repository, branch, and platform.
id: build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
env:
DOCKER_BUILDKIT: 1
with:
context: ./examples/dsh-agents-as-a-service/gateway/
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
annotations: ${{ steps.meta.outputs.annotations }}
outputs: type=image,name=${{ env.GHCR_IMAGE }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true
cache-from: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }}
cache-to: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }}
- name: Export digest
# This step exports the digest of the built image to a file.
# It creates a directory in /tmp/digests and saves the digest of the image to a file.
# The digest is obtained from the output of the build step.
# The digest is used to uniquely identify the built image and can be used for further processing or verification.
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
# This step uploads the digest file to the GitHub Actions artifact storage.
# It uses the actions/upload-artifact action to upload the file created in the previous step.
# The artifact is named digests-${{ env.PLATFORM_PAIR }}, where PLATFORM_PAIR is the platform name with '/' replaced by '-'.
# The artifact is retained for 1 day, and if no files are found, it will throw an error.
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
# This job merges the Docker manifests for the different platforms built in the previous job.
name: Merge Docker manifests
runs-on: ubuntu-latest
permissions:
attestations: write
actions: read
checks: read
contents: read
deployments: none
id-token: write
issues: read
discussions: read
packages: write
pages: none
pull-requests: read
repository-projects: read
security-events: read
statuses: read
needs:
- build
# This job depends on the build job to complete before it starts.
# It ensures that the Docker images for all platforms are built before merging the manifests.
steps:
- name: Download digests
# This step downloads the digest files uploaded in the build job.
# It uses the actions/download-artifact action to download the artifacts with the pattern digests-*.
# The downloaded files are merged into the /tmp/digests directory.
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
- name: Docker meta
# This step generates metadata for the Docker image.
# It uses the docker/metadata-action to create metadata based on the repository information.
# The metadata includes information such as the image name, tags, and labels.
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.GHCR_IMAGE }}
annotations: |
type=org.opencontainers.image.description,value=${{ github.event.repository.description || 'No description provided' }}
tags: |
type=raw,value=gateway
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
# This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit.
with:
driver-opts: |
network=host
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
# This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action.
# It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication.
# The login is necessary to push the merged manifest list to GHCR.
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Get execution timestamp with RFC3339 format
# This step gets the current execution timestamp in RFC3339 format.
# It uses the date command to get the current UTC time and formats it as a string.
# The timestamp is used for annotating the Docker manifest list.
id: timestamp
run: |
echo "timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")" >> $GITHUB_OUTPUT
- name: Create manifest list and pushs
# This step creates a manifest list for the Docker images built for different platforms.
# It uses the docker buildx imagetools create command to create the manifest list.
# The manifest list is annotated with metadata such as description, creation timestamp, and source URL.
# The annotations are obtained from the metadata generated in the previous steps.
# The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags.
working-directory: /tmp/digests
id: manifest-annotate
continue-on-error: true
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
--annotation='index:org.opencontainers.image.description=${{ github.event.repository.description }}' \
--annotation='index:org.opencontainers.image.created=${{ steps.timestamp.outputs.timestamp }}' \
--annotation='index:org.opencontainers.image.url=${{ github.event.repository.url }}' \
--annotation='index:org.opencontainers.image.source=${{ github.event.repository.url }}' \
$(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *)
- name: Create manifest list and push without annotations
# This step creates a manifest list for the Docker images built for different platforms.
# It uses the docker buildx imagetools create command to create the manifest list.
# The manifest list is created without annotations if the previous step fails.
# The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags.
if: steps.manifest-annotate.outcome == 'failure'
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *)
- name: Inspect image
# This step inspects the created manifest list to verify its contents.
# It uses the docker buildx imagetools inspect command to display information about the manifest list.
# The inspection output will show the platforms and tags associated with the manifest list.
id: inspect
run: |
docker buildx imagetools inspect '${{ env.GHCR_IMAGE }}:${{ steps.meta.outputs.version }}'

View File

@@ -1,295 +0,0 @@
# This workflow builds a multi-arch Docker image using GitHub Actions and separated Github Runners with native support for ARM64 and AMD64 architectures, without using QEMU emulation.
# It uses Docker Buildx to build and push the image to GitHub Container Registry (GHCR).
name: Build multi arch Docker Image with separate Github Runners
on:
workflow_dispatch:
inputs:
dsh_version:
description: "DSH git ref to build (tag or branch)"
required: true
default: dsh-v0.2.1-rc.2
type: string
pnpm_version:
description: "pnpm version used to build DSH"
required: true
default: "11.28.5"
type: string
push:
branches:
- main
paths:
- 'Dockerfile'
- '.github/workflows/multi-build.yaml'
env:
# The name of the Docker image to be built and pushed to GHCR
# The image name is derived from the GitHub repository name and the GitHub Container Registry (GHCR) URL.
# The image name will be in the format: ghcr.io/<owner>/<repo>
GHCR_IMAGE: ghcr.io/${{ github.repository }}
# workflow_dispatch inputs win; pushes to main fall back to the pinned
# defaults. Both values flow into the Docker build args and the image tags.
DSH_VERSION: ${{ inputs.dsh_version || 'dsh-v0.2.1-rc.2' }}
PNPM_VERSION: ${{ inputs.pnpm_version || '11.28.5' }}
permissions:
# Global permissions for the workflow, which can be overridden at the job level
contents: read
concurrency:
# This concurrency group ensures that only one job in the group runs at a time.
# If a new job is triggered, the previous one will be canceled.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# The build job builds the Docker image for each platform specified in the matrix.
build:
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm64
# The matrix includes two platforms: linux/amd64 and linux/arm64.
# The build job will run for each platform in the matrix.
permissions:
# Permissions for the build job, which can be overridden at the step level
# The permissions are set to allow the job to write to the GitHub Container Registry (GHCR) and read from the repository.
attestations: write
actions: read
checks: write
contents: write
deployments: none
id-token: write
issues: read
discussions: read
packages: write
pages: none
pull-requests: read
repository-projects: read
security-events: read
statuses: read
runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' }}
# The job runs on different runners based on the platform.
# For linux/amd64, it runs on the latest Ubuntu runner.
# For linux/arm64, it runs on an Ubuntu 24.04 ARM runner.
# The runner is selected based on the platform specified in the matrix.
name: Build Docker image for ${{ matrix.platform }}
steps:
-
name: Prepare environment for current platform
# This step sets up the environment for the current platform being built.
# It replaces the '/' character in the platform name with '-' and sets it as an environment variable.
# This is useful for naming artifacts and other resources that cannot contain '/'.
# The environment variable PLATFORMS_PAIR will be used later in the workflow.
id: prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# This step checks out the code from the repository.
# It uses the actions/checkout action to clone the repository into the runner's workspace.
- name: Docker meta default
# This step generates metadata for the Docker image.
# It uses the docker/metadata-action to create metadata based on the repository information.
# The metadata includes information such as the image name, tags, and labels.
# The metadata will be used later in the workflow to build and push the Docker image.
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.GHCR_IMAGE }}
- name: Set up Docker Context for Buildx
# This step sets up a Docker context for Buildx.
# It creates a new context named "builders" that will be used for building the Docker image.
# The context allows Buildx to use the Docker daemon for building images.
id: buildx-context
run: |
docker context create builders
- name: Set up Docker Buildx
# This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit.
# It uses the docker/setup-buildx-action to configure Buildx with the specified context and platforms.
# The platforms are specified in the matrix and will be used for building the Docker image.
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
with:
endpoint: builders
platforms: ${{ matrix.platform }}
- name: Login to GitHub Container Registry
# This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action.
# It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication.
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest
# This step builds and pushes the Docker image using Buildx.
# It uses the docker/build-push-action to build the image with the specified context and platforms.
# The image is built with the labels and annotations generated in the previous steps.
# The outputs are configured to push the image by digest, which allows for better caching and versioning.
# The cache-from and cache-to options are used to enable caching for the build process.
# The cache is stored in GitHub Actions cache and is scoped to the repository, branch, and platform.
id: build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
env:
DOCKER_BUILDKIT: 1
DSH_PORT: 3080
with:
context: .
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
DSH_VERSION=${{ env.DSH_VERSION }}
PNPM_VERSION=${{ env.PNPM_VERSION }}
annotations: ${{ steps.meta.outputs.annotations }}
outputs: type=image,name=${{ env.GHCR_IMAGE }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true
cache-from: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }}
cache-to: type=gha,scope=${{ github.repository }}-${{ github.ref_name }}-${{ matrix.platform }}
- name: Export digest
# This step exports the digest of the built image to a file.
# It creates a directory in /tmp/digests and saves the digest of the image to a file.
# The digest is obtained from the output of the build step.
# The digest is used to uniquely identify the built image and can be used for further processing or verification.
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
# This step uploads the digest file to the GitHub Actions artifact storage.
# It uses the actions/upload-artifact action to upload the file created in the previous step.
# The artifact is named digests-${{ env.PLATFORM_PAIR }}, where PLATFORM_PAIR is the platform name with '/' replaced by '-'.
# The artifact is retained for 1 day, and if no files are found, it will throw an error.
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
# This job merges the Docker manifests for the different platforms built in the previous job.
name: Merge Docker manifests
runs-on: ubuntu-latest
permissions:
attestations: write
actions: read
checks: read
contents: read
deployments: none
id-token: write
issues: read
discussions: read
packages: write
pages: none
pull-requests: read
repository-projects: read
security-events: read
statuses: read
needs:
- build
# This job depends on the build job to complete before it starts.
# It ensures that the Docker images for all platforms are built before merging the manifests.
steps:
- name: Download digests
# This step downloads the digest files uploaded in the build job.
# It uses the actions/download-artifact action to download the artifacts with the pattern digests-*.
# The downloaded files are merged into the /tmp/digests directory.
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true
- name: Docker meta
# This step generates metadata for the Docker image.
# It uses the docker/metadata-action to create metadata based on the repository information.
# The metadata includes information such as the image name, tags, and labels.
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.GHCR_IMAGE }}
annotations: |
type=org.opencontainers.image.description,value=${{ github.event.repository.description || 'No description provided' }}
tags: |
type=raw,value=main,enable=${{ github.ref_name == 'main' }}
type=raw,value=latest,enable=${{ github.ref_name == 'main' }}
type=raw,value=${{ env.DSH_VERSION }},enable=${{ github.ref_name == 'main' || github.event_name == 'workflow_dispatch' }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
# This step sets up Docker Buildx, which is a Docker CLI plugin for extended build capabilities with BuildKit.
with:
driver-opts: |
network=host
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
# This step logs in to the GitHub Container Registry (GHCR) using the docker/login-action.
# It uses the GitHub actor's username and the GITHUB_TOKEN secret for authentication.
# The login is necessary to push the merged manifest list to GHCR.
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Get execution timestamp with RFC3339 format
# This step gets the current execution timestamp in RFC3339 format.
# It uses the date command to get the current UTC time and formats it as a string.
# The timestamp is used for annotating the Docker manifest list.
id: timestamp
run: |
echo "timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")" >> $GITHUB_OUTPUT
- name: Create manifest list and pushs
# This step creates a manifest list for the Docker images built for different platforms.
# It uses the docker buildx imagetools create command to create the manifest list.
# The manifest list is annotated with metadata such as description, creation timestamp, and source URL.
# The annotations are obtained from the metadata generated in the previous steps.
# The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags.
working-directory: /tmp/digests
id: manifest-annotate
continue-on-error: true
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
--annotation='index:org.opencontainers.image.description=${{ github.event.repository.description }}' \
--annotation='index:org.opencontainers.image.created=${{ steps.timestamp.outputs.timestamp }}' \
--annotation='index:org.opencontainers.image.url=${{ github.event.repository.url }}' \
--annotation='index:org.opencontainers.image.source=${{ github.event.repository.url }}' \
$(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *)
- name: Create manifest list and push without annotations
# This step creates a manifest list for the Docker images built for different platforms.
# It uses the docker buildx imagetools create command to create the manifest list.
# The manifest list is created without annotations if the previous step fails.
# The manifest list is pushed to the GitHub Container Registry (GHCR) with the specified tags.
if: steps.manifest-annotate.outcome == 'failure'
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.GHCR_IMAGE }}@sha256:%s ' *)
- name: Inspect image
# This step inspects the created manifest list to verify its contents.
# It uses the docker buildx imagetools inspect command to display information about the manifest list.
# The inspection output will show the platforms and tags associated with the manifest list.
id: inspect
run: |
docker buildx imagetools inspect '${{ env.GHCR_IMAGE }}:${{ steps.meta.outputs.version }}'

38
.github/workflows/pr-title.yaml vendored Normal file
View File

@@ -0,0 +1,38 @@
# Enforce Conventional Commit pull request titles so git-cliff can group
# release notes. The PR is squash-merged, so the title becomes the commit.
name: PR title
on:
pull_request:
types: [opened, edited, synchronize, reopened]
permissions:
contents: read
pull-requests: read
jobs:
lint:
name: pr-title
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Validate title
uses: amannn/action-semantic-pull-request@e32d7e603df1aa1ba07e981f2a23455dee596825 # v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
types: |
feat
fix
docs
refactor
perf
test
build
ci
chore
revert
requireScope: false
subjectPattern: ^(?![A-Z]).+$
subjectPatternError: |
The PR title subject must start in lowercase and must not end with a period.

268
.github/workflows/release.yaml vendored Normal file
View File

@@ -0,0 +1,268 @@
# Release pipeline. Pushing a tag that matches the identity in versions.env
# publishes both images (version + latest), creates the GitHub Release from
# git-cliff, and opens a PR that refreshes CHANGELOG.md on main.
#
# The tag must be dsh-vX.Y.Z or dsh-vX.Y.Z-rN and must match versions.env;
# tags are immutable, so a broken release is fixed with a new -rN revision.
name: Release
on:
push:
tags:
- "dsh-v*"
workflow_dispatch:
inputs:
tag:
description: Existing tag to release (must match versions.env)
required: true
type: string
publish_legacy_gateway_alias:
description: Also publish the legacy :gateway alias (one release only)
required: false
default: true
type: boolean
permissions:
contents: write
packages: write
id-token: write
attestations: write
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
jobs:
meta:
name: Validate release identity
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
identity: ${{ steps.meta.outputs.identity }}
dsh_version: ${{ steps.meta.outputs.dsh_version }}
pnpm_version: ${{ steps.meta.outputs.pnpm_version }}
release_revision: ${{ steps.meta.outputs.release_revision }}
ghcr_image: ${{ steps.meta.outputs.ghcr_image }}
gateway_image: ${{ steps.meta.outputs.gateway_image }}
tag: ${{ steps.meta.outputs.tag }}
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Validate tag against versions.env
id: meta
shell: bash
run: |
set -euo pipefail
tag="${{ inputs.tag || github.ref_name }}"
while IFS= read -r line; do
case "$line" in ''|'#'*) continue ;; esac
echo "$line" >> "$GITHUB_ENV"
done < versions.env
identity="$DSH_VERSION"
[ -z "$RELEASE_REVISION" ] || identity="$DSH_VERSION-$RELEASE_REVISION"
if [ "$tag" != "$identity" ]; then
echo "tag '$tag' does not match versions.env identity '$identity'" >&2
exit 1
fi
if git rev-parse -q --verify "refs/tags/$tag" >/dev/null; then
git fetch origin main '+refs/heads/release/*:refs/remotes/origin/release/*'
commit=$(git rev-list -n1 "refs/tags/$tag")
allowed=0
for ref in origin/main origin/release/*; do
if git rev-parse -q --verify "$ref" >/dev/null && git merge-base --is-ancestor "$commit" "$ref"; then
allowed=1
break
fi
done
if [ "$allowed" -ne 1 ]; then
echo "tag commit $commit is not on main or a release branch" >&2
exit 1
fi
else
echo "tag '$tag' does not exist in this repository" >&2
exit 1
fi
if ! git ls-remote --tags --exit-code "$DSH_REPO" "$DSH_VERSION" >/dev/null; then
echo "upstream ref '$DSH_VERSION' not found in $DSH_REPO" >&2
exit 1
fi
{
echo "identity=$identity"
echo "dsh_version=$DSH_VERSION"
echo "pnpm_version=$PNPM_VERSION"
echo "release_revision=$RELEASE_REVISION"
echo "ghcr_image=$GHCR_IMAGE"
echo "gateway_image=$GATEWAY_IMAGE"
echo "tag=$tag"
} >> "$GITHUB_OUTPUT"
- name: Verify repository pins
run: ./scripts/check-versions.sh
tests:
name: Unit tests and lint
needs: meta
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: examples/dsh-agents-as-a-service/gateway
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: |
examples/dsh-agents-as-a-service/gateway/requirements.txt
examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
- name: Install dependencies
run: |
set -euo pipefail
python -m pip install --upgrade pip
pip install -r requirements.txt -r requirements-dev.txt
- name: Lint
run: |
set -euo pipefail
python -m ruff check .
python -m ruff format --check .
- name: Unit tests
run: python -m pytest -q
build-main:
name: Publish main image
needs: [meta, tests]
uses: ./.github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
with:
image: ${{ needs.meta.outputs.ghcr_image }}
version: ${{ needs.meta.outputs.identity }}
context: .
dockerfile: Dockerfile
build-args: |
DSH_VERSION=${{ needs.meta.outputs.dsh_version }}
RELEASE_REVISION=${{ needs.meta.outputs.release_revision }}
PNPM_VERSION=${{ needs.meta.outputs.pnpm_version }}
VCS_REF=${{ github.sha }}
tags: |
${{ needs.meta.outputs.ghcr_image }}:${{ needs.meta.outputs.identity }}
${{ needs.meta.outputs.ghcr_image }}:latest
build-gateway:
name: Publish gateway image
needs: [meta, tests]
uses: ./.github/workflows/_build-image.yaml
permissions:
contents: read
packages: write
id-token: write
attestations: write
with:
image: ${{ needs.meta.outputs.gateway_image }}
version: ${{ needs.meta.outputs.identity }}
context: examples/dsh-agents-as-a-service/gateway
dockerfile: Dockerfile
tags: |
${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }}
${{ needs.meta.outputs.gateway_image }}:latest
legacy-gateway-alias:
name: Publish legacy :gateway alias (temporary)
needs: [meta, build-gateway]
if: github.event_name != 'workflow_dispatch' || inputs.publish_legacy_gateway_alias
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Retag gateway as the legacy package tag
env:
SOURCE: ${{ needs.meta.outputs.gateway_image }}:${{ needs.meta.outputs.identity }}
LEGACY: ${{ needs.meta.outputs.ghcr_image }}:gateway
run: |
set -euo pipefail
docker buildx imagetools create -t "$LEGACY" "$SOURCE"
echo "Deprecated alias $LEGACY now points at $SOURCE" >> "$GITHUB_STEP_SUMMARY"
publish-release:
name: Changelog and GitHub Release
needs: [meta, build-main, build-gateway]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: Install git-cliff
env:
GIT_CLIFF_VERSION: v2.14.2
run: |
set -euo pipefail
version="${GIT_CLIFF_VERSION#v}"
curl -fsSL -o /tmp/git-cliff.tar.gz "https://github.com/orhun/git-cliff/releases/download/$GIT_CLIFF_VERSION/git-cliff-$version-x86_64-unknown-linux-gnu.tar.gz"
tar -xzf /tmp/git-cliff.tar.gz -C /tmp
install -m 0755 "/tmp/git-cliff-$version/git-cliff" /usr/local/bin/git-cliff
git-cliff --version
- name: Generate CHANGELOG and release notes
env:
IDENTITY: ${{ needs.meta.outputs.identity }}
run: |
set -euo pipefail
git-cliff --tag "$IDENTITY" --output CHANGELOG.md
git-cliff --tag "$IDENTITY" --latest --strip header --output /tmp/release-notes.md
echo "### Release $IDENTITY" >> "$GITHUB_STEP_SUMMARY"
head -n 40 /tmp/release-notes.md >> "$GITHUB_STEP_SUMMARY"
- name: Create GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
IDENTITY: ${{ needs.meta.outputs.identity }}
run: |
set -euo pipefail
if gh release view "$IDENTITY" >/dev/null 2>&1; then
echo "release $IDENTITY already exists; skipping" >&2
else
gh release create "$IDENTITY" --title "$IDENTITY" --notes-file /tmp/release-notes.md --verify-tag
fi
- name: Open CHANGELOG pull request
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
IDENTITY: ${{ needs.meta.outputs.identity }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
if git diff --quiet -- CHANGELOG.md; then
echo "CHANGELOG.md already up to date"
exit 0
fi
branch="chore/changelog-$IDENTITY"
git switch -c "$branch"
git add CHANGELOG.md
git commit -m "chore(changelog): refresh for $IDENTITY"
git push origin "$branch"
gh pr create \
--base main \
--head "$branch" \
--title "chore(changelog): refresh for $IDENTITY" \
--body "Generated by git-cliff for $IDENTITY. Do not edit CHANGELOG.md by hand."

View File

@@ -1,137 +0,0 @@
name: Validate
# Validates the Kubernetes manifests against the real agent-sandbox CRD
# schema (server-side dry-run on a throwaway kind cluster), runs the gateway
# unit tests + lint, and builds/smoke-tests the gateway image without a
# cluster. Runs on every PR and on pushes to main.
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
validate-manifests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Create kind cluster
uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1
with:
version: v0.33.0
cluster_name: dsh-validate
- name: Install agent-sandbox (extensions)
run: |
VERSION=v1.0.7
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/sandbox-with-extensions.yaml
kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s
kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s
- name: Validate manifests (server dry-run)
run: |
set -euo pipefail
kubectl create namespace dsh-demo
for f in examples/dsh-agents-as-a-service/*.yaml; do
echo "== $f =="
kubectl apply --dry-run=server -f "$f"
done
kubectl apply --dry-run=server -f deploy/k8s/sandbox-template.yaml
echo "all manifests validated"
gateway-tests:
# The gateway lives in examples/dsh-agents-as-a-service/gateway/. Its
# runtime pins and dev pins are kept separate so the image only ships
# requirements.txt while CI installs both files.
name: Gateway unit tests and lint
runs-on: ubuntu-latest
defaults:
run:
working-directory: examples/dsh-agents-as-a-service/gateway
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: |
examples/dsh-agents-as-a-service/gateway/requirements.txt
examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r requirements-dev.txt
- name: Lint (ruff)
run: python -m ruff check .
- name: Run gateway unit tests
run: python -m pytest -v
gateway-image:
# No cluster and no real credentials: the gateway must start and answer
# /healthz with a dummy kubeconfig, which catches broken imports, missing
# runtime deps, and an unresponsive ASGI entrypoint.
name: Build and smoke-test gateway image
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Build gateway image
run: |
docker build \
-t dsh-aaas-gateway:ci \
examples/dsh-agents-as-a-service/gateway/
- name: Smoke-test /healthz without a cluster
run: |
set -euo pipefail
cat > /tmp/kubeconfig <<'EOF'
apiVersion: v1
kind: Config
clusters:
- name: dummy
cluster:
server: https://127.0.0.1:6443
insecure-skip-tls-verify: true
contexts:
- name: dummy
context:
cluster: dummy
user: dummy
users:
- name: dummy
user: {}
current-context: dummy
EOF
docker run -d --name gateway-smoke \
-p 18080:8080 \
-e NAMESPACE=dsh-demo \
-e POOL=dsh-pool \
-e KUBECONFIG=/tmp/kubeconfig \
-v /tmp/kubeconfig:/tmp/kubeconfig:ro \
dsh-aaas-gateway:ci
trap 'docker logs gateway-smoke || true; docker rm -f gateway-smoke || true' EXIT
code=""
for _ in $(seq 1 30); do
code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:18080/healthz || true)
[ "$code" = "200" ] && break
sleep 1
done
if [ "$code" != "200" ]; then
echo "gateway /healthz returned '$code' instead of 200" >&2
exit 1
fi
echo "gateway image healthy: GET /healthz -> 200"

5
.gitignore vendored
View File

@@ -159,3 +159,8 @@ __pycache__/
htmlcov/ htmlcov/
.venv/ .venv/
venv/ venv/
# Local repository tooling
.uvcache/
coverage.xml

11
.hadolint.yaml Normal file
View File

@@ -0,0 +1,11 @@
# Container lint policy. Base images are supplied as build args defaulting to
# pinned tags, and the image is rebuilt on every DSH release, so unpinned
# packages inside a pinned base image are intentional.
failure-threshold: warning
ignored:
- DL3008 # apt-get packages are intentionally not version-pinned
- DL3018 # apk packages are intentionally not version-pinned
- DL3006 # FROM uses build args (NODE_BUILD_IMAGE/NODE_RUNTIME_IMAGE) from versions.env
- DL3003 # the DSH packaging step deliberately cds inside a single layer
- DL3059 # consecutive RUN layers are kept for cache reuse and readability
- DL4006 # the upstream uv installer pipes curl to sh by design

13
.lychee.toml Normal file
View File

@@ -0,0 +1,13 @@
# Link checker for README/docs. Placeholders used by the examples are excluded
# on purpose; real URLs must resolve.
verbose = "info"
no_progress = true
max_retries = 2
exclude = [
"localhost",
"127\\.0\\.0\\.1",
"example\\.com",
"your-github-org",
"mailto:",
"ghcr\\.io/.*@sha256",
]

19
.yamllint.yml Normal file
View File

@@ -0,0 +1,19 @@
extends: default
rules:
document-start: disable
line-length:
max: 200
level: error
truthy:
check-keys: false
comments-indentation: disable
comments:
min-spaces-from-content: 1
ignore: |
.venv/
.uvcache/
node_modules/
# Upstream CodeQL boilerplate: long documentation comments, linted by actionlint.
.github/workflows/codeql.yml

140
AGENTS.md Normal file
View File

@@ -0,0 +1,140 @@
# AGENTS.md
Guidance for AI coding agents (and humans) working in this repository. Read it
before changing anything; it encodes the conventions the CI enforces.
## What this repository is
- **The root is the primary artifact.** The Dockerfile packages DeepSeek
Harness Web (dsh web) as a slim, non-root, multi-arch image. Everything else
exists to support it or to demonstrate an implementation on top of it.
- **Every implementation is a self-contained example** under examples/<name>/.
There is exactly one example for the agents-as-a-service platform; do not
split it into parallel directories.
- **versions.env is the single source of truth** for the DSH ref, pnpm, the
control-plane versions and the image names.
- **Docs describe the shipped behavior**, not intentions. If you change a
manifest, port, env var or workflow, update the docs in the same change.
- **CHANGELOG.md is generated** by git-cliff from Conventional Commits. Never
edit it by hand.
## Golden rules
1. Keep the Dockerfile working for both linux/amd64 and linux/arm64; it must
stay the first-class artifact.
2. Never hard-code a version, image tag or registry path. Edit versions.env and
run make set-version when bumping.
3. Never commit a real secret or usable credential. Secret-shaped YAML lives in
*.yaml.example files; keys are documented in .env.sample only.
4. Every manifest under examples/*/manifests/ must parse and pass a server-side
dry-run in CI. Keep manifests numbered (NN-*.yaml), and keep exposure
variants under manifests/exposure/<variant>/ documented by the single
manifests/exposure/README.md (variants are mutually exclusive).
5. Do not edit CHANGELOG.md, and do not move or delete published tags.
Releasing is documented in docs/versioning.md.
6. Keep the gateway HTTP/WebSocket contract stable unless the change is
explicitly about it; clients and examples depend on it.
## Repository map
| Path | What lives there |
| --- | --- |
| Dockerfile | Main DSH Web image (build + runtime stages, entrypoint) |
| versions.env | Every pinned version and image name |
| Makefile | Canonical local entrypoint |
| build.sh | Deprecated shim that forwards to make |
| docs/ | main-image.md, examples.md, gitflow.md, versioning.md |
| examples/ | One implementation per directory (contract in docs/examples.md) |
| examples/dsh-agents-as-a-service/ | The multi-tenant platform example (manifests, gateway, docs, scripts) |
| examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart |
| scripts/ | check-versions.sh, check-examples.sh, check-manifests.py, set-version.sh |
| .github/workflows/ | ci.yaml, reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml |
## Commands
make check # versions, example layout and manifest YAML (run before every commit)
make test # gateway unit tests (109+ tests)
make lint # ruff + shellcheck (+ yamllint/actionlint when installed)
make image # build the main DSH Web image (slow: clones and builds DSH)
make gateway # build the example gateway image (fast)
make e2e # full kind walkthrough; needs a cluster with agent-sandbox
make changelog # regenerate CHANGELOG.md with git-cliff (release only)
make release-dry-run
make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=rN
No cluster is needed for check, test, lint, gateway or image. Only make e2e
needs kind plus the agent-sandbox CRDs (see the example README).
## Testing expectations per change
| Change | Minimum verification |
| --- | --- |
| Documentation only | make check; verify relative links |
| Example manifest | make check; kubectl apply --dry-run=server in CI |
| Gateway code | make test; make lint; add or update tests |
| Dockerfile / versions.env | make check; make image (amd64) and a container smoke test |
| Workflow | actionlint in CI; keep actions SHA-pinned; no hard-coded versions |
| New example | make check; all files described in docs/examples.md |
## Versioning and releases
- Release identity = DSH_VERSION, optionally with a repository revision suffix
for re-releases at the same DSH ref (first release: dsh-v0.2.1-rc.2-r2).
- The current identity is dsh-v0.2.1-rc.2-r2 (stored in versions.env). The
dsh-v0.2.1-rc.2 image tag already exists and tags are immutable, which is why
the first release under this scheme carries -r2.
- Channels: release tags are immutable; latest follows the newest release;
main is the development channel for both image packages.
- A broken release is fixed by publishing -r(N+1), never by re-tagging.
- The full procedure, compatibility table and rollback rules are in
docs/versioning.md.
## Branch and PR conventions
- Trunk-based with release branches, documented in docs/gitflow.md.
- Branch names: feat/, fix/, docs/, chore/, refactor/, test/.
- Conventional Commit subjects and PR titles (feat(scope): ..., fix(scope): ...).
- PRs are squash-merged; main stays linear and protected.
- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL,
dependency-review.
- Keep diffs scoped; update docs with the code; never mix a version bump with
unrelated changes.
## Adding an example
1. Read docs/examples.md and examples/README.md.
2. Create examples/<name>/ with README.md, manifests/NN-*.yaml, optional
manifests/exposure/<variant>/ (plus README), optional components with their
own Dockerfile/README/tests, and scripts/ (executable, idempotent,
self-cleaning).
3. Publish component images under <registry>/<owner>/<repo>/<example>/<component>.
4. Add the example to examples/README.md; scripts/check-examples.sh enforces
the layout.
## Security notes
- AUTH_PROXY_HEADER is opt-in. Only enable it when the gateway Service is
reachable exclusively through the authenticating proxy; otherwise any client
can spoof X-Forwarded-User.
- The example gateway reads the DSH launch token from pod logs. Treat pod logs
as sensitive and keep its RBAC namespace-scoped.
- The example shares one DEEPSEEK_API_KEY across sandboxes; do not describe it
as tenant-isolated.
- TLS terminates outside the gateway; set COOKIE_SECURE=true for HTTPS and keep
the base manifest safe for local HTTP.
## Ask before
- Changing release workflows, tag formats or published image names.
- Editing the gateway's public routes or auth semantics.
- Adding a new top-level tool, service or dependency to the main image.
- Rewriting generated files (CHANGELOG.md) or git history.
## Where to look first
- [docs/main-image.md](docs/main-image.md) - container contract and tags
- [docs/examples.md](docs/examples.md) - example authoring rules
- [docs/gitflow.md](docs/gitflow.md) - branches, PRs, required checks
- [docs/versioning.md](docs/versioning.md) - versioning, changelog, releases
- [CONTRIBUTING.md](CONTRIBUTING.md) - development setup
- [SECURITY.md](SECURITY.md) - vulnerability reporting and threat notes

46
CHANGELOG.md Normal file
View File

@@ -0,0 +1,46 @@
# Changelog
All notable changes to this repository are documented here.
Generated by git-cliff from Conventional Commits; do not edit this file by hand.
## [Unreleased]
### Added
- **gateway-api:** Expose dsh-gateway externally via Traefik Gateway API (#12) (7d3faf0)
- **gateway-api:** GitHub SSO via oauth2-proxy in front of dsh-gateway (#13) (3f46279)
- **gateway:** SSO landing auto-connects and offers recreate sandbox (#15) (ed1398d)
- **repo:** Restructure around the dsh-harness image and one example per implementation (e6e109d)
### Fixed
- **Dockerfile:** Update runtime dependencies and enable corepack for pnpm (6d97f61)
- **gateway-api:** Correct oauth2-proxy cookie-secret generation (#14) (73a65d9)
### Other
- Dockerfile extra tools (2b47afa)
- Modify gateway.yaml for image policy and timeout values (53e55fc)
## [v0.0.1] - 2026-10-21
### Other
- Initial commit (f3d4312)
- Update Dockerfile, README, and Kubernetes manifests for DSH Web deployment (#1) (0d66cba)
- Add DeepSeek Harness Web agents-as-a-service platform pattern (#2) (98cb547)
- Update action versions and cluster version (d9cb265)
- Add Python to CodeQL workflow languages (b0f992f)
- Add multi-arch Docker image build workflow with GitHub Actions (de11d34)
- Add step to create manifest list for Docker images (c9be929)
- Make DSH agents-as-a-service browser-usable; fix probe crash-loop (#3) (6301c21)
- Fix idle sweeper suspending claims the gateway does not own (#4) (29eefd6)
- Fix DSH crash-loop after suspend/resume (fsGroup chmod) (#5) (a8ef211)
- Add a startup probe so slow DSH cold starts are not killed (#6) (c193ed2)
- Normalise the pod model returned by CoreV1Api (#7) (9dea4a7)
- Forward the upstream query byte-for-byte (#8) (6876406)
- Do not read-time-out proxied upstream streams (#9) (163abae)
- Dial the sandbox pod IP for WebSocket upgrades (#10) (abe5b4d)
- Handshake under the pod-IP authority for WebSocket upgrades (#11) (0a61e57)

46
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,46 @@
# Code of Conduct
This project adopts the Contributor Covenant, version 2.1:
https://www.contributor-covenant.org/version/2/1/code_of_conduct/
## Our standards
We are committed to a welcoming, harassment-free experience for everyone,
regardless of age, body size, visible or invisible disability, ethnicity, sex
characteristics, gender identity and expression, level of experience,
education, socio-economic status, nationality, personal appearance, race,
religion, or sexual identity and orientation.
Examples of behavior that contributes to a positive environment:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes
- Focusing on what is best for the community and the project
Examples of unacceptable behavior:
- Sexualized language or imagery, and sexual attention or advances
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information without explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement
Maintainers are responsible for clarifying and enforcing these standards and
may remove, edit, or reject comments, commits, code, issues, and other
contributions that do not align with this Code of Conduct.
Report abusive, harassing, or otherwise unacceptable behavior to the
maintainers through a private GitHub Security Advisory or by contacting the
repository owner on GitHub. All complaints will be reviewed and investigated
promptly and fairly. Maintainers must respect the privacy and security of the
reporter.
## Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1,
available at https://www.contributor-covenant.org/version/2/1/code_of_conduct/

88
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,88 @@
# Contributing
Thanks for improving deepseek-kubernetes-sandbox. This repository packages
DeepSeek Harness Web as a container and ships self-contained implementation
examples around it. The root is the container; every implementation lives
under examples/.
## Repository layout
| Path | Purpose |
| --- | --- |
| Dockerfile | The primary artifact: the DSH Web image |
| versions.env | Single source of truth for every pinned version |
| Makefile | Canonical local entrypoint (build, test, lint, check, e2e) |
| docs/ | Repository-wide docs: main image, example contract, gitflow, versioning |
| examples/ | One self-contained implementation per directory |
| scripts/ | Repository checks (versions, example layout, manifests, version bump) |
| .github/ | CI, releases, dependency updates, templates |
## Requirements
- Docker (for image builds)
- Python 3.12+ (gateway tests and repository checks)
- make and git
- kubectl and kind only for the end-to-end example
## Local workflow
git switch -c feat/<short-slug>
make check # versions, example layout, manifest YAML
make test # gateway unit tests
make lint # ruff, shellcheck (+ yamllint/actionlint when installed)
make image # build the main DSH Web image
make gateway # build the example gateway image
make e2e # full kind walkthrough (optional, slow)
Install the gateway dev dependencies once:
python -m pip install -r examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
## Pull requests
- Use a Conventional Commit title, for example feat(gateway): add wake metrics
or docs(examples): clarify SSO trust boundary. PR titles are linted.
- Keep the diff scoped. Squash merge into main; history stays linear.
- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL,
and dependency-review. Branch protection is described in docs/gitflow.md.
- Update the docs that the change affects; docs are part of the change, not a
follow-up.
- Never commit a real secret. Secret-shaped YAML belongs in a *.yaml.example
file, and versions live in versions.env only.
## Changing versions
make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=r2
make check
CHANGELOG.md is generated by git-cliff from Conventional Commits. Do not edit
it by hand; the release workflow regenerates it.
## Adding an example
Read docs/examples.md, then add examples/<name>/ with:
- README.md (what it demonstrates, prerequisites, steps, teardown)
- manifests/NN-*.yaml (numbered, plain kubectl)
- optional manifests/exposure/<variant>/ with its own README.md
- optional components (a Dockerfile plus README and tests per component)
- scripts/ with executable, idempotent, self-cleaning scripts
- docs/ for deeper material
List it in examples/README.md. scripts/check-examples.sh enforces this layout.
## Documentation
- Root README.md is the front door: main image first, then examples.
- docs/ holds repository-wide material; example-specific material belongs in
the example directory.
## Branches and releases
See docs/gitflow.md for the branch model and docs/versioning.md for the
release identity, channels and rollback rules.
## Security
See SECURITY.md. Report vulnerabilities privately through GitHub Security
Advisories; do not open a public issue.

View File

@@ -1,12 +1,23 @@
# syntax=docker/dockerfile:1.7 # syntax=docker/dockerfile:1.7
#
# The primary artifact of this repository: DeepSeek Harness Web (dsh web),
# packaged as a slim, non-root, multi-arch runtime image.
#
# Build args (all pinned in versions.env, verified by scripts/check-versions.sh):
# DSH_REPO upstream git remote to package
# DSH_VERSION upstream git ref to build (tag or branch)
# RELEASE_REVISION optional repository revision label (r2, r3, ...)
# PNPM_VERSION pnpm used to build DSH
# VCS_REF this repository's commit, recorded in OCI labels
# NODE_BUILD_IMAGE node image used to build DSH
# NODE_RUNTIME_IMAGE node image used at runtime
ARG NODE_BUILD_IMAGE=docker.io/node:24-trixie ARG NODE_BUILD_IMAGE=docker.io/node:24-trixie
ARG NODE_RUNTIME_IMAGE=docker.io/node:24-trixie-slim ARG NODE_RUNTIME_IMAGE=docker.io/node:24-trixie-slim
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Build stage # Build stage: clone DSH at the pinned ref, build, and package the CLI.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
FROM ${NODE_BUILD_IMAGE} AS build FROM $NODE_BUILD_IMAGE AS build
ARG DSH_REPO=https://github.com/deepseek-ai/deepseek-harness.git ARG DSH_REPO=https://github.com/deepseek-ai/deepseek-harness.git
ARG DSH_VERSION=dsh-v0.2.1-rc.2 ARG DSH_VERSION=dsh-v0.2.1-rc.2
@@ -16,11 +27,11 @@ ENV CI=true \
NODE_OPTIONS=--max-old-space-size=4096 NODE_OPTIONS=--max-old-space-size=4096
RUN corepack enable \ RUN corepack enable \
&& corepack prepare "pnpm@${PNPM_VERSION}" --activate && corepack prepare "pnpm@$PNPM_VERSION" --activate
WORKDIR /src WORKDIR /src
RUN git clone --depth 1 --branch "${DSH_VERSION}" "${DSH_REPO}" . RUN git clone --depth 1 --branch "$DSH_VERSION" "$DSH_REPO" .
RUN pnpm install --frozen-lockfile RUN pnpm install --frozen-lockfile
@@ -44,22 +55,30 @@ RUN cd /opt/dsh \
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Runtime stage # Runtime stage
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
FROM ${NODE_RUNTIME_IMAGE} AS runtime FROM $NODE_RUNTIME_IMAGE AS runtime
ARG DSH_VERSION ARG DSH_VERSION=dsh-v0.2.1-rc.2
ARG RELEASE_REVISION=r2
ARG PNPM_VERSION=11.28.5 ARG PNPM_VERSION=11.28.5
ARG VCS_REF=unknown
LABEL org.opencontainers.image.title="DeepSeek Harness Web packaged by SREDevOps.org" \ LABEL org.opencontainers.image.title="DeepSeek Harness Web packaged by SREDevOps.org" \
org.opencontainers.image.version="${DSH_VERSION}" \ org.opencontainers.image.description="DeepSeek Harness Web (dsh web) as a slim, non-root, Kubernetes-ready image" \
org.opencontainers.image.source="https://github.com/sredevopsorg/deepseek-kubernetes-sandbox" org.opencontainers.image.source="https://github.com/sredevopsorg/deepseek-kubernetes-sandbox" \
org.opencontainers.image.version="$DSH_VERSION" \
org.opencontainers.image.revision="$VCS_REF" \
org.opencontainers.image.licenses="MIT" \
io.sredevopsorg.dsh.version="$DSH_VERSION" \
io.sredevopsorg.release.revision="$RELEASE_REVISION"
ENV NODE_ENV=production \ ENV NODE_ENV=production \
HOME=/home/node \ HOME=/home/node \
DSH_HOME=/home/node/.dsh \ DSH_HOME=/home/node/.dsh \
DSH_PORT=3080 \ DSH_PORT=3080 \
PATH="/app/node_modules/.bin:${PATH}" PATH="/app/node_modules/.bin:$PATH"
# git/ripgrep are useful for DSH agent tools; tini handles signals. # git/ripgrep/jq/yq/curl/gh are deliberate agent tooling parity: a sandboxed DSH
# instance expects the usual developer CLI. tini reaps signals for a clean stop.
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends \ && apt-get install -y --no-install-recommends \
ca-certificates \ ca-certificates \
@@ -73,9 +92,9 @@ RUN apt-get update \
gh \ gh \
&& rm -rf /var/lib/apt/lists/* \ && rm -rf /var/lib/apt/lists/* \
&& curl -LsSf https://astral.sh/uv/install.sh | sh \ && curl -LsSf https://astral.sh/uv/install.sh | sh \
&& install -d -o node -g node -m 0755 /workspace "${DSH_HOME}" \ && install -d -o node -g node -m 0755 /workspace "$DSH_HOME" \
&& corepack enable \ && corepack enable \
&& corepack prepare "pnpm@${PNPM_VERSION}" --activate \ && corepack prepare "pnpm@$PNPM_VERSION" --activate \
&& pnpm --version && pnpm --version
COPY --from=build --chown=node:node /opt/dsh /app COPY --from=build --chown=node:node /opt/dsh /app
@@ -98,7 +117,7 @@ if [ -n "$DSH_PUBLIC_URL" ]; then
set -- --public-url "$DSH_PUBLIC_URL" "$@" set -- --public-url "$DSH_PUBLIC_URL" "$@"
fi fi
# Additional trusted hosts (e.g. the public gateway host behind an Ingress). # Additional trusted hosts (e.g. the public gateway host behind a proxy).
# DSH Host/Origin-fences /api: without this, requests forwarded with the # DSH Host/Origin-fences /api: without this, requests forwarded with the
# browser's Host header are rejected. Accepts comma- or space-separated names; # browser's Host header are rejected. Accepts comma- or space-separated names;
# localhost/127.0.0.1 are always trusted below. # localhost/127.0.0.1 are always trusted below.
@@ -127,4 +146,4 @@ USER node
EXPOSE 3080 EXPOSE 3080
VOLUME ["/home/node/.dsh"] VOLUME ["/home/node/.dsh"]
STOPSIGNAL SIGTERM STOPSIGNAL SIGTERM
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/dsh-entrypoint"] ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/dsh-entrypoint"]

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Nicolás Georger
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,37 +1,82 @@
# Local build and test helpers. Every variable can be overridden, e.g. # Canonical local entrypoint. Includes versions.env so there is exactly one
# make image DSH_VERSION=dsh-v0.2.1-rc.2 IMAGE=dsh-web:dev # place to bump a version (run "make check" to verify pins across the repo).
# make test #
DSH_VERSION ?= dsh-v0.2.1-rc.2 # make image build the main DSH Web image
PNPM_VERSION ?= 11.28.5 # make gateway build the example gateway image
IMAGE ?= dsh-web:0.2.1-rc.2 # make test gateway unit tests
GATEWAY_IMAGE ?= ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway # make lint ruff + shellcheck (+ yamllint/actionlint when installed)
GATEWAY_DIR := examples/dsh-agents-as-a-service/gateway # make check versions, example layout and manifest YAML checks
PYTHON ?= python3 # make e2e full example walkthrough on a kind cluster
DSH_PORT ?= 3080 # make changelog regenerate CHANGELOG.md with git-cliff
include versions.env
.PHONY: help image gateway run test lint RELEASE_IDENTITY := $(DSH_VERSION)$(if $(RELEASE_REVISION),-$(RELEASE_REVISION))
IMAGE ?= $(GHCR_IMAGE):$(RELEASE_IDENTITY)
GATEWAY_LOCAL_IMAGE ?= dsh-aaas-gateway:dev
GATEWAY_DIR := examples/dsh-agents-as-a-service/gateway
EXAMPLE_DIR := examples/dsh-agents-as-a-service
PYTHON ?= $(if $(wildcard $(CURDIR)/.venv/bin/python),$(CURDIR)/.venv/bin/python,python3)
DSH_PORT ?= 3080
VCS_REF ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
.PHONY: help image run gateway test cov lint check e2e changelog release-dry-run set-version clean
help: ## Show available targets help: ## Show available targets
@grep -E '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) | sed 's/:.*## /\t/' @grep -E '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) | sed 's/:.*## / /'
image: ## Build the DSH Web image image: ## Build the main DSH Web image (no push)
docker build \ docker build \
--build-arg DSH_VERSION=$(DSH_VERSION) \ --build-arg DSH_VERSION=$(DSH_VERSION) \
--build-arg RELEASE_REVISION=$(RELEASE_REVISION) \
--build-arg PNPM_VERSION=$(PNPM_VERSION) \ --build-arg PNPM_VERSION=$(PNPM_VERSION) \
--build-arg VCS_REF=$(VCS_REF) \
-t $(IMAGE) . -t $(IMAGE) .
gateway: ## Build the gateway image run: image ## Build and run the main image (needs DEEPSEEK_API_KEY)
docker build -t $(GATEWAY_IMAGE) $(GATEWAY_DIR)/
run: image ## Build and run the DSH Web image
docker run --rm -it \ docker run --rm -it \
-p $(DSH_PORT):3080 \ -p $(DSH_PORT):3080 \
-v dsh-home:/home/node/.dsh \ -v dsh-home:/home/node/.dsh \
-e DEEPSEEK_API_KEY \ -e DEEPSEEK_API_KEY \
$(IMAGE) $(IMAGE)
test: ## Run the gateway unit tests (needs requirements + requirements-dev installed) gateway: ## Build the example gateway image locally
docker build -t $(GATEWAY_LOCAL_IMAGE) $(GATEWAY_DIR)/
test: ## Gateway unit tests
cd $(GATEWAY_DIR) && $(PYTHON) -m pytest -q cd $(GATEWAY_DIR) && $(PYTHON) -m pytest -q
lint: ## Lint the gateway with ruff cov: ## Gateway unit tests with coverage (needs pytest-cov)
cd $(GATEWAY_DIR) && ruff check . cd $(GATEWAY_DIR) && $(PYTHON) -m pytest -q --cov=. --cov-report=term-missing --cov-report=xml:coverage.xml
lint: ## Lint Python, shell, YAML and Dockerfiles (skips missing tools locally)
cd $(GATEWAY_DIR) && $(PYTHON) -m ruff check .
cd $(GATEWAY_DIR) && $(PYTHON) -m ruff format --check .
shellcheck build.sh scripts/*.sh $(EXAMPLE_DIR)/scripts/*.sh
@command -v yamllint >/dev/null 2>&1 && yamllint . || echo "yamllint not installed; CI enforces it"
@command -v actionlint >/dev/null 2>&1 && actionlint || echo "actionlint not installed; CI enforces it"
check: ## Verify version pins, example layout and manifest YAML
./scripts/check-versions.sh
./scripts/check-examples.sh
$(PYTHON) scripts/check-manifests.py
e2e: ## Run the full example walkthrough on the current kind cluster
$(EXAMPLE_DIR)/scripts/e2e-kind.sh
changelog: ## Regenerate CHANGELOG.md from Conventional Commits
@command -v git-cliff >/dev/null 2>&1 || { echo "install git-cliff: https://git-cliff.org" >&2; exit 1; }
git-cliff --output CHANGELOG.md
release-dry-run: check ## Validate the release identity and preview release notes
@echo "release identity: $(RELEASE_IDENTITY)"
@echo "main image: $(GHCR_IMAGE):$(RELEASE_IDENTITY)"
@echo "gateway image: $(GATEWAY_IMAGE):$(RELEASE_IDENTITY)"
@if git rev-parse -q --verify "refs/tags/$(RELEASE_IDENTITY)" >/dev/null; then echo "ERROR: tag already exists" >&2; exit 1; fi
@command -v git-cliff >/dev/null 2>&1 && git-cliff --unreleased --tag $(RELEASE_IDENTITY) || echo "git-cliff not installed; skipping notes preview"
set-version: ## Bump pins: make set-version DSH_VERSION=dsh-vX [RELEASE_REVISION=rN]
@test -n "$(DSH_VERSION)" || { echo "DSH_VERSION is required" >&2; exit 1; }
./scripts/set-version.sh "$(DSH_VERSION)" "$(RELEASE_REVISION)"
clean: ## Remove local build/test caches
rm -rf .pytest_cache .ruff_cache $(GATEWAY_DIR)/.pytest_cache $(GATEWAY_DIR)/.ruff_cache $(GATEWAY_DIR)/__pycache__

324
README.md
View File

@@ -1,180 +1,214 @@
# deepseek-kubernetes-sandbox # deepseek-kubernetes-sandbox
Packages [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web (`dsh web`) into **DeepSeek Harness Web, packaged as a container.** This repository's primary
a multi-arch, Kubernetes-ready container image, and ships the artifact is a slim, non-root, multi-arch image of
[agents-as-a-service](examples/dsh-agents-as-a-service/) platform pattern — running a personal [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web
DSH Web instance per user on [kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) (`dsh web`), ready for Docker and Kubernetes. Everything else exists to support
behind a small browser-usable gateway. that image or to demonstrate an implementation on top of it.
## What's in this repo The flagship implementation is
[`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/): the
"agents-as-a-service" platform pattern with one DSH Web instance per user,
provisioned from a warm pool on
[kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox),
suspended when idle, and fronted by a small browser-usable gateway.
| Path | Purpose | ## Quick start
| --- | --- |
| [`Dockerfile`](Dockerfile) | Multi-stage build that clones DSH, `pnpm pack`s the CLI, and produces a slim runtime image (`:3080`) |
| [`build.sh`](build.sh), [`Makefile`](Makefile) | Local image builds: DSH Web and gateway, plus test/lint targets |
| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | The full multi-tenant platform pattern (warm pool, per-user claims, gateway, smoke test) |
| [`examples/dsh-agents-as-a-service/gateway/`](examples/dsh-agents-as-a-service/gateway/) | Starlette ASGI gateway: signup, cookie sessions, `/u/{user}/**` HTTP + WebSocket proxy, idle sweeper |
| [`deploy/k8s/sandbox-template.yaml`](deploy/k8s/sandbox-template.yaml) | Minimal single-tenant `SandboxTemplate` |
| [`docs/architecture.md`](docs/architecture.md) | Container and sequence diagrams, design decisions, security notes |
| [`.github/workflows/`](.github/workflows/) | CI: manifests dry-run, gateway tests/lint, multi-arch image builds |
## Published images
Everything is published to GitHub Container Registry as
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox`:
| Tag | Contents | Port |
| --- | --- | --- |
| `dsh-v0.2.1-rc.2` (also `main`, `latest`) | DSH Web image | `3080` |
| `gateway` | Starlette gateway image | `8080` |
The DSH Web tag matches the upstream DSH git tag it was built from.
## Quick start (DSH Web image)
```bash ```bash
docker run -d --name dsh-web \ docker run -d --name dsh-web \
-p 3080:3080 \ -p 3080:3080 \
-v dsh-home:/home/node/.dsh \ -v dsh-home:/home/node/.dsh \
-e DEEPSEEK_API_KEY \ -e DEEPSEEK_API_KEY \
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
``` ```
**DSH Web is not usable at a bare `/`.** Without its launch-token cookie it answers `401`, and DSH Web is **not usable at a bare `/`**. Without its launch-token cookie it
the token is only printed in the startup logs: answers `401`, and the token is printed only in the startup logs:
```bash ```bash
docker logs dsh-web 2>&1 | grep -m1 '?token=' docker logs dsh-web 2>&1 | grep -m1 '?token='
# dsh web: http://localhost:3080/?token=... # dsh web: http://localhost:3080/?token=...
``` ```
Open the printed `?token=...` URL once — DSH sets its auth cookie and the UI loads normally Open the printed `?token=...` URL once: DSH sets its auth cookie and the UI loads
afterwards. Hitting `/` directly (or after the cookie expires) returns `401` by design; that is normally afterwards. Hitting `/` directly (or after the cookie expires) returns
why Kubernetes probes use `/favicon.svg` (see below) and why the gateway performs this `401` by design.
handshake for the browser.
> **Different host port?** If `3080` is already taken on your workstation, map another port and ### Host port and public URL
> set the public URL so the printed token URL matches: `-p 18081:3080 -e
> DSH_PUBLIC_URL=http://localhost:18081`, then open `http://localhost:18081/?token=...`.
## Quick start (Kubernetes) If `3080` is taken locally, map another port and set the public URL so the quoted
token URL matches:
```bash ```bash
kubectl create namespace dsh-demo --dry-run=client -o yaml | kubectl apply -f - docker run -d --name dsh-web -p 18081:3080 \
kubectl -n dsh-demo create secret generic dsh-provider-keys \ -e DSH_PUBLIC_URL=http://localhost:18081 \
--from-literal=DEEPSEEK_API_KEY="sk-..." -v dsh-home:/home/node/.dsh \
kubectl apply -f deploy/k8s/sandbox-template.yaml -e DEEPSEEK_API_KEY \
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
# A template needs a warm pool and a claim to become a running Sandbox. # then open http://localhost:18081/?token=...
kubectl apply -f - <<'EOF'
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
name: dsh-pool
namespace: dsh-demo
spec:
replicas: 1
sandboxTemplateRef:
name: dsh-web-template
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
name: dsh
namespace: dsh-demo
spec:
warmPoolRef:
name: dsh-pool
EOF
kubectl -n dsh-demo wait sandboxclaim/dsh --for=condition=Ready --timeout=180s
``` ```
Then reach the instance through its launch-token URL: ### Hosts, probes and the 401
- **Probes must not hit `/`.** It returns `401` without the launch token, so a
readiness/liveness probe on `/` crash-loops the pod. Probe `/favicon.svg`
instead: it is unauthenticated. The shipped manifests use it.
- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and
`127.0.0.1` are always trusted; add any other public hostname to
`DSH_TRUSTED_HOSTS` (comma or space separated). A gateway or Ingress that
forwards the browser's `Host` needs its hostname listed there.
- **State lives in `/home/node/.dsh`.** Mount a volume (Docker) or PVC
(Kubernetes) to keep conversations, sessions and config across restarts.
### Runtime environment
| Variable | Default | Purpose |
| --- | --- | --- |
| `DEEPSEEK_API_KEY` | empty | Provider credential consumed by DSH. Inject it; never bake it into the image or commit it. |
| `DSH_PORT` | `3080` | Port DSH Web listens on inside the container. |
| `DSH_HOST` | auto-detected pod/container IP | Bind address. Leave empty in Kubernetes. |
| `DSH_PUBLIC_URL` | `http://localhost:${DSH_PORT}` | Public base URL used in generated links and the launch-token URL in the logs. |
| `DSH_TRUSTED_HOSTS` | empty | Extra trusted `Host`/`Origin` entries, comma/space separated. |
| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (mount a volume here). |
Full contract, build args, probes, volumes and signals:
[docs/main-image.md](docs/main-image.md).
## Published images
Published to GitHub Container Registry. The main image is the repository image;
example components are nested packages below it:
```
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox # DSH Web (main image)
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway # example gateway
```
| Tag | Contents | Meaning |
| --- | --- | --- |
| `dsh-v0.2.1-rc.2-r2` | DSH Web, port `3080` | **Current release identity.** Immutable: once published it never moves. |
| `latest` | DSH Web | Most recent release; same digest as the current identity. |
| `main` | DSH Web | Development channel rebuilt from every push to `main`. Not a release; not for production. |
| `dsh-v0.2.1-rc.2` | DSH Web | **Legacy tag** published before the release-identity scheme. It is not updated. New deployments use `dsh-v0.2.1-rc.2-r2`. |
The example gateway is published under its nested path with the same release
identity, e.g.
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2`.
Release identities, channels and rollback rules are explained in
[docs/versioning.md](docs/versioning.md). Production deployments should pin a
digest:
```bash ```bash
SB=$(kubectl -n dsh-demo get sandboxclaim dsh -o jsonpath='{.status.sandbox.name}') docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
kubectl -n dsh-demo logs "$SB" | grep -m1 '?token=' # image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest>
kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
# open http://localhost:18081/?token=... from the log line
``` ```
Two things bite first-time setups: ## Build from source
- **Probes must not hit `/`.** DSH returns `401` there without the launch token, so a
readiness/liveness probe on `/` crash-loops the pod. The shipped templates use
`/favicon.svg`, which is unauthenticated; keep that path in custom templates.
- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and `127.0.0.1` are
always trusted; for any other hostname add it to the template's `DSH_TRUSTED_HOSTS` (comma or
space separated). The gateway forwards the browser's `Host`, so the public gateway hostname
must be listed there.
## The multi-user platform pattern
`examples/dsh-agents-as-a-service/` is a port of the upstream
[hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service)
example to DSH Web. It demonstrates the platform shape — one `SandboxTemplate`, one warm pool,
one tiny `SandboxClaim` per user, and `operatingMode` as the cost dial — plus a gateway that
turns it into a usable service.
**Browser flow:** open the gateway landing page, sign up, and get an `HttpOnly` session cookie.
The browser then browses `http://<gateway>/u/<user>/`, which proxies HTTP **and WebSocket** to
that user's DSH Web. The gateway reads the DSH launch token from the pod logs, performs the
token handshake server-side, and injects the resulting DSH cookie on every proxied request — the
browser never sees the token. Suspended instances are woken transparently on the next request;
idle ones are suspended again after `IDLE_TIMEOUT`.
See the [example README](examples/dsh-agents-as-a-service/README.md) for the full walkthrough
and [docs/architecture.md](docs/architecture.md) for the diagrams and trade-offs.
### Mechanism mapping: Hermes → DSH
| Hermes example mechanism | Role | Adaptation for DSH |
| :--- | :--- | :--- |
| `SandboxTemplate` | Reusable Pod blueprint | Image `dsh-web`, port `3080`, probes on `/favicon.svg` |
| `volumeClaimTemplates` | Auto-create a PVC per user | Mount `/home/node/.dsh` (DSH state) |
| `SandboxWarmPool` | Pre-warmed instances | Keep as-is; size for expected concurrency |
| `SandboxClaim` + `warmPoolRef` | "Registration" adopts a pooled instance | Keep exactly as-is; one claim per user |
| `service: true` | Stable DNS per Sandbox | Enabled, so the gateway routes to each user's DSH Web |
| `envVarsInjectionPolicy: Disallowed` | Reject claims that would cold-start | `DEEPSEEK_API_KEY` injected via `envFrom` on the template (shared platform key) |
| `operatingMode` | Cost switch: Running / Suspended | Reused directly — suspend idle instances to save cost |
### Key differences from Hermes
- **Multi-arch** — the DSH image is built for amd64 and arm64 (no amd64 nodeSelector).
- **Single port** — DSH Web serves everything on `3080`; no API/dashboard port split.
- **One secret** — only `DEEPSEEK_API_KEY`; no dashboard/session/api-server credentials.
- **Host forwarding** — the gateway forwards `Host` + `X-Forwarded-*` (and keeps
`DSH_PUBLIC_URL` empty) so DSH generates reachable links; add the public hostname to
`DSH_TRUSTED_HOSTS`.
- **Browser sessions + WebSocket** — the gateway is Starlette ASGI, so it proxies WebSocket
upgrades as well as HTTP/SSE, and authenticates browsers with an `HttpOnly` cookie instead of
exposing per-user bearer tokens in URLs.
## Build it yourself
```bash ```bash
./build.sh # build the DSH Web image, then run it interactively make image # build the main DSH Web image (clones and builds upstream DSH)
./build.sh web # build the DSH Web image only make run # build it, then run it interactively (needs DEEPSEEK_API_KEY)
./build.sh gateway # build the gateway image
make image # same as ./build.sh web
make gateway # gateway image
make test # gateway unit tests (pytest)
make lint # gateway lint (ruff)
``` ```
Or directly: The pins (`DSH_VERSION`, `RELEASE_REVISION`, `PNPM_VERSION`) come from
[versions.env](versions.env) — never hard-code them
(`make check` verifies). [`build.sh`](build.sh) is a deprecated shim that
forwards to `make`; use the Makefile targets.
```bash ## Examples
docker build \
--build-arg DSH_VERSION=dsh-v0.2.1-rc.2 \
--build-arg PNPM_VERSION=11.28.5 \
-t dsh-web:0.2.1-rc.2 .
docker build -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway \ Self-contained implementations that consume the image. Index and contract:
examples/dsh-agents-as-a-service/gateway/ [`examples/README.md`](examples/README.md) and
``` [docs/examples.md](docs/examples.md).
See [`Dockerfile`](Dockerfile) for build args (`DSH_REPO`, `DSH_VERSION`, `PNPM_VERSION`) and | Example | What it demonstrates |
the runtime image contract (non-root `node` user, `DSH_HOME=/home/node/.dsh`, `EXPOSE 3080`). | --- | --- |
`.env.sample` documents the runtime variables for both images. | [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Full multi-tenant platform: SandboxTemplate with per-user PVCs, warm pool, one claim per user, suspend-when-idle, wake-on-return, browser gateway with signup + `HttpOnly` sessions + HTTP/WebSocket proxy, Ingress or Gateway API + GitHub SSO exposure. Start with the [walkthrough](examples/dsh-agents-as-a-service/README.md) and the [architecture](examples/dsh-agents-as-a-service/docs/architecture.md). |
| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | The smallest useful setup: one `SandboxTemplate` and one `SandboxClaim`, reached through `kubectl port-forward` + the launch-token URL. Manifest: [`manifests/sandbox-template.yaml`](examples/minimal-single-tenant/manifests/sandbox-template.yaml). |
The gateway is a component of the multi-tenant example and has its own
[README](examples/dsh-agents-as-a-service/gateway/README.md) (API, configuration,
auth/SSO, limitations).
## Repository layout
| Path | Purpose |
| --- | --- |
| [`Dockerfile`](Dockerfile) | Primary artifact: the multi-stage DSH Web image (port `3080`). |
| [`versions.env`](versions.env) | Single source of truth for the DSH ref, repository revision, pnpm/control-plane versions and image names. |
| [`Makefile`](Makefile), [`build.sh`](build.sh) | Canonical local entrypoint; `build.sh` is a deprecated shim. |
| [`docs/`](docs/README.md) | Repository docs: image contract, example contract, gitflow, versioning. |
| [`examples/`](examples/README.md) | One self-contained implementation per directory. |
| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Multi-tenant platform example: `manifests/`, `gateway/`, `docs/`, `scripts/`. |
| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | Single-sandbox quickstart. |
| [`scripts/`](scripts/) | Repository checks: versions, example layout, manifest YAML, version bump. |
| [`.github/`](.github/) | CI, release automation, dependency updates, PR/issue templates. |
| [`.env.sample`](.env.sample) | Documented local configuration (copy to `.env`; never commit it). |
## Development
The Makefile is the canonical entrypoint. No cluster is needed for `check`,
`test`, `lint`, `gateway` or `image`:
| Target | What it does |
| --- | --- |
| `make check` | Version pins, example layout and manifest YAML checks. Run before every commit. |
| `make test` | Gateway unit tests (`pytest`). |
| `make lint` | `ruff` + `shellcheck` (+ `yamllint`/`actionlint` when installed). |
| `make image` | Build the main DSH Web image. |
| `make gateway` | Build the example gateway image locally (`dsh-aaas-gateway:dev`). |
| `make e2e` | Full kind walkthrough of the multi-tenant example (needs a cluster with agent-sandbox). |
| `make changelog` | Regenerate `CHANGELOG.md` with git-cliff (release only). |
| `make set-version` | Bump pins in `versions.env`; e.g. `make set-version DSH_VERSION=<ref> RELEASE_REVISION=rN`. |
Local setup and the PR workflow: [CONTRIBUTING.md](CONTRIBUTING.md); branches,
required checks and releases: [docs/gitflow.md](docs/gitflow.md).
## Versioning and releases
The release identity is the upstream DSH ref plus an optional repository
revision: `DSH_VERSION` and `RELEASE_REVISION` in [versions.env](versions.env).
The current identity is **`dsh-v0.2.1-rc.2-r2`** — the first release under the
mirrored-DSH scheme at the same upstream DSH ref as the legacy
`dsh-v0.2.1-rc.2` tag. Tags are immutable; a broken release is fixed by
publishing `-r(N+1)`, never by re-tagging.
`CHANGELOG.md` is generated by [git-cliff](https://git-cliff.org) from
Conventional Commits and is never edited by hand. See
[`CHANGELOG.md`](CHANGELOG.md), [docs/versioning.md](docs/versioning.md) and
[cliff.toml](cliff.toml).
## Migrating from the previous layout
If you are coming from an older checkout, these paths moved. Nothing else changed
semantically; the content was split into the example it belongs to.
| Old path | New path |
| --- | --- |
| `gateway-api/dsh-demo/` (Gateway API + oauth2-proxy / GitHub SSO) | `examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/` (now `70-gateway.yaml` ... `74-tls-secret.yaml.example`) |
| `examples/dsh-agents-as-a-service/60-ingress.yaml` | `examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml` |
| `examples/dsh-agents-as-a-service/NN-*.yaml` (`00-` ... `50-`) | `examples/dsh-agents-as-a-service/manifests/NN-*.yaml` |
| `examples/dsh-agents-as-a-service/run-test-kind.sh` | `examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` |
| `deploy/k8s/sandbox-template.yaml` | `examples/minimal-single-tenant/manifests/sandbox-template.yaml` |
| `docs/architecture.md` | `examples/dsh-agents-as-a-service/docs/architecture.md` |
The SSO walkthrough that lived in `gateway-api/dsh-demo/README.md` is merged
into the [multi-tenant example README](examples/dsh-agents-as-a-service/README.md)
(step 9) and the [exposure README](examples/dsh-agents-as-a-service/manifests/exposure/README.md).
## Contributing
- [CONTRIBUTING.md](CONTRIBUTING.md) — local setup, PR workflow, what a change needs.
- [AGENTS.md](AGENTS.md) — repository conventions for AI coding agents (and humans) and the CI commands.
- [SECURITY.md](SECURITY.md) — private vulnerability reporting and project threat notes.
- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — community standards.
- [docs/gitflow.md](docs/gitflow.md) — branches, protected `main`, required checks, release process.
Do not commit secrets; secret-shaped YAML lives in `*.yaml.example` files, and
version pins change only through `versions.env` / `make set-version`.
## License
[MIT](LICENSE) © Nicolás Georger.

41
SECURITY.md Normal file
View File

@@ -0,0 +1,41 @@
# Security policy
## Supported versions
Only the latest published release is supported. Image tags are immutable;
fixes ship as a new release (see docs/versioning.md).
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/sredevopsorg/deepseek-kubernetes-sandbox/security/advisories/new
Do not open a public issue, and do not include live credentials in a report.
You can expect an acknowledgement within a few days and a coordinated
disclosure once a fix is available.
## Threat notes specific to this project
- **Launch token in pod logs.** The example gateway reads DSH's launch token
from pod logs (pods/log). Anyone with access to those logs can impersonate a
tenant. Keep the gateway's RBAC namespace-scoped and treat pod logs as
sensitive. Replacing log scraping with an upstream token API is tracked in
the example architecture document.
- **SSO header trust boundary.** AUTH_PROXY_HEADER makes the gateway trust an
identity header (for example X-Forwarded-User). Enable it only when the
gateway Service is reachable exclusively through the authenticating proxy;
otherwise any client can spoof the header.
- **Shared provider key.** The example injects one DEEPSEEK_API_KEY into every
tenant sandbox (a warm-pool constraint). Treat tenants as mutually trusting
until per-tenant credentials or an egress proxy exist.
- **TLS is external.** Terminate TLS at an Ingress or Gateway and set
COOKIE_SECURE=true. Plain HTTP outside localhost exposes session cookies.
- **No secrets in the repository.** The repository only ships empty Secret
manifests and *.yaml.example files.
## Dependency and supply chain
GitHub Actions are pinned by commit SHA. The CodeQL workflow scans Python and
Actions. The release workflow publishes images with SBOM and provenance
attestations.

View File

@@ -1,68 +1,19 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build the DSH Web and/or gateway images locally. # Deprecated compatibility shim: "make" is the canonical entrypoint.
# #
# Usage: # ./build.sh -> make run (build the main image, then run it)
# ./build.sh # build the DSH Web image, then run it interactively # ./build.sh web -> make image
# ./build.sh web # build the DSH Web image only # ./build.sh gateway -> make gateway
# ./build.sh gateway # build the gateway image (examples/.../gateway/)
# #
# Overridable environment: # Use the Makefile targets directly for everything else (test, lint, check, e2e).
# DSH_VERSION DSH git ref to build (default dsh-v0.2.1-rc.2)
# PNPM_VERSION pnpm version (default 11.28.5)
# IMAGE DSH Web image tag (default dsh-web:<version>)
# DSH_PORT host port for docker run (default 3080)
# GATEWAY_IMAGE gateway image tag (default ghcr.io/...:gateway)
# RUN 0 to build without running (default 1)
set -euo pipefail set -euo pipefail
cd "$(dirname "$0")" cd "$(dirname "$0")"
DSH_VERSION="${DSH_VERSION:-dsh-v0.2.1-rc.2}" echo "build.sh is deprecated; use 'make image', 'make run' or 'make gateway'." >&2
PNPM_VERSION="${PNPM_VERSION:-11.28.5}"
IMAGE="${IMAGE:-dsh-web:${DSH_VERSION#dsh-v}}"
GATEWAY_IMAGE="${GATEWAY_IMAGE:-ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway}"
DSH_PORT="${DSH_PORT:-3080}"
RUN="${RUN:-1}"
GATEWAY_DIR=examples/dsh-agents-as-a-service/gateway
build_web() {
docker build \
--build-arg "DSH_VERSION=${DSH_VERSION}" \
--build-arg "PNPM_VERSION=${PNPM_VERSION}" \
-t "${IMAGE}" \
"$@" \
.
}
build_gateway() {
docker build -t "${GATEWAY_IMAGE}" "$@" "${GATEWAY_DIR}/"
}
usage() {
echo "usage: $0 [run|web|gateway] [extra docker build args...]" >&2
exit 2
}
case "${1:-run}" in case "${1:-run}" in
run) run) exec make run ;;
shift web) exec make image ;;
build_web "$@" gateway) exec make gateway ;;
if [ "${RUN}" = "1" ]; then *) echo "usage: $0 [run|web|gateway]" >&2; exit 2 ;;
docker run --rm -it \
-p "${DSH_PORT}:3080" \
-v dsh-home:/home/node/.dsh \
-e DEEPSEEK_API_KEY \
"${IMAGE}"
fi
;;
web)
shift
build_web "$@"
;;
gateway)
shift
build_gateway "$@"
;;
*)
usage
;;
esac esac

49
cliff.toml Normal file
View File

@@ -0,0 +1,49 @@
# git-cliff configuration: release notes and CHANGELOG.md are generated from
# Conventional Commits. See docs/versioning.md for the release procedure.
[changelog]
header = """
# Changelog
All notable changes to this repository are documented here.
Generated by git-cliff from Conventional Commits; do not edit this file by hand.
"""
body = """
{% if version %}\
## [{{ version }}] - {{ timestamp | date(format="%Y-%m-%d") }}
{% else %}\
## [Unreleased]
{% endif %}\
{% for group, commits in commits | group_by(attribute="group") %}
### {{ group | upper_first }}
{% for commit in commits %}
- {% if commit.scope %}**{{ commit.scope }}:** {% endif %}{{ commit.message | split(pat="\n") | first | upper_first }} ({{ commit.id | truncate(length=7, end="") }})
{%- endfor %}
{% endfor %}\n
"""
footer = ""
trim = true
[git]
conventional_commits = true
filter_unconventional = false
split_commits = false
commit_parsers = [
{ message = "^feat", group = "Added" },
{ message = "^fix", group = "Fixed" },
{ message = "^perf", group = "Performance" },
{ message = "^refactor", group = "Changed" },
{ message = "^docs", group = "Documentation" },
{ message = "^build", group = "Build" },
{ message = "^ci", group = "CI" },
{ message = "^test", group = "Tests" },
{ message = "^chore", group = "Maintenance" },
{ message = "^revert", group = "Reverted" },
{ message = ".*", group = "Other" },
]
protect_breaking_commits = false
filter_commits = false
# Release tags are dsh-vX.Y.Z[-rN]; the single legacy pre-scheme tag is v0.0.1.
tag_pattern = "^(v|dsh-v).*$"
sort_commits = "oldest"

38
docs/README.md Normal file
View File

@@ -0,0 +1,38 @@
# Repository documentation
Repository-wide documentation lives here. Example-specific material lives with
its example under [`examples/<name>/docs/`](../examples/) — see the
[agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md)
for the deep-dive that used to live in this directory.
If you are new, read the [root README](../README.md) first: it is image-first and
gets a DSH Web container running before anything else.
## Index
| Document | Read it for |
| --- | --- |
| [main-image.md](main-image.md) | The primary artifact: build args, runtime environment, entrypoint, probes, volumes/ports/signals, non-root contract, image names, tags and digests. |
| [examples.md](examples.md) | The example contract: required layout, manifest rules, exposure variants, components and image naming, scripts, docs, and how to add a new example. |
| [gitflow.md](gitflow.md) | Branches, protected `main`, squash merges, Conventional PR titles, required CI checks, release branches, hotfixes and the one-time GitHub settings checklist. |
| [versioning.md](versioning.md) | Release identity (`DSH ref` + optional `-rN`), why the current identity is `dsh-v0.2.1-rc.2-r2`, channels, compatibility matrix, bumping pins, the release checklist, rollback, git-cliff and digest pinning. |
## Example documentation
| Document | Read it for |
| --- | --- |
| [examples/README.md](../examples/README.md) | The index of implementations in this repository. |
| [dsh-agents-as-a-service README](../examples/dsh-agents-as-a-service/README.md) | The multi-tenant platform walkthrough: warm pool, claims, suspend/resume, gateway, exposure and troubleshooting. |
| [dsh-agents-as-a-service architecture](../examples/dsh-agents-as-a-service/docs/architecture.md) | Container/sequence diagrams, ADRs, security and RBAC, failure modes and the evolution path. |
| [gateway README](../examples/dsh-agents-as-a-service/gateway/README.md) | The gateway's HTTP/WebSocket API, configuration, auth (including SSO), and limitations. |
## Project policies
| Document | Read it for |
| --- | --- |
| [AGENTS.md](../AGENTS.md) | Repository conventions for AI agents and humans, and the commands the CI runs. |
| [CONTRIBUTING.md](../CONTRIBUTING.md) | Local setup, the change workflow and what a pull request needs. |
| [SECURITY.md](../SECURITY.md) | Vulnerability reporting and project-specific threat notes (pod logs, SSO header trust, shared provider key, TLS). |
| [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) | Community standards. |
| [CHANGELOG.md](../CHANGELOG.md) | Generated release history (do not edit by hand). |
| [LICENSE](../LICENSE) | MIT. |

149
docs/examples.md Normal file
View File

@@ -0,0 +1,149 @@
# Example contract
Every implementation in this repository is a **self-contained example** under
[`examples/<name>/`](../examples/). Examples demonstrate how to run the main
DSH Web image (and, when needed, auxiliary components such as the
agents-as-a-service gateway) on Kubernetes. They are reference material, not
libraries: each one is applied with plain `kubectl`, documented end to end, and
deletable in one command.
[`examples/README.md`](../examples/README.md) is the index. If an example is not
listed there, it does not exist as far as `make check` is concerned.
## Required layout
```
examples/<name>/
+-- README.md # required: what it demonstrates, prerequisites,
| # ordered steps, troubleshooting, cleanup
+-- manifests/ # required
| +-- NN-*.yaml # numbered top-level manifests, plain kubectl
| +-- exposure/ # optional exposure variants
| +-- README.md # required when exposure/ exists
| +-- <variant>/NN-*.yaml # e.g. ingress/, gateway-api-sso/
+-- scripts/ # optional
| +-- *.sh # executable, idempotent, self-cleaning
+-- docs/ # optional deeper material (architecture, ADRs)
+-- <component>/ # optional published component
+-- Dockerfile
+-- README.md # required whenever a Dockerfile exists
+-- tests...
```
### Rules the CI enforces
[scripts/check-examples.sh](../scripts/check-examples.sh) fails the build when:
- `examples/README.md` is missing or does not mention `examples/<name>/` for every
example directory;
- an example has no `README.md` or no `manifests/` directory;
- `manifests/` contains no top-level `*.yaml` files;
- a `*.yaml` file sits at the example root instead of under `manifests/`;
- `manifests/exposure/` exists without a `manifests/exposure/README.md` describing
the variants;
- a subdirectory with a `Dockerfile` has no `README.md`;
- a file in `scripts/` is not executable.
[scripts/check-manifests.py](../scripts/check-manifests.py) additionally parses
every `manifests/**/*.yaml` and requires `apiVersion`, `kind` and
`metadata.name`. CI also runs a server-side dry-run against the real
agent-sandbox CRDs on a throwaway kind cluster.
Run both locally with:
```bash
make check
```
## Manifests
- **Numbered and ordered.** `NN-*.yaml` files are applied in filename order and
read as a walkthrough: `00-prereqs.yaml`, `10-sandboxtemplate.yaml`,
`20-sandboxwarmpool.yaml`, ... Keep the numbering stable; it is referenced from
the example README.
- **Plain kubectl.** No Helm, Kustomize, or custom installer. `kubectl apply -f`
against a namespace the example creates is the whole interface.
- **Safe by default.** Never ship a usable Secret. Secret-shaped manifests are
`*.yaml.example` files with obvious placeholders and an inline creation
command; real values are generated per install.
- **Cluster-specific state stays out.** Node affinity, replica counts tuned for a
cluster, and similar fields belong in a local patch, not in the shipped
manifest.
- **Placeholders are explicit.** Hostnames, GitHub orgs and TLS material use
values like `dsh.example.com` / `your-github-org` with a TODO comment.
## Exposure variants
An example that can be reached from outside the cluster puts each option under
`manifests/exposure/<variant>/` and describes the trade-offs in
`manifests/exposure/README.md`. Variants are alternatives, not a stack: apply one.
`dsh-agents-as-a-service` currently ships:
| Variant | Path | What it gives you |
| --- | --- | --- |
| Ingress | `manifests/exposure/ingress/` | Traefik-style Ingress + TLS; the gateway does its own session auth. |
| Gateway API + SSO | `manifests/exposure/gateway-api-sso/` | Gateway API `Gateway`/`HTTPRoute` in front of oauth2-proxy (GitHub OAuth); the gateway inherits `X-Forwarded-User` via `AUTH_PROXY_HEADER`. |
## Components
A component is a program the example needs beyond the main image: today only the
agents-as-a-service gateway. A component lives in its own subdirectory with:
- `Dockerfile` — its image build;
- `README.md` — purpose, API/config reference, local development, limitations;
- tests — unit tests that run without a cluster, plus whatever else applies.
Component images are published as **nested packages below the repository image**:
```
<registry>/<owner>/<repo>/<example>/<component>
```
so the gateway is
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway`.
Never create an unrelated top-level package; `check-versions.sh` verifies the
nesting.
## Scripts
`scripts/*.sh` must be:
- **executable** (the CI checks the mode bit);
- **idempotent** — safe to re-run, cleaning up before creating;
- **self-cleaning** — a trap removes the namespace, port-forwards and temp files,
even on failure;
- **non-interactive** — usable in CI, with environment-variable knobs instead of
prompts.
`examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` is the reference
implementation: it runs the README walkthrough and deletes everything it created.
## Documentation
- The example `README.md` is the walkthrough. Keep it runnable: every command
block should work when pasted in order.
- Deeper material (architecture, sequence diagrams, ADRs) belongs in the
example's `docs/`, not in the repository-level [docs/](../docs/).
- Repository-wide docs describe the container and the repo rules; example docs
describe one implementation.
## Adding a new example
1. Create `examples/<name>/` with the layout above.
2. Write `README.md` first: what it demonstrates, prerequisites, numbered steps,
troubleshooting, limitations, cleanup.
3. Add `manifests/NN-*.yaml`; keep secrets as `*.yaml.example`.
4. If a component needs an image, publish it under
`<registry>/<owner>/<repo>/<example>/<component>`.
5. Add an entry to [`examples/README.md`](../examples/README.md) using the literal
`examples/<name>/` path.
6. Run `make check` and, if you added a component, `make test` and `make lint`.
7. If the example is expensive to run (kind, images, cluster CRDs), ship a
`scripts/e2e-kind.sh` and wire it into CI.
## Current examples
| Example | Demonstrates |
| --- | --- |
| [`examples/dsh-agents-as-a-service/`](../examples/dsh-agents-as-a-service/) | The full multi-tenant platform pattern: SandboxTemplate, warm pool, per-user claims, suspend/resume, and the browser gateway. |
| [`examples/minimal-single-tenant/`](../examples/minimal-single-tenant/) | A single DSH Web sandbox: one SandboxTemplate and a claim. |

128
docs/gitflow.md Normal file
View File

@@ -0,0 +1,128 @@
# Gitflow: branches, pull requests and CI
This repository uses **trunk-based development with release branches**. `main` is
always releasable; feature work happens on short-lived branches; releases are cut
from `main` onto a release branch and tagged with an immutable release identity.
See [versioning](versioning.md) for what the tags mean.
## Branches
| Branch | Branched from | Merged into | Purpose |
| --- | --- | --- | --- |
| `main` | — | — | The integration trunk. Protected, linear history, always releasable. |
| `feat/<slug>` | `main` | `main` (squash) | New capability or example. |
| `fix/<slug>` | `main` | `main` (squash) | Bug fix. |
| `docs/<slug>` | `main` | `main` (squash) | Documentation only. |
| `refactor/<slug>`, `chore/<slug>`, `test/<slug>` | `main` | `main` (squash) | Non-behavioral rework, maintenance, tests. |
| `release/<identity>` | `main` at the release commit | — | Release stabilization and tagging. Only release-blocking fixes land here. |
| `hotfix/<slug>` | The release branch (or the tagged commit) | `main` and the release branch | Urgent production fix. |
Keep branches short-lived. Rebase on `main` rather than merging `main` into the
branch; the PR is squash-merged, so individual branch commits are never part of
`main` history.
## Pull requests
1. **One scoped change per PR.** Do not mix a version bump with unrelated work,
and update the docs the change touches in the same PR.
2. **Conventional PR title.** The title becomes the squash commit message, so it
is linted by the `pr-title` check: `feat(scope): ...`, `fix(scope): ...`,
`docs(scope): ...`, `build: ...`, `ci: ...`, `refactor: ...`, `test: ...`,
`chore: ...`. Breaking changes use `feat(scope)!: ...` or a
`BREAKING CHANGE:` footer.
3. **CI must be green.** All required checks below must pass before merge.
4. **Squash merge only.** `main` stays linear and protected; merged PRs are one
commit each.
5. **No direct pushes to `main`** or to a release branch. Both are protected.
6. **Review for behavior changes.** Infrastructure, workflow, release and
gateway-contract changes need a maintainer review; trivial docs changes still
need the checks.
## Required checks
| Check | Workflow / job | What it verifies |
| --- | --- | --- |
| `lint` | `ci.yaml` | `scripts/check-versions.sh` (version pins and markdown tokens), ruff on the gateway, shellcheck on shell scripts, yamllint/actionlint when available. |
| `python` | `ci.yaml` | Gateway unit tests (`pytest`). |
| `manifests` | `ci.yaml` | `scripts/check-examples.sh`, `scripts/check-manifests.py` and a server-side dry-run of every example manifest against the agent-sandbox CRDs. |
| `gateway-image` | `ci.yaml` | Builds the gateway image and smoke-tests `/healthz` without a cluster. |
| `pr-title` | `pr-title.yaml` | Conventional Commit PR title. |
| `CodeQL` | `codeql.yml` | Static analysis for Python and GitHub Actions. |
| `dependency-review` | `ci.yaml` | Dependency changes introduced by the PR (job `dependency-review`; runs on pull requests only). |
The image build workflows (`_build-image.yaml`, `build-main.yaml`,
`build-gateway.yaml`) and `e2e-kind.yaml` are **not** required on every PR: the
image builds run where they are needed, and the kind end-to-end suite is reserved
for release validation and manual dispatch because it is slow.
## Releases
Releases are cut from `main`:
1. Confirm `main` is green and `make check`, `make test`, `make lint` pass.
2. Bump [versions.env](../versions.env) if the DSH ref changed, using
`make set-version`; otherwise keep the same `DSH_VERSION` and bump
`RELEASE_REVISION`.
3. Create `release/<identity>` from the release commit.
4. Run `make release-dry-run` and review the generated notes.
5. Create and push the annotated `dsh-*` release tag. The `release.yaml`
workflow validates that the tag matches [versions.env](../versions.env), builds
and pushes the multi-arch images (version + latest), creates the GitHub
Release from the git-cliff notes, and opens a `chore(changelog)` PR that
refreshes `CHANGELOG.md` on `main`.
6. Merge the changelog PR and any release-branch-only fixes back to `main`.
Full procedure: [versioning](versioning.md).
## Hotfixes
A released tag is **never** moved or re-tagged — tags and image identities are
immutable. To fix a broken release:
1. Branch `hotfix/<slug>` from the release tag (or its release branch).
2. Make the minimal fix, add a test, open a PR; the same required checks apply.
3. Publish a new identity with a higher `RELEASE_REVISION` (`-r(N+1)`). Do not
reuse or delete the broken tag.
4. Merge the fix back into `main` (or cherry-pick it) so the trunk is fixed too.
Rollback is also a re-release, never a retag: see
[versioning](versioning.md#rollback).
## Tag protection
All release tags match `dsh-*` and are protected:
- only maintainers (the release workflow) may create them;
- they cannot be deleted or force-updated;
- they are annotated and point at the release commit on a release branch.
Image tags derive from the same identity and are equally immutable; the mutable
`latest` and `main` image channels always point at a release or a trunk build
respectively.
## One-time GitHub settings checklist
Configure these on the repository before relying on the workflow:
- [ ] **Branches → main**: require a pull request; require the checks `lint`,
`python`, `manifests`, `gateway-image`, `pr-title`, `CodeQL`,
`dependency-review`; require branches to be up to date; require conversation
resolution; restrict direct pushes; allow only squash merges and enable
"Automatically delete head branches".
- [ ] **Branches → release/***: same protections, plus allow the release workflow
to push tags.
- [ ] **Tags → dsh-***: restrict creation/update/deletion to maintainers; a tag
push triggers the release workflow.
- [ ] **Actions → General**: allow GitHub Actions; set workflow permissions to
read-only by default; allow `GITHUB_TOKEN` to create pull requests only
where required.
- [ ] **Actions → Runners**: make the arm64 runners (`ubuntu-24.04-arm`) available
to the public/private repository as appropriate.
- [ ] **Packages**: grant the release workflow `packages: write` (workflow
permissions do this) and protect the
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox` packages from deletion.
- [ ] **Security**: enable CodeQL default setup **or** keep the advanced
`codeql.yml` workflow (not both), enable Dependabot alerts and
dependency review.
- [ ] **Secrets/Variables**: none are needed for image publishing; the workflows
use `GITHUB_TOKEN`.

169
docs/main-image.md Normal file
View File

@@ -0,0 +1,169 @@
# The main DSH Web image
This repository's primary artifact is a container image that packages
[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web
(`dsh web`) for Kubernetes and plain Docker. The [Dockerfile](../Dockerfile) at the
repository root builds it; every version pin it uses lives in
[versions.env](../versions.env).
The image is image-first: it is the product, and the examples under
[`examples/`](../examples/) are self-contained implementations that consume it.
## What the build does
The Dockerfile is a two-stage build:
1. **Build stage** (`node:24-trixie`) clones DSH at the pinned git ref, installs
dependencies with pnpm, runs the production build, packages the `@deepseek-ai/dsh`
CLI with `pnpm pack`, and installs its production dependencies with npm.
2. **Runtime stage** (`node:24-trixie-slim`) copies `/opt/dsh` to `/app`, adds the
CLI tools DSH agents commonly use (`git`, `ripgrep`, `jq`, `yq`, `curl`, `wget`,
`gh`, `tini`, `uv`), and starts `dsh web` through a small entrypoint.
## Build it
```bash
make image # canonical: builds the DSH Web image, no push
```
`make image` reads [versions.env](../versions.env) and passes the pins as build args.
To reproduce the equivalent by hand:
```bash
docker build \
--build-arg DSH_VERSION=dsh-v0.2.1-rc.2 \
--build-arg RELEASE_REVISION=r2 \
--build-arg PNPM_VERSION=11.28.5 \
--build-arg VCS_REF="$(git rev-parse --short HEAD)" \
-t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 .
```
### Build args
| Arg | Default / source | Purpose |
| --- | --- | --- |
| `DSH_REPO` | `https://github.com/deepseek-ai/deepseek-harness.git` | Upstream DSH repository to clone. |
| `DSH_VERSION` | `DSH_VERSION` in [versions.env](../versions.env) | Upstream DSH git tag/ref to package; recorded as `org.opencontainers.image.version`. |
| `RELEASE_REVISION` | `RELEASE_REVISION` in [versions.env](../versions.env) | Repository revision suffix (`rN`); the release identity is `DSH_VERSION-RELEASE_REVISION`, recorded in the `io.sredevopsorg.release.revision` label. See [versioning](versioning.md). |
| `PNPM_VERSION` | `PNPM_VERSION` in [versions.env](../versions.env) | pnpm version activated with corepack in both stages. |
| `VCS_REF` | `git rev-parse --short HEAD` (via `make image`) | Source revision recorded for traceability. |
| `NODE_BUILD_IMAGE` | `docker.io/node:24-trixie` | Build-stage base image. |
| `NODE_RUNTIME_IMAGE` | `docker.io/node:24-trixie-slim` | Runtime-stage base image. |
Never hard-code a version outside [versions.env](../versions.env):
`./scripts/check-versions.sh` (and `make check`) fails when a pin drifts.
## Runtime contract
### Entrypoint
The entrypoint is `tini -- /usr/local/bin/dsh-entrypoint`:
1. `tini` runs as PID 1 and reaps children/forwards signals.
2. The shell entrypoint applies defaults, derives a bind address when `DSH_HOST` is
empty (the pod/container's first non-loopback IPv4), builds the
`dsh web` command line, and `exec`s it so signals reach the Node process.
3. `localhost` and `127.0.0.1` are always added to DSH's trusted-host list; each
`DSH_TRUSTED_HOSTS` entry is added as an extra `--trusted-host`.
### Environment variables
| Variable | Default | Purpose |
| --- | --- | --- |
| `DSH_PORT` | `3080` | Port DSH Web listens on inside the container. |
| `DSH_HOST` | auto-detected non-loopback IPv4 | Bind address. Leave empty in Kubernetes so the pod IP is used automatically. |
| `DSH_PUBLIC_URL` | `http://localhost:${DSH_PORT}` | Public base URL used to build links and the launch-token URL printed in the logs. Set it when the container is published on a different host port or behind a proxy. |
| `DSH_TRUSTED_HOSTS` | empty | Extra `Host`/`Origin` allow-list entries, comma- or space-separated. `localhost`/`127.0.0.1` are always trusted. Required when a proxy forwards the browser's `Host`. |
| `DEEPSEEK_API_KEY` | empty | Provider credential read by DSH. Inject it with `-e`/`envFrom`; never bake it into the image. Only model calls fail when it is missing. |
| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (conversations, sessions, config, credentials). Mount a volume here to persist it. |
The image also sets `NODE_ENV=production`, `HOME=/home/node`, and a `PATH` that
includes `/app/node_modules/.bin`.
### Ports, volumes and signals
| Item | Value | Notes |
| --- | --- | --- |
| Exposed port | `3080/tcp` | `EXPOSE 3080`; one port serves the UI, assets, API, SSE and WebSocket traffic. |
| State volume | `/home/node/.dsh` | Declared with `VOLUME`; use a named volume (Docker) or PVC (Kubernetes). |
| Working directory | `/workspace` | The agent workspace; mount a volume here if agents should keep files. |
| Stop signal | `SIGTERM` | `tini` forwards it to `dsh web`; allow a graceful shutdown period. |
| User | `node`, UID/GID `1000` | The process is non-root. A PVC mounted at `DSH_HOME` must be writable by UID 1000. |
### Health probes
`/` is **not** a health endpoint: DSH returns `401` until a browser completes the
launch-token handshake. Probe `/favicon.svg` instead — it is unauthenticated and
served by the web layer, so it proves the HTTP server is up without triggering auth.
```yaml
startupProbe:
httpGet: {path: /favicon.svg, port: 3080}
periodSeconds: 5
failureThreshold: 24
readinessProbe:
httpGet: {path: /favicon.svg, port: 3080}
periodSeconds: 5
livenessProbe:
httpGet: {path: /favicon.svg, port: 3080}
periodSeconds: 15
```
The shipped templates use exactly this shape; see
[`examples/minimal-single-tenant/manifests/sandbox-template.yaml`](../examples/minimal-single-tenant/manifests/sandbox-template.yaml).
A `startupProbe` is important because plugin activation (or the first start of a
large sandbox) can exceed a default liveness budget.
### Launch token
DSH prints its one-time launch URL to the logs:
```
dsh web: http://<host>:${DSH_PORT}/?token=...
```
Open that URL once to obtain DSH's auth cookie. After that the UI works at the bare
URL; without the cookie `/` answers `401` by design. The
[agents-as-a-service gateway](../examples/dsh-agents-as-a-service/gateway/) performs
this handshake server-side so browsers never see the token.
## Published image names and tags
Registry layout: the main image is the repository itself, and every example
component is a nested package below it:
```
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox # main DSH Web image
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway # example component
```
The main image is published multi-arch (`linux/amd64`, `linux/arm64`) to
GitHub Container Registry:
| Tag | Meaning |
| --- | --- |
| `dsh-v0.2.1-rc.2-r2` | Current release identity. **Immutable**: once published it never moves. |
| `latest` | Most recent release; points at the same digest as the current identity. |
| `main` | Development channel rebuilt from every push to `main`; not a release, not for production. |
| `dsh-v0.2.1-rc.2` | Legacy tag published before the release-identity scheme existed. Do not use it for new deployments; see the migration table in the [root README](../README.md). |
Release identities, channels and rollback rules are defined in
[versioning](versioning.md). Tags are immutable, so production deployments should
pin a digest:
```bash
docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
# ... pin the digest in your deployment manifests:
# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest>
```
## Verifying an image
```bash
docker inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2 \
--format '{{json .Config.Labels}}'
# org.opencontainers.image.title / .version / .source
```
Release builds also publish SBOM and provenance attestations from the release
workflow (see [SECURITY.md](../SECURITY.md) and [gitflow](gitflow.md)).

163
docs/versioning.md Normal file
View File

@@ -0,0 +1,163 @@
# Versioning, releases and changelog
[versions.env](../versions.env) is the **single source of truth** for the DSH ref,
the repository revision, the pnpm version, the control-plane component versions
and the image names. Nothing else may hard-code a version:
[scripts/check-versions.sh](../scripts/check-versions.sh) fails when a pin drifts,
and it also verifies that prose mentions only the current release identity or its
base DSH tag.
## Release identity
```
release identity = DSH_VERSION + optional "-" + RELEASE_REVISION
```
| Piece | Meaning |
| --- | --- |
| `DSH_VERSION` | The upstream DeepSeek Harness git ref being packaged, e.g. `dsh-v0.2.1-rc.2`. It mirrors upstream's own tag, which is what "mirrored-DSH scheme" means. |
| `RELEASE_REVISION` | An optional repository revision at that DSH ref: empty for the first release, `rN` (`N >= 2`) for a repository-only re-release. |
Examples:
- `DSH_VERSION=dsh-v0.2.1-rc.2`, `RELEASE_REVISION=` → identity `dsh-v0.2.1-rc.2`
(first release at that upstream ref).
- `DSH_VERSION=dsh-v0.2.1-rc.2`, `RELEASE_REVISION=r2` → identity
`dsh-v0.2.1-rc.2-r2` (documentation, CI or gateway changes on top of the same
upstream DSH; no DSH upgrade).
A DSH upgrade produces a new base identity (for example the next upstream ref
with an empty revision); a repository-only change at the same DSH ref increments
`rN`.
## Why the current identity is `dsh-v0.2.1-rc.2-r2`
The image tag `dsh-v0.2.1-rc.2` was already published to GHCR for an earlier
repository state. **Container tags are immutable** in this project: re-pushing a
tag would silently change what an existing deployment runs. The mirrored-DSH
scheme therefore starts its first identity at `-r2`: the plain tag is
conceptually revision 1 and is taken, so the first publishable repository
revision at the same upstream ref is `r2`.
The identity is stored in [versions.env](../versions.env) and consumed everywhere:
the Dockerfile label, the example manifests, the Makefile and the release
workflow. The legacy `dsh-v0.2.1-rc.2` tag remains published but is not updated;
new deployments use `dsh-v0.2.1-rc.2-r2`.
The gateway image moved from the repository-level `:gateway` tag to a nested
package (`<repository>/dsh-agents-as-a-service/gateway`). For exactly one
release the release workflow also republishes the legacy
`<repository>:gateway` tag as a deprecated alias; it is removed in the next
release, so deployments should pin the nested package.
## Channels
| Channel | Image tag | Meaning |
| --- | --- | --- |
| Release | `dsh-v0.2.1-rc.2-r2` | The current immutable release identity. What production should pin (preferably by digest). |
| `latest` | `latest` | Alias for the most recent release; moves only when a new release is published. |
| `main` | `main` | Development channel, rebuilt from every push to `main`. Never treat it as a release; it exists so `main` can be tested. |
The same channels exist for every published package, including the nested
example component image.
## Compatibility matrix
Values from [versions.env](../versions.env) at the current identity:
| Component | Pin | Source |
| --- | --- | --- |
| Upstream DSH source | `DSH_REPO` | [versions.env](../versions.env) |
| DSH ref | `dsh-v0.2.1-rc.2` | `DSH_VERSION` |
| Repository revision | `r2` | `RELEASE_REVISION` |
| pnpm | `11.28.5` | `PNPM_VERSION` |
| agent-sandbox (CRDs) | `v1.0.7` | `AGENT_SANDBOX_VERSION`, used by the examples and CI dry-run |
| Kubernetes Gateway API | `v1.2.1` | `GATEWAY_API_VERSION`, used by the SSO exposure variant |
| oauth2-proxy | `quay.io/oauth2-proxy/oauth2-proxy:v7.15.5` | `OAUTH2_PROXY_IMAGE` |
| Main image | `ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox` | `GHCR_IMAGE` |
| Gateway image | `ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway` | `GATEWAY_IMAGE`, nested below `GHCR_IMAGE` |
Example component images are always nested below the main package:
`<GHCR_IMAGE>/<example>/<component>`. `check-versions.sh` rejects an unrelated
top-level package.
## Bumping versions
```bash
make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=rN
make check
```
`set-version` updates [versions.env](../versions.env) through
[scripts/set-version.sh](../scripts/set-version.sh) and leaves every consuming
file to derive from it. Never edit the Dockerfile, manifests, Makefile or
workflows with a literal version.
- New upstream DSH ref: change `DSH_VERSION` and clear `RELEASE_REVISION`.
- Repository-only change at the same DSH ref: keep `DSH_VERSION`, set
`RELEASE_REVISION=r(N+1)`.
## Release checklist
1. `main` is green and `make check`, `make test`, `make lint` pass locally.
2. `make image` and `make gateway` build; smoke-test the images.
3. Bump the pins with `make set-version` (commit that change through a PR).
4. `make release-dry-run` — verifies the identity, prints the image names, checks
the tag does not already exist, and previews the generated notes.
5. Create and push the annotated `dsh-*` release tag (or dispatch
`release.yaml` with an existing tag). The workflow validates that the tag
matches [versions.env](../versions.env).
6. The workflow builds and pushes multi-arch images for the main image and the
example components (the identity tag plus `latest`), creates the GitHub
Release from the git-cliff notes, records the published digests, and opens a
`chore(changelog)` PR that refreshes `CHANGELOG.md` on `main`.
7. Merge the changelog PR. Verify:
`docker buildx imagetools inspect <image>:<identity>` shows both platforms,
and the GitHub Release references the tag and the digest.
8. Announce the identity and the digest; deployments pin the digest.
## Rollback
Tags and release identities are **immutable**, so a bad release is never re-tagged
or deleted. Fix forward:
1. Fix the problem on a `hotfix/<slug>` branch (see [gitflow](gitflow.md)).
2. Publish a new identity with the next revision, `-r(N+1)`, at the same
`DSH_VERSION`.
3. Point deployments at the new digest; keep the broken tag for audit history.
Emergency rollback to the previous release is also a re-deploy of a **previous
digest**, not a move of `latest`.
## Changelog
`CHANGELOG.md` is generated by [git-cliff](https://git-cliff.org) from
Conventional Commits, configured in [cliff.toml](../cliff.toml):
```bash
make changelog # regenerate CHANGELOG.md locally
```
- Do not edit `CHANGELOG.md` by hand; the release workflow regenerates it and opens a PR to refresh it on `main`.
- Commit messages and PR titles must be Conventional Commits so they are grouped
correctly (Added / Fixed / Changed / Documentation / Build / CI / Tests /
Maintenance).
- The changelog groups commits since the previous `dsh-*` tag; release notes use
the same rendering.
- The changelog is excluded from the version-token check because it records
historical identities on purpose.
## Digest pinning
Release identities are mutable only in theory; deployments should still pin the
resolved digest for reproducibility and supply-chain safety:
```bash
docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
# then, in the deployment:
# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest>
```
The example manifests use the identity tag so they stay readable and upgrade with
`versions.env`; production overlays should pin the digest. Release images carry
SBOM and provenance attestations (see [SECURITY.md](../SECURITY.md)).

47
examples/README.md Normal file
View File

@@ -0,0 +1,47 @@
# Examples
Every implementation in this repository is a self-contained example: one
directory, plain `kubectl`, an end-to-end README, and a cleanup command.
Examples consume the [main DSH Web image](../docs/main-image.md); they are
reference material, not libraries.
| Example | What it demonstrates | Needs |
| --- | --- | --- |
| [`examples/dsh-agents-as-a-service/`](dsh-agents-as-a-service/) | The full multi-tenant platform pattern: one `SandboxTemplate` with per-user PVCs, a warm pool, one ~10-line `SandboxClaim` per user, suspend-when-idle / wake-on-return, and a browser gateway with signup, `HttpOnly` sessions, an HTTP + WebSocket proxy and two exposure variants (Ingress, or Gateway API + GitHub SSO). | agent-sandbox with extensions; kind for the e2e script. |
| [`examples/minimal-single-tenant/`](minimal-single-tenant/) | The smallest useful setup: a single `SandboxTemplate` and one `SandboxClaim`, no gateway — reach DSH Web with a launch-token URL through `kubectl port-forward`. | agent-sandbox with extensions. |
## The example contract (short version)
Full rules: [docs/examples.md](../docs/examples.md). `scripts/check-examples.sh`
enforces the layout, and `make check` runs it.
```
examples/<name>/
+-- README.md # walkthrough: prerequisites, steps, troubleshooting, cleanup
+-- manifests/ # NN-*.yaml, applied with plain kubectl
| +-- exposure/<variant>/ # optional per-variant manifests...
| +-- exposure/README.md # ...required whenever exposure/ exists
+-- scripts/ # optional executable, idempotent, self-cleaning *.sh
+-- docs/ # optional architecture/ADR material
+-- <component>/ # optional component with Dockerfile + README + tests
```
- Secrets never ship populated: use `*.yaml.example` files with placeholders.
- Manifests are numbered (`00-`, `10-`, ...) and read as an ordered walkthrough.
- Component images are published nested below the main image:
`<registry>/<owner>/<repo>/<example>/<component>`.
- Every example must be listed above with its literal `examples/<name>/` path.
- Scripts under `scripts/` must be executable, idempotent and clean up after
themselves.
## Running an example
```bash
# e.g. the multi-tenant platform example
kubectl apply -f examples/dsh-agents-as-a-service/manifests/
# or run the whole walkthrough (kind + agent-sandbox required) and let it clean up
examples/dsh-agents-as-a-service/scripts/e2e-kind.sh
```
Each example README has the exact prerequisites, exposure options and teardown.

View File

@@ -2,20 +2,41 @@
This example distills the upstream This example distills the upstream
[hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service) [hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service)
platform pattern into pure agent-sandbox resources + kubectl, running **DeepSeek Harness Web platform pattern into pure agent-sandbox resources + kubectl, running **DeepSeek
(DSH)** instead of Hermes. Every user of such a platform gets a personal, long-lived DSH Web Harness Web (DSH)** instead of Hermes. Every user of such a platform gets a
instance that is **provisioned in ~2 seconds from a warm pool, suspended when idle to save personal, long-lived DSH Web instance that is **provisioned in ~2 seconds from a
cost, and woken transparently when the user returns** — with all state (conversations, warm pool, suspended when idle to save cost, and woken transparently when the
sessions, config) surviving on a per-user PVC at `/home/node/.dsh`. user returns** — with all state (conversations, sessions, config) surviving on a
per-user PVC at `/home/node/.dsh`.
It ships in two layers: It ships in three layers:
1. **The resource layer** (steps 1–7): pure manifests + kubectl, so every mechanism is visible. 1. **The resource layer** (steps 1–7): pure manifests + kubectl, so every
2. **The service layer** (step 8): a small included **gateway** (`gateway/`) that makes the mechanism is visible.
platform browser-usable — a signup page/API, `HttpOnly` cookie sessions, a `/u/{user}/**` 2. **The service layer** (step 8): a small included **gateway** (`gateway/`) that
proxy for HTTP **and WebSocket** that transparently **wakes suspended instances on makes the platform browser-usable — a signup page/API, `HttpOnly` cookie
connect**, and an idle sweeper that suspends them again. The gateway performs the DSH sessions, a `/u/{user}/**` proxy for HTTP **and WebSocket** that transparently
launch-token handshake server-side, so the browser never sees the token. **wakes suspended instances on connect**, and an idle sweeper that suspends
them again. The gateway performs the DSH launch-token handshake server-side,
so the browser never sees the token.
3. **The exposure layer** (step 9): choose exactly one way in — a plain Traefik
Ingress, or Gateway API + oauth2-proxy (GitHub SSO).
All manifests live under [`manifests/`](manifests/):
```
manifests/
+-- 00-prereqs.yaml # namespace + empty provider-key Secret
+-- 10-sandboxtemplate.yaml # the DSH Web blueprint
+-- 20-sandboxwarmpool.yaml # 2 pre-warmed spares
+-- 30-claim-alice.yaml # one user
+-- 40-claim-rejected.yaml # negative demo: env injection rejected
+-- 50-gateway.yaml # Deployment/RBAC/Service for the gateway
+-- exposure/
+-- README.md # the two exposure variants, when to use which
+-- ingress/60-ingress.yaml
+-- gateway-api-sso/70-gateway.yaml ... 74-tls-secret.yaml.example
```
What it demonstrates, feature by feature: What it demonstrates, feature by feature:
@@ -34,20 +55,21 @@ What it demonstrates, feature by feature:
- **Multi-arch**: the DSH image is built for amd64 and arm64, so there is no - **Multi-arch**: the DSH image is built for amd64 and arm64, so there is no
`kubernetes.io/arch: amd64` nodeSelector. `kubernetes.io/arch: amd64` nodeSelector.
- **Single port**: DSH Web serves everything on `3080`; the gateway proxies to one upstream - **Single port**: DSH Web serves everything on `3080`; the gateway proxies to one
instead of splitting `/v1/*` (API) from the dashboard. upstream instead of splitting `/v1/*` (API) from the dashboard.
- **One shared secret**: only `DEEPSEEK_API_KEY` is injected (via `envFrom`); there is no - **One shared secret**: only `DEEPSEEK_API_KEY` is injected (via `envFrom`); there
dashboard basic-auth, session-cookie secret, or API-server key. is no dashboard basic-auth, session-cookie secret, or API-server key.
- **Launch-token handshake**: DSH gates `/` behind a token it prints only in its logs. The - **Launch-token handshake**: DSH gates `/` behind a token it prints only in its
gateway reads it, performs the handshake, and keeps the resulting auth cookie server-side; logs. The gateway reads it, performs the handshake, and keeps the resulting
the browser authenticates to the gateway with its own cookie. See auth cookie server-side; the browser authenticates to the gateway with its own
[How the DSH handshake works](#how-the-dsh-handshake-works). cookie. See [How the DSH handshake works](#how-the-dsh-handshake-works).
- **Host allow-list**: DSH rejects unknown `Host` headers. The gateway forwards the browser's - **Host allow-list**: DSH rejects unknown `Host` headers. The gateway forwards
`Host`, so the public hostname must be added to `DSH_TRUSTED_HOSTS` in the template. the browser's `Host`, so the public hostname must be added to
- **Probes target `/favicon.svg`**: `/` returns `401` without the launch token, so a probe on `DSH_TRUSTED_HOSTS` in the template.
`/` crash-loops the pod. `/favicon.svg` is unauthenticated, and a `startupProbe` gates - **Probes target `/favicon.svg`**: `/` returns `401` without the launch token, so
liveness/readiness so a slow cold start (plugin activation, or a resumed sandbox) is not a probe on `/` crash-loops the pod. `/favicon.svg` is unauthenticated, and a
restarted mid-boot. `startupProbe` gates liveness/readiness so a slow cold start (plugin
activation, or a resumed sandbox) is not restarted mid-boot.
## Prerequisites ## Prerequisites
@@ -55,43 +77,49 @@ What it demonstrates, feature by feature:
- agent-sandbox with extensions installed: - agent-sandbox with extensions installed:
```bash ```bash
VERSION=v1.0.7 VERSION=v1.0.7 # AGENT_SANDBOX_VERSION in versions.env
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/sandbox-with-extensions.yaml kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$VERSION/sandbox-with-extensions.yaml
kubectl -n agent-sandbox-system rollout status deployment --timeout=180s kubectl -n agent-sandbox-system rollout status deployment --timeout=180s
``` ```
Or run everything below in one shot with `./run-test-kind.sh` (expects a kind cluster + the For step 9's SSO variant you additionally need the Gateway API CRDs and a
manifest above already applied; it creates and deletes the namespace itself). Gateway implementation (`gatewayClassName: traefik` in the manifests); see the
[exposure README](manifests/exposure/README.md).
- Optional: run the whole resource + gateway walkthrough in one shot with
[`scripts/e2e-kind.sh`](scripts/e2e-kind.sh). It expects a kind cluster with
agent-sandbox installed, then creates and deletes the `dsh-demo` namespace
itself. It stops after the gateway layer (step 8); exposure is not applied.
## 1. Install the platform resources ## 1. Install the platform resources
First the namespace, then the shared provider key. The key reaches **every** tenant sandbox First the namespace, then the shared provider key. The key reaches **every**
(a warm-pool constraint: pod env is stamped before any user exists), so it is **never tenant sandbox (a warm-pool constraint: pod env is stamped before any user
committed** — generate a fresh value per install: exists), so it is **never committed** — generate a fresh value per install:
```bash ```bash
kubectl apply -f 00-prereqs.yaml kubectl apply -f manifests/00-prereqs.yaml
kubectl -n dsh-demo create secret generic dsh-provider-keys \ kubectl -n dsh-demo create secret generic dsh-provider-keys \
--from-literal=DEEPSEEK_API_KEY="sk-..." --from-literal=DEEPSEEK_API_KEY="sk-..."
kubectl apply -f 10-sandboxtemplate.yaml -f 20-sandboxwarmpool.yaml kubectl apply -f manifests/10-sandboxtemplate.yaml -f manifests/20-sandboxwarmpool.yaml
``` ```
Watch the two warm spares boot (first time pulls the image — subsequent operations never pay Watch the two warm spares boot (first time pulls the image — subsequent operations
that again; that is the point of the pool): never pay that again; that is the point of the pool):
```bash ```bash
kubectl -n dsh-demo get sandboxes -l agents.x-k8s.io/warm-pool-sandbox -w kubectl -n dsh-demo get sandboxes -l agents.x-k8s.io/warm-pool-sandbox -w
``` ```
Wait until both show `READY: True`. Each spare already has its PVC bound and DSH Web serving Wait until both show `READY: True`. Each spare already has its PVC bound and DSH
on port `3080`. Web serving on port `3080`.
## 2. "Signup": claim an instance (~2s) ## 2. "Signup": claim an instance (~2s)
A user signing up is one small claim — `warmPoolRef` plus a pod label: A user signing up is one small claim — `warmPoolRef` plus a pod label:
```bash ```bash
kubectl apply -f 30-claim-alice.yaml kubectl apply -f manifests/30-claim-alice.yaml
kubectl -n dsh-demo wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s kubectl -n dsh-demo wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s
``` ```
@@ -105,20 +133,21 @@ kubectl -n dsh-demo get sandboxes --show-labels
Three things worth noticing: Three things worth noticing:
- **The sandbox keeps its pool-generated name** (`dsh-pool-…`), not the claim's name — - **The sandbox keeps its pool-generated name** (`dsh-pool-…`), not the claim's
adoption re-labels an existing warm sandbox rather than creating one. A platform must always name — adoption re-labels an existing warm sandbox rather than creating one. A
resolve user → sandbox through `claim.status.sandbox.name`. platform must always resolve user → sandbox through `claim.status.sandbox.name`.
- The `agents.x-k8s.io/warm-pool-sandbox` label is **gone** from the adopted sandbox (it left - The `agents.x-k8s.io/warm-pool-sandbox` label is **gone** from the adopted
the pool's inventory), and the pool has already started replenishing back to 2 spares. sandbox (it left the pool's inventory), and the pool has already started
replenishing back to 2 spares.
- The pod carries the claim's `additionalPodMetadata` label - The pod carries the claim's `additionalPodMetadata` label
(`sandbox.users.io/dsh-user=alice`), and (`sandbox.users.io/dsh-user=alice`), and
`kubectl -n dsh-demo get sandbox $SB -o jsonpath='{.status.serviceFQDN}'` shows the stable `kubectl -n dsh-demo get sandbox $SB -o jsonpath='{.status.serviceFQDN}'` shows
DNS name the gateway proxies to. the stable DNS name the gateway proxies to.
## 3. Talk to the instance ## 3. Talk to the instance
DSH Web needs its launch token on first contact — a bare `/` returns `401`. The token is in the DSH Web needs its launch token on first contact — a bare `/` returns `401`. The
pod logs: token is in the pod logs:
```bash ```bash
kubectl -n dsh-demo logs "$SB" | grep -m1 '?token=' kubectl -n dsh-demo logs "$SB" | grep -m1 '?token='
@@ -127,16 +156,16 @@ kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
# open http://localhost:18081/?token=... (from the log line) # open http://localhost:18081/?token=... (from the log line)
``` ```
> **Local port note:** DSH Web listens on `3080` **in-cluster**. If `3080` is already in use > **Local port note:** DSH Web listens on `3080` **in-cluster**. If `3080` is
> on your workstation (e.g. by a local DSH), forward to a different local port (here `18081`), > already in use on your workstation (e.g. by a local DSH), forward to a
> never `3080`. `localhost` is always in DSH's trusted-host list, so the forwarded request is > different local port (here `18081`), never `3080`. `localhost` is always in
> accepted. > DSH's trusted-host list, so the forwarded request is accepted.
> **Why the pod stays up:** this claim was applied by hand, so it carries no gateway ownership > **Why the pod stays up:** this claim was applied by hand, so it carries no
> annotation (`aaas.example.com/managed-by`). The gateway's idle sweeper only suspends claims it > gateway ownership annotation (`aaas.example.com/managed-by`). The gateway's idle
> created through `POST /users`; the resource-layer claims from these manifests are never > sweeper only suspends claims it created through `POST /users`; the resource-layer
> touched. You can deploy the gateway (step 8) and still keep a sandbox you are port-forwarding > claims from these manifests are never touched. You can deploy the gateway
> to. > (step 8) and still keep a sandbox you are port-forwarding to.
## 4. Write state on the PVC ## 4. Write state on the PVC
@@ -161,20 +190,20 @@ kubectl -n dsh-demo wait sandbox "$SB" --for=condition=Ready --timeout=180s
kubectl -n dsh-demo exec "$SB" -- cat /home/node/.dsh/marker.txt # "remember me" kubectl -n dsh-demo exec "$SB" -- cat /home/node/.dsh/marker.txt # "remember me"
``` ```
> **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`. DSH requires > **Why resume works:** the pod template sets `fsGroupChangePolicy: OnRootMismatch`.
> its `.credentials.yaml` to be `0600`, while the kubelet's default fsGroup policy widens every > DSH requires its `.credentials.yaml` to be `0600`, while the kubelet's default
> file to `0660` on each pod start — a resumed sandbox would crash-loop. See > fsGroup policy widens every file to `0660` on each pod start — a resumed sandbox
> [Troubleshooting](#troubleshooting) for the one-time fix on sandboxes created before this > would crash-loop. See [Troubleshooting](#troubleshooting) for the one-time fix
> setting existed. > on sandboxes created before this setting existed.
## 7. Injection policy rejects env-carrying claims ## 7. Injection policy rejects env-carrying claims
```bash ```bash
kubectl apply -f 40-claim-rejected.yaml kubectl apply -f manifests/40-claim-rejected.yaml
kubectl -n dsh-demo get sandboxclaim dsh-mallory \ kubectl -n dsh-demo get sandboxclaim dsh-mallory \
-o jsonpath='{.status.conditions[?(@.type=="Ready")].reason}' -o jsonpath='{.status.conditions[?(@.type=="Ready")].reason}'
# EnvVarsInjectionRejected — the claim never bound a sandbox # EnvVarsInjectionRejected — the claim never bound a sandbox
kubectl delete -f 40-claim-rejected.yaml kubectl delete -f manifests/40-claim-rejected.yaml
``` ```
## 8. The gateway: signup, browser proxy, idle-suspend, wake-on-connect ## 8. The gateway: signup, browser proxy, idle-suspend, wake-on-connect
@@ -182,59 +211,66 @@ kubectl delete -f 40-claim-rejected.yaml
### Deploy it ### Deploy it
```bash ```bash
# Published image (on kind, load it into the cluster; on a real cluster it is # Published image (on kind, load it into the cluster; on a real cluster the
# pulled from GHCR): # kubelet pulls it from GHCR):
docker pull ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway GW=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2
kind load docker-image ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway --name <kind-cluster> docker pull "$GW"
kind load docker-image "$GW" --name <kind-cluster>
# ... or build it locally and tag it exactly as 50-gateway.yaml expects: # ... or build it locally and tag it exactly as manifests/50-gateway.yaml
docker build -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway gateway/ # expects (make gateway builds dsh-aaas-gateway:dev):
kind load docker-image ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway --name <kind-cluster> make gateway
docker tag dsh-aaas-gateway:dev "$GW"
kind load docker-image "$GW" --name <kind-cluster>
kubectl apply -f 50-gateway.yaml kubectl apply -f manifests/50-gateway.yaml
kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s
kubectl -n dsh-demo port-forward svc/dsh-gateway 18080:8080 kubectl -n dsh-demo port-forward svc/dsh-gateway 18080:8080
``` ```
> The gateway binds `8080`; the local port-forward uses `18080` and never `3080`. > The gateway binds `8080`; the local port-forward uses `18080` and never `3080`.
> >
> **Re-deploying after a new image?** The workflow publishes the mutable `:gateway` tag and the > **Image pull policy.** `manifests/50-gateway.yaml` uses
> Deployment uses `imagePullPolicy: IfNotPresent`, so a node that already cached the tag keeps > `imagePullPolicy: IfNotPresent` with the release identity, so a locally built
> the old image across `kubectl apply`/`rollout restart`. Pin the digest for a reproducible > image must be loaded into the cluster under exactly that name (kind:
> rollout (the workflow's `Inspect image` step and `docker buildx imagetools inspect` both show > `kind load`). A node that already cached a different tag is the usual cause of
> it): > "my new build is not running"; pin the digest for a reproducible rollout:
> >
> ```bash > ```bash
> kubectl -n dsh-demo set image deploy/dsh-gateway \ > kubectl -n dsh-demo set image deploy/dsh-gateway \
> gateway=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest> > gateway=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway@sha256:<digest>
> kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s > kubectl -n dsh-demo rollout status deploy/dsh-gateway --timeout=120s
> ``` > ```
> >
> **Applying over an existing install?** Run `kubectl diff -f 10-sandboxtemplate.yaml` > **Applying over an existing install?** Run
> first. Cluster-specific fields such as `nodeAffinity` and the warm-pool replica count are > `kubectl diff -f manifests/10-sandboxtemplate.yaml` first. Cluster-specific
> intentionally not in the repo manifests; preserve them with a targeted patch instead of a > fields such as `nodeAffinity` and the warm-pool replica count are intentionally
> not in the repo manifests; preserve them with a targeted patch instead of a
> blind apply. Moving the probes to `/favicon.svg` is the crash-loop fix. > blind apply. Moving the probes to `/favicon.svg` is the crash-loop fix.
> >
> The template also sets `fsGroupChangePolicy: OnRootMismatch` (see the note in step 6). > The template also sets `fsGroupChangePolicy: OnRootMismatch` (see the note in
> Existing `Sandbox` objects snapshot their pod template, so a template apply does not update > step 6). Existing `Sandbox` objects snapshot their pod template, so a template
> them: patch each sandbox's `spec.podTemplate.spec.securityContext.fsGroupChangePolicy` and > apply does not update them: patch each sandbox's
> `spec.podTemplate.spec.securityContext.fsGroupChangePolicy` and
> `chmod 600` its `.credentials.yaml`, or recreate it. > `chmod 600` its `.credentials.yaml`, or recreate it.
### Browser flow ### Browser flow
Open <http://localhost:18080>. The landing page has a one-field signup form. Enter `bob`, and Open <http://localhost:18080>. The landing page has a one-field signup form.
the gateway: Enter `bob`, and the gateway:
1. creates the `SandboxClaim` over the warm pool, 1. creates the `SandboxClaim` over the warm pool,
2. waits for it to become `Ready`, 2. waits for it to become `Ready`,
3. sets an `HttpOnly`, `SameSite=Lax` cookie `dsh_gw_session=<user>:<token>` on the browser, 3. sets an `HttpOnly`, `SameSite=Lax` cookie `dsh_gw_session=<user>:<token>` on
the browser,
4. redirects to `/u/bob/`, where the DSH Web UI loads through the gateway. 4. redirects to `/u/bob/`, where the DSH Web UI loads through the gateway.
From then on every request under `/u/bob/` (assets, API calls, SSE, WebSocket) is proxied to From then on every request under `/u/bob/` (assets, API calls, SSE, WebSocket) is
bob's sandbox. The DSH launch token and DSH's own auth cookie stay in the gateway; the browser proxied to bob's sandbox. The DSH launch token and DSH's own auth cookie stay in
only holds the gateway cookie. the gateway; the browser only holds the gateway cookie.
The same flow with `curl`, no JavaScript (the session cookie is the only credential): The same flow with `curl`, no JavaScript (the session cookie is the only
credential):
```bash ```bash
JAR=$(mktemp) JAR=$(mktemp)
@@ -252,48 +288,53 @@ curl -s -b "$JAR" -H 'Origin: http://localhost:18080' -X DELETE http://localhost
``` ```
API clients that prefer bearer tokens can use `POST /users` with a JSON body API clients that prefer bearer tokens can use `POST /users` with a JSON body
(`{"user":"bob"}`) and the returned one-time token, `POST /login` to exchange a token for a (`{"user":"bob"}`) and the returned one-time token, `POST /login` to exchange a
cookie, and `Authorization: Bearer <token>` on subsequent calls. token for a cookie, and `Authorization: Bearer <token>` on subsequent calls.
### Gateway API ### Gateway API
| Route | Purpose | | Route | Purpose |
| --- | --- | | --- | --- |
| `GET /healthz` | Liveness/readiness; no cluster access | | `GET /healthz` | Liveness/readiness; no cluster access |
| `GET /` | Landing page: signup, token login, logout | | `GET /` | Landing page: signup, token login, logout. With SSO it auto-provisions and connects instead. |
| `POST /users` | Signup. JSON or form; `201` (JSON), `202` (waking), `303` (browser redirect), `409` if user exists; sets the session cookie | | `POST /users` | Signup. JSON or form; `201` (JSON), `202` (waking), `303` (browser redirect), `409` if user exists; sets the session cookie |
| `POST /login` | Exchange a bearer token for the session cookie | | `POST /login` | Exchange a bearer token for the session cookie |
| `POST /logout` | Clear the session cookie | | `POST /logout` | Clear the session cookie |
| `POST /recreate` | SSO only: delete the authenticated user's claim (wiping workspace data) and provision a fresh one; `303` to `/u/{user}/`. Cookie-authenticated calls must be same-origin. |
| `GET /users/{user}` | Derived state (`Ready`/`Waking`/`Suspended`) | | `GET /users/{user}` | Derived state (`Ready`/`Waking`/`Suspended`) |
| `DELETE /users/{user}` | Delete claim (cascades sandbox + PVC) | | `DELETE /users/{user}` | Delete claim (cascades sandbox + PVC) |
| `GET /u/{user}` | `308` redirect to `/u/{user}/` | | `GET /u/{user}` | `308` redirect to `/u/{user}/` |
| `GET /u/{user}/**` | HTTP/SSE proxy to the user's DSH Web | | `GET /u/{user}/**` | HTTP/SSE proxy to the user's DSH Web |
| `WS /u/{user}/**` | WebSocket proxy to the user's DSH Web | | `WS /u/{user}/**` | WebSocket proxy to the user's DSH Web |
The full API reference, configuration table, auth semantics and failure matrix
live in the [gateway README](gateway/README.md).
### How the DSH handshake works ### How the DSH handshake works
DSH does not expose an API for its launch token; it prints a URL containing `?token=…` in the DSH does not expose an API for its launch token; it prints a URL containing
pod logs. For each `(user, pod UID, Host)` the gateway: `?token=…` in the pod logs. For each `(user, pod UID, Host)` the gateway:
1. resolves the claim → sandbox → pod (annotation `agents.x-k8s.io/pod-name`, or the sandbox 1. resolves the claim → sandbox → pod (annotation `agents.x-k8s.io/pod-name`, or
name as pod name), the sandbox name as pod name),
2. reads the pod's logs (last `DSH_LOG_TAIL_LINES` lines) and parses the most recent 2. reads the pod's logs (last `DSH_LOG_TAIL_LINES` lines) and parses the most
`dsh web: …?token=…` line, recent `dsh web: …?token=…` line,
3. requests `http://<serviceFQDN>:3080/?token=…` with the browser's `Host`, 3. requests `http://<serviceFQDN>:3080/?token=…` with the browser's `Host`,
4. captures DSH's `dsh-auth-*` `Set-Cookie` response and caches it, 4. captures DSH's `dsh-auth-*` `Set-Cookie` response and caches it,
5. injects that cookie (and strips the browser's `Cookie`/`Authorization` and any `?token=`) 5. injects that cookie (and strips the browser's `Cookie`/`Authorization` and any
on every proxied HTTP and WebSocket request, `?token=`) on every proxied HTTP and WebSocket request,
6. refreshes the handshake once if DSH answers `401`, and drops the cached cookie when the 6. refreshes the handshake once if DSH answers `401`, and drops the cached cookie
sandbox suspends or is deleted. when the sandbox suspends or is deleted.
This is why the gateway's RBAC includes `pods` `get`/`list` and `pods/log` `get`. It is also This is why the gateway's RBAC includes `pods` `get`/`list` and `pods/log`
the most fragile part of the design; see [Limitations](#limitations). `get`. It is also the most fragile part of the design; see
[Limitations](#limitations).
### DSH_TRUSTED_HOSTS ### DSH_TRUSTED_HOSTS
DSH rejects requests whose `Host` is not allow-listed (`localhost` and `127.0.0.1` are always DSH rejects requests whose `Host` is not allow-listed (`localhost` and
trusted). Because the gateway forwards the browser's `Host`, set the public hostname in the `127.0.0.1` are always trusted). Because the gateway forwards the browser's
template: `Host`, set the public hostname in the template:
```yaml ```yaml
env: env:
@@ -301,36 +342,151 @@ env:
value: "dsh.example.com" # comma or space separated; empty for port-forward only value: "dsh.example.com" # comma or space separated; empty for port-forward only
``` ```
The same applies to a local `docker run`: the printed token URL uses `DSH_PUBLIC_URL` if set, The same applies to a local `docker run`: the printed token URL uses
otherwise `http://localhost:${DSH_PORT}`. `DSH_PUBLIC_URL` if set, otherwise `http://localhost:<DSH_PORT>`.
### WebSocket ### WebSocket
The gateway is a Starlette ASGI app and proxies WebSocket upgrades with a bidirectional pump, so The gateway is a Starlette ASGI app and proxies WebSocket upgrades with a
the DSH web GUI is fully interactive. Requirements: bidirectional pump, so the DSH web GUI is fully interactive. Requirements:
- the ingress/load balancer in front of the gateway must pass `Upgrade`/`Connection` headers; - the ingress/load balancer in front of the gateway must pass `Upgrade`/`Connection`
`60-ingress.yaml` is a starting point (Traefik handles upgrades by default), headers; [`manifests/exposure/ingress/60-ingress.yaml`](manifests/exposure/ingress/60-ingress.yaml)
- idle suspension is disabled for users with in-flight proxied requests or open WebSockets, so is a starting point (Traefik handles upgrades by default),
a long-running chat stream is not cut off mid-turn, - idle suspension is disabled for users with in-flight proxied requests or open
- the upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`, and performs WebSockets, so a long-running chat stream is not cut off mid-turn,
the launch-token handshake under that same `pod-IP:3080` authority. DSH fences on `Host` - the upstream upgrade dials the sandbox **pod IP**, forwards no `Host`/`Origin`,
(only its bind address, loopback, or an allow-listed host passes) and binds its auth cookie to and performs the launch-token handshake under that same `pod-IP:3080` authority.
the request authority (`payload.authority === authority`), while `websockets` derives `Host` DSH fences on `Host` (only its bind address, loopback, or an allow-listed host
from the URI and cannot be overridden. A browser-host cookie or forwarded origin makes the passes) and binds its auth cookie to the request authority
upgrade fail (`401`/`403`, surfaced as close `1011`), which the client retries forever as (`payload.authority === authority`), while `websockets` derives `Host` from the
`[connection] connection lost, retry #N`, URI and cannot be overridden. A browser-host cookie or forwarded origin makes
- missing/invalid gateway auth is rejected right after the upgrade with close code `4401` the upgrade fail (`401`/`403`, surfaced as close `1011`), which the client
(WebSocket auth has no HTTP `401` path); cross-origin or Origin-less upgrades with the cookie retries forever as `[connection] connection lost, retry #N`,
are rejected the same way (CSWSH protection), so clients must send a matching `Origin`; - missing/invalid gateway auth is rejected right after the upgrade with close code
upstream bridging failures close with `1011`. `4401` (WebSocket auth has no HTTP `401` path); cross-origin or Origin-less
upgrades with the cookie are rejected the same way (CSWSH protection), so
clients must send a matching `Origin`; upstream bridging failures close with
`1011`.
### Path routing ### Path routing
Each user's DSH Web is served under `/u/<user>/`. DSH injects `<base href="./">` and uses Each user's DSH Web is served under `/u/<user>/`. DSH injects `<base href="./">`
relative asset/API URLs, so the UI works under a path prefix without a per-user hostname or and uses relative asset/API URLs, so the UI works under a path prefix without a
wildcard DNS. The gateway also rewrites absolute-path `Location` headers into `/u/<user>/…` and per-user hostname or wildcard DNS. The gateway also rewrites absolute-path
redirects `/u/<user>` to the trailing-slash form with `308` so relative URLs resolve correctly. `Location` headers into `/u/<user>/…` and redirects `/u/<user>` to the
trailing-slash form with `308` so relative URLs resolve correctly.
## 9. Expose it: choose exactly one variant
The base manifests expose nothing outside the cluster (`dsh-gateway` is a
ClusterIP Service on port `8080`). Pick **one** variant — they are alternatives,
not a stack. Details, commands and the switch procedure are in the
[exposure README](manifests/exposure/README.md).
> Apply the base resources first, then the variant **by path**. Do not use
> `kubectl apply -R`: it would apply both variants and create two paths to the
> same gateway, which is unsafe once SSO is enabled.
### Option A — plain Ingress (gateway's own session auth)
```bash
# Replace the dsh.example.com host and TLS Secret (or drop the tls: block for
# plain HTTP), then:
kubectl apply -f manifests/exposure/ingress/
```
1. Set `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml` to the same
public host (the gateway forwards the browser's `Host`) and re-apply the
template.
2. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS:
`kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`.
(The base manifest keeps `false` so the plain-HTTP port-forward demo works.)
3. Verify: `curl -sI https://<host>/healthz` returns `200`; sign up in a browser
at `https://<host>/`.
Keep the router free of buffering/compression middlewares: they break WebSocket
upgrades and streaming.
### Option B — Gateway API + GitHub SSO (oauth2-proxy)
`manifests/exposure/gateway-api-sso/` puts a TLS `Gateway` + `HTTPRoute` in
front of **oauth2-proxy**, which authenticates the browser against GitHub
(restricted to a configured organization) and forwards to the gateway with
`X-Forwarded-User` set. The gateway trusts that header only when
`AUTH_PROXY_HEADER` is set (SSO is **opt-in**).
Prerequisites:
- Gateway API CRDs and a Gateway implementation; the manifests assume
`gatewayClassName: traefik` (the CRD version is pinned as
`GATEWAY_API_VERSION` in `versions.env`).
- A GitHub OAuth App whose homepage/callback match the public FQDN
(`https://<host>/` and `https://<host>/oauth2/callback`).
- The oauth2-proxy credentials Secret (template:
`73-oauth2-proxy-secret.yaml.example`) and a TLS Secret for the Gateway
listener (template: `74-tls-secret.yaml.example`).
Steps:
```bash
# 1. Real Secrets; the committed *.yaml.example files are templates only.
kubectl -n dsh-demo create secret generic oauth2-proxy \
--from-literal=client-id='<GitHub OAuth App client ID>' \
--from-literal=client-secret='<GitHub OAuth App client secret>' \
--from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"
kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key
# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72).
# 3. Enable SSO on the gateway and trust the public host in the template.
kubectl -n dsh-demo set env deploy/dsh-gateway \
AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true
# ... and set DSH_TRUSTED_HOSTS=dsh-demo.example.com in
# manifests/10-sandboxtemplate.yaml, then re-apply the template.
# 4. Apply the variant.
kubectl apply -f manifests/exposure/gateway-api-sso/
```
Verification:
```bash
kubectl get gateway dsh-gateway -n dsh-demo -o yaml # Programmed=True
kubectl get httproute -n dsh-demo -o wide # backend oauth2-proxy:4180, Accepted
kubectl get deploy oauth2-proxy -n dsh-demo # 1/1 Ready
kubectl get endpoints oauth2-proxy -n dsh-demo # backend present
kubectl -n dsh-demo logs deploy/oauth2-proxy --tail=50 # no config errors
```
Browser check: open `https://<host>/` — you are redirected to GitHub, and after
signing in as a member of the allowed organization the gateway auto-provisions
your sandbox (the first page can show `Provisioning`) and connects you. A
non-member is denied before reaching the gateway. GitHub logins are lowercase
DNS-1123 labels, so they map 1:1 onto the gateway's `user` validation. The SSO
landing page also offers **Recreate sandbox** (`POST /recreate`), which deletes
the claim — wiping the workspace PVC — and provisions a fresh one.
Sign-out: the gateway `/logout` clears `dsh_gw_session`; oauth2-proxy's session
can be cleared at `/oauth2/sign_out`.
> **SSO trust boundary — read this before enabling it.** After SSO is on, the
> `dsh-gateway` Service must be reachable **only** through oauth2-proxy (it is
> ClusterIP). Anyone who can reach `dsh-gateway:8080` directly can send
> `X-Forwarded-User: <victim>` and be treated as that user. Never apply the
> Ingress variant (or any other route to the Service) while SSO is enabled, and
> do not expose the gateway Service with `kubectl port-forward` on a shared
> machine while `AUTH_PROXY_HEADER` is set. To turn SSO off, remove the variable
> (`kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER-`) and
> delete the SSO variant; the gateway falls back to bearer-token/session auth.
### Switching variants
Delete the variant you had before applying the other one
(`kubectl delete -f manifests/exposure/ingress/`), revert
`AUTH_PROXY_HEADER`/`COOKIE_SECURE`, and verify with
`kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to the
gateway remains.
## Troubleshooting ## Troubleshooting
@@ -346,36 +502,65 @@ redirects `/u/<user>` to the trailing-slash form with `308` so relative URLs res
| Console spams `[connection] connection lost, retry #N` | The upstream WS upgrade is being rejected, surfaced as close `1011`. The gateway dials the sandbox **pod IP**, forwards no `Host`/`Origin`, and handshakes under the same `pod-IP:3080` authority DSH binds its cookie to; an older gateway image, or a sandbox with an empty `status.podIPs`, hits this loop | | Console spams `[connection] connection lost, retry #N` | The upstream WS upgrade is being rejected, surfaced as close `1011`. The gateway dials the sandbox **pod IP**, forwards no `Host`/`Origin`, and handshakes under the same `pod-IP:3080` authority DSH binds its cookie to; an older gateway image, or a sandbox with an empty `status.podIPs`, hits this loop |
| `manifest.webmanifest` returns `401` in the console | Browsers fetch the Web App Manifest without credentials, so the gateway's session cookie is absent and the request fails closed. The UI is unaffected; only the PWA install prompt is lost | | `manifest.webmanifest` returns `401` in the console | Browsers fetch the Web App Manifest without credentials, so the gateway's session cookie is absent and the request fails closed. The UI is unaffected; only the PWA install prompt is lost |
| Signup is slow / `202` then `Waking` | Warm pool exhausted (`kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox`); scale the pool | | Signup is slow / `202` then `Waking` | Warm pool exhausted (`kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox`); scale the pool |
| `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `50-gateway.yaml`) | | `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `manifests/50-gateway.yaml`) |
| Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Upgrade the gateway to a build that honors the marker, or resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` | | Gateway pod `ErrImagePull` / `ImagePullBackOff` (especially on kind) | The manifest uses `imagePullPolicy: IfNotPresent` with the release identity. Build with `make gateway`, retag to exactly the manifest's image, `kind load docker-image <that-ref>`, or set the Deployment image to your local tag / pinned digest. On a real cluster, make sure the GHCR package is readable (public or a pull secret) |
| Resume crash-loops: `credentials-local: … is readable beyond its owner (mode 660)` | The pod template lacks `fsGroupChangePolicy: OnRootMismatch`, so the kubelet's default fsGroup policy rewrote `.credentials.yaml` to `0660`; DSH requires `0600` and refuses to start. New installs get the policy from the template. For an existing sandbox: suspend it, run a throwaway pod on `dsh-home-$SB` to `chmod 600 /home/node/.dsh/.credentials.yaml`, patch the sandbox's `podTemplate` with the policy, then resume. | | Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` |
| Resume crash-loops: `credentials-local: … is readable beyond its owner (mode 660)` | The pod template lacks `fsGroupChangePolicy: OnRootMismatch`, so the kubelet's default fsGroup policy rewrote `.credentials.yaml` to `0660`; DSH requires `0600` and refuses to start. New installs get the policy from the template. For an existing sandbox: suspend it, run a throwaway pod on `dsh-home-$SB` to `chmod 600 /home/node/.dsh/.credentials.yaml`, patch the sandbox's `podTemplate` with the policy, then resume |
| SSO: GitHub login succeeds but the gateway asks for a token / returns `401` to `/recreate` | `AUTH_PROXY_HEADER` is not set on `deploy/dsh-gateway` (SSO is opt-in). Set `AUTH_PROXY_HEADER=X-Forwarded-User` and confirm the request actually reaches the gateway through oauth2-proxy |
| SSO: gateway treats a caller as another user / bad `X-Forwarded-User` accepted | Trust boundary violation: something other than oauth2-proxy can reach `dsh-gateway:8080`. Remove the second path (Ingress/Gateway/port-forward), keep the Service ClusterIP, and only re-enable SSO once the proxy is the sole entry point. Never put a trusted identity header in front of a service that is directly reachable |
| SSO: oauth2-proxy `cookie_secret must be 16, 24, or 32 bytes ... but is 44 bytes` | The cookie secret was generated as standard base64. Use URL-safe unpadded base64 (see `73-oauth2-proxy-secret.yaml.example`) |
| SSO: redirect loop (`ERR_TOO_MANY_REDIRECTS`) | oauth2-proxy `--redirect-url` / callback does not match the public FQDN, or TLS is not terminating at the Gateway listener; fix the OAuth App URLs and the manifest placeholders |
| SSO: DSH rejects the public host behind the proxy | Add the public hostname to `DSH_TRUSTED_HOSTS` in `manifests/10-sandboxtemplate.yaml`; the gateway forwards the browser's `Host` |
## Limitations ## Limitations
Deliberately **not** production code: Deliberately **not** production code:
- **Single replica.** Idle clocks, in-flight tracking, and the handshake cache are in memory. - **Single replica.** Idle clocks, in-flight tracking, and the handshake cache
Running two gateway replicas would double-suspend or lose wake tracking; scale up only after are in memory. Running two gateway replicas would double-suspend or lose wake
moving that state to a shared store. tracking; scale up only after moving that state to a shared store.
- **Log scraping.** The launch token is recovered from pod logs. Log rotation, a restarted pod, - **Log scraping.** The launch token is recovered from pod logs. Log rotation, a
or a changed DSH log format can break the handshake until the next `401` refresh. restarted pod, or a changed DSH log format can break the handshake until the
- **Shared provider key.** `DEEPSEEK_API_KEY` is shared by all tenants via the template. Real next `401` refresh.
multi-tenancy needs per-tenant credentials or an egress proxy. - **Shared provider key.** `DEEPSEEK_API_KEY` is shared by all tenants via the
- **No TLS in the gateway.** Terminate TLS at an Ingress/reverse proxy and set template. Real multi-tenancy needs per-tenant credentials or an egress proxy.
- **No TLS in the gateway.** Terminate TLS at an Ingress or Gateway and set
`COOKIE_SECURE=true`. `COOKIE_SECURE=true`.
- **In-memory user directory.** Users are claims + annotations; there is no database, password - **SSO trust is hop-based.** `AUTH_PROXY_HEADER` trusts whoever sets the header;
reset, or admin API. Tokens are stored only as SHA-256 hashes in claim annotations. the only guarantee is that the gateway Service is not directly reachable.
- **Bearer tokens in URLs.** Avoid putting the signup token in a URL/log; the browser flow uses There is no signature, mTLS, or NetworkPolicy enforcing that in this example.
the `HttpOnly` cookie instead. - **OAuth org is the only authorization.** Any GitHub user in the configured
organization gets an auto-provisioned sandbox; there is no per-user allow-list,
quota, or admin API.
- **In-memory user directory.** Users are claims + annotations; there is no
database, password reset, or admin API. Tokens are stored only as SHA-256
hashes in claim annotations.
- **Bearer tokens in URLs.** Avoid putting the signup token in a URL/log; the
browser flow uses the `HttpOnly` cookie instead.
## Cost model ## Cost model
The same economics as the upstream pattern: suspend-when-idle plus warm pools turn an The same economics as the upstream pattern: suspend-when-idle plus warm pools turn
always-on instance into a sub-second cold-start pause per return-after-absence, at a fraction an always-on instance into a sub-second cold-start pause per return-after-absence,
of the cost. at a fraction of the cost.
## Cleanup ## Cleanup
```bash ```bash
# Delete an exposure variant first if you applied one (optional; namespace
# deletion below removes everything):
kubectl delete -f manifests/exposure/ingress/ # or gateway-api-sso/
# Then the whole example:
kubectl delete namespace dsh-demo kubectl delete namespace dsh-demo
``` ```
If you created the oauth2-proxy/TLS Secrets or the GitHub OAuth App for step 9,
delete the Secrets with the namespace and revoke the OAuth App in GitHub.
## More
- [Gateway README](gateway/README.md) — API, configuration, auth/SSO, failure
semantics, limitations.
- [Architecture](docs/architecture.md) — container/sequence diagrams, ADRs,
security and RBAC, failure modes, evolution path.
- [Exposure README](manifests/exposure/README.md) — the two variants in detail.

View File

@@ -1,7 +1,7 @@
# Architecture # Architecture
This document describes the browser-usable **agents-as-a-service** platform in This document describes the browser-usable **agents-as-a-service** platform in this example
[`examples/dsh-agents-as-a-service/`](../examples/dsh-agents-as-a-service/): how the gateway, ([README](../README.md), [manifests](../manifests/), [gateway](../gateway/)): how the gateway,
Kubernetes, and DSH Web fit together, and why the design looks the way it does. Kubernetes, and DSH Web fit together, and why the design looks the way it does.
Scope: one Kubernetes namespace (`dsh-demo`), one shared DSH Web image, one gateway deployment, Scope: one Kubernetes namespace (`dsh-demo`), one shared DSH Web image, one gateway deployment,
@@ -49,6 +49,24 @@ Responsibilities:
| **DSH Web pod** | One `dsh web` process per user, state on the per-user PVC, listening on `3080`. | | **DSH Web pod** | One `dsh web` process per user, state on the per-user PVC, listening on `3080`. |
| **Warm pool** | Keeps `N` Ready sandboxes so signup is a label flip instead of a cold start. | | **Warm pool** | Keeps `N` Ready sandboxes so signup is a label flip instead of a cold start. |
## Exposure variants
The base manifests deploy the gateway as a ClusterIP Service (`dsh-gateway:8080`) and expose
nothing outside the cluster. Two **mutually exclusive** variants live under
[`manifests/exposure/`](../manifests/exposure/):
| Variant | Entry point | Authentication | Notes |
| --- | --- | --- | --- |
| [Ingress](../manifests/exposure/ingress/60-ingress.yaml) | Traefik `Ingress` + optional TLS | the gateway's own signup/session cookie | simplest; no external IdP; set `COOKIE_SECURE=true` with HTTPS |
| [Gateway API + SSO](../manifests/exposure/gateway-api-sso/70-gateway.yaml) | Gateway API `Gateway`/`HTTPRoute` terminating TLS | GitHub OAuth via [oauth2-proxy](../manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml), identity forwarded as `X-Forwarded-User` | requires `AUTH_PROXY_HEADER` on the gateway; the trust boundary applies |
They are alternatives, not a stack: applying both creates a second path to the same gateway
and defeats SSO. When SSO is enabled, the Service must stay ClusterIP and be reachable only
through oauth2-proxy — otherwise any client can send `X-Forwarded-User` itself and be treated
as that user. The gateway ignores the header unless `AUTH_PROXY_HEADER` is set. See the
[exposure README](../manifests/exposure/README.md) and the
[gateway README](../gateway/README.md#sso-opt-in).
## Flows ## Flows
### Signup (browser) ### Signup (browser)
@@ -247,7 +265,7 @@ sequenceDiagram
## Security and RBAC ## Security and RBAC
The gateway runs with a namespace-scoped ServiceAccount (`50-gateway.yaml`): The gateway runs with a namespace-scoped ServiceAccount (`manifests/50-gateway.yaml`):
| API group | Resource | Verbs | Why | | API group | Resource | Verbs | Why |
| --- | --- | --- | --- | | --- | --- | --- | --- |

View File

@@ -0,0 +1,11 @@
# Runtime image only needs the app and its pinned runtime requirements.
__pycache__/
*.py[cod]
.pytest_cache/
.ruff_cache/
.venv/
test_gateway.py
requirements-dev.txt
pyproject.toml
README.md
coverage.xml

View File

@@ -4,7 +4,7 @@ A single-file, browser-usable control + data plane for
[DeepSeek Harness Web (DSH)](https://github.com/deepseek-ai/deepseek-harness) [DeepSeek Harness Web (DSH)](https://github.com/deepseek-ai/deepseek-harness)
sandboxes running behind the `agent-sandbox` CRDs. sandboxes running behind the `agent-sandbox` CRDs.
It does three things: It does four things:
1. **Users** — creates a `SandboxClaim` from a warm pool, mints a bearer token 1. **Users** — creates a `SandboxClaim` from a warm pool, mints a bearer token
(only its SHA-256 is stored as a claim annotation), reports derived state, (only its SHA-256 is stored as a claim annotation), reports derived state,
@@ -15,10 +15,15 @@ It does three things:
3. **Reverse proxy** — proxies HTTP (including SSE) and WebSocket traffic to 3. **Reverse proxy** — proxies HTTP (including SSE) and WebSocket traffic to
the sandbox's DSH Web server on `:3080`, including DSH's own cookie the sandbox's DSH Web server on `:3080`, including DSH's own cookie
handshake. handshake.
4. **Optional SSO** — when `AUTH_PROXY_HEADER` is set, trusts the identity
header an authenticating reverse proxy (for example oauth2-proxy) forwards,
auto-provisions that user, and exposes `POST /recreate` for a clean restart.
DSH is pinned to `dsh-v0.2.1-rc.2` in the example manifests. It binds `:3080`, The DSH version pin lives in [`versions.env`](../../../versions.env)
serves `/`, gates `/api` behind a `dsh-auth-*` cookie and requires a WebSocket (`DSH_VERSION` plus the optional repository `RELEASE_REVISION`); the example
at `/api/remote.mux`. manifests and published images use the release identity built from it (currently
`dsh-v0.2.1-rc.2-r2`). DSH binds `:3080`, serves `/`, gates `/api` behind a
`dsh-auth-*` cookie and requires a WebSocket at `/api/remote.mux`.
## Flow ## Flow
@@ -58,10 +63,11 @@ token. A failed handshake returns `503` with `Retry-After: 10`.
| Method | Path | Auth | Purpose | | Method | Path | Auth | Purpose |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `GET` | `/healthz` | none | Liveness/readiness. Never touches the cluster. | | `GET` | `/healthz` | none | Liveness/readiness. Never touches the cluster. |
| `GET` | `/` | none | Landing page with signup + token login forms. | | `GET` | `/` | none | Landing page with signup + token login forms. With SSO it auto-provisions the proxy-identified user and redirects to their workspace. |
| `POST` | `/users` | none | Signup. JSON body `{"user": "<dns-1123 label>"}` or form field `user`. JSON returns `201` when Ready, `202` when accepted but still Waking; form returns `303` to `/u/<user>/`. Both set the session cookie. `409` if the user exists, `400` if invalid, `413` if the body is too large. | | `POST` | `/users` | none | Signup. JSON body `{"user": "<dns-1123 label>"}` or form field `user`. JSON returns `201` when Ready, `202` when accepted but still Waking; form returns `303` to `/u/<user>/`. Both set the session cookie. `409` if the user exists, `400` if invalid, `413` if the body is too large. |
| `POST` | `/login` | none | Form (`user`, `token`) returns `303` to `/u/<user>/`; JSON returns `200`. Sets the session cookie. `401` on a bad token, `404` for an unknown user. | | `POST` | `/login` | none | Form (`user`, `token`) returns `303` to `/u/<user>/`; JSON returns `200`. Sets the session cookie. `401` on a bad token, `404` for an unknown user. |
| `POST` | `/logout` | none | Clears the session cookie and `303`s to `/`. | | `POST` | `/logout` | none | Clears the session cookie and `303`s to `/`. |
| `POST` | `/recreate` | trusted proxy header | **SSO only.** Deletes the authenticated user's claim (cascading sandbox + PVC, so workspace data is wiped), waits out finalizers, provisions a fresh claim and `303`s to `/u/<user>/`. `401` without a valid identity header; cookie-authenticated calls must be same-origin. |
| `GET` | `/users/{user}` | bearer or cookie | Derived state: `Provisioning`, `Waking`, `Ready`, `Suspending`, `Suspended`. | | `GET` | `/users/{user}` | bearer or cookie | Derived state: `Provisioning`, `Waking`, `Ready`, `Suspending`, `Suspended`. |
| `DELETE` | `/users/{user}` | bearer or cookie | Delete the claim (sandbox + PVC cascade). Cookie-auth mutations require a same-origin `Origin`/`Referer`. | | `DELETE` | `/users/{user}` | bearer or cookie | Delete the claim (sandbox + PVC cascade). Cookie-auth mutations require a same-origin `Origin`/`Referer`. |
| `GET` | `/u/{user}` | — | `308` redirect to `/u/{user}/`. | | `GET` | `/u/{user}` | — | `308` redirect to `/u/{user}/`. |
@@ -86,6 +92,37 @@ token. A failed handshake returns `503` with `Retry-After: 10`.
absolute paths are rewritten under `/u/<user>/`, and any `token` query absolute paths are rewritten under `/u/<user>/`, and any `token` query
parameter is stripped. parameter is stripped.
### SSO (opt-in)
By default the gateway is the only authenticator: users sign up with a token or
the session cookie, and the gateway ignores all identity headers. Set
`AUTH_PROXY_HEADER` to enable SSO behind an authenticating reverse proxy such as
oauth2-proxy:
```bash
kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER=X-Forwarded-User
```
When set, the gateway treats a request carrying that header as authenticated as
that user, provided the value is a valid DNS-1123 label; the header is never
trusted when the variable is unset. The landing page then auto-provisions the
user (creates their claim idempotently, mints a rotated session token) and
redirects to `/u/<user>/`, and `POST /recreate` becomes available for a clean
restart. The base manifests leave it unset on purpose; the
[SSO exposure variant](../manifests/exposure/gateway-api-sso/) documents how to
enable it on the Deployment (`kubectl set env`).
> **Trust boundary — this is the security-critical part.** A trusted identity
> header is only safe when **nothing else can reach the gateway**. The gateway
> Service must stay ClusterIP and be reachable exclusively through the
> authenticating proxy; if a client can reach `dsh-gateway:8080` directly (a
> second Ingress/Gateway/Route, a port-forward on a shared host, a broad
> NetworkPolicy), it can send `X-Forwarded-User` itself and impersonate any user.
> Treat `AUTH_PROXY_HEADER` as "the proxy already authenticated this request",
> not as an authentication mechanism of its own. Disable SSO by removing the
> variable: the header is then ignored and bearer-token/session auth applies
> again.
### Proxy failure semantics ### Proxy failure semantics
| Situation | Result | | Situation | Result |
@@ -123,6 +160,7 @@ can also be injected into `create_app` for tests.
| `MAX_BODY_MB` | `32` | Maximum buffered request body (and `413` threshold). | | `MAX_BODY_MB` | `32` | Maximum buffered request body (and `413` threshold). |
| `HTTP_TIMEOUT` | `300` | Upstream connect/read timeout in seconds. Proxied streams (SSE, long responses) keep the connect/write budget but have **no read timeout**, so an idle event stream is not cut every five minutes. | | `HTTP_TIMEOUT` | `300` | Upstream connect/read timeout in seconds. Proxied streams (SSE, long responses) keep the connect/write budget but have **no read timeout**, so an idle event stream is not cut every five minutes. |
| `SWEEP_INTERVAL` | `15` | Seconds between idle sweeps. | | `SWEEP_INTERVAL` | `15` | Seconds between idle sweeps. |
| `AUTH_PROXY_HEADER` | unset | Opt-in SSO: name of the trusted identity header set by an authenticating reverse proxy (e.g. `X-Forwarded-User`). Unset disables SSO and the header is never trusted. Enable only when the Service is reachable exclusively through that proxy (see [SSO](#sso-opt-in)). |
### Required RBAC ### Required RBAC
@@ -147,12 +185,21 @@ python -m ruff check .
uvicorn gateway:create_app --factory --host 0.0.0.0 --port 8080 uvicorn gateway:create_app --factory --host 0.0.0.0 --port 8080
``` ```
Container image: Container image, from the repository root:
```bash ```bash
docker build -t dsh-gateway examples/dsh-agents-as-a-service/gateway make gateway # builds dsh-aaas-gateway:dev from this directory
``` ```
The equivalent direct build is
`docker build -t dsh-aaas-gateway:dev examples/dsh-agents-as-a-service/gateway`.
For kind, retag it to the image reference in
`examples/dsh-agents-as-a-service/manifests/50-gateway.yaml` and `kind load` it.
Published images are nested below the main image as
`<GHCR_IMAGE>/dsh-agents-as-a-service/gateway:<identity>`, where the identity
comes from [`versions.env`](../../../versions.env) — see
[`docs/versioning.md`](../../../docs/versioning.md).
The image runs as UID `10000`, sets `PYTHONDONTWRITEBYTECODE=1` and The image runs as UID `10000`, sets `PYTHONDONTWRITEBYTECODE=1` and
`HOME=/tmp` (read-only-rootfs compatible), listens on `8080`, and disables `HOME=/tmp` (read-only-rootfs compatible), listens on `8080`, and disables
uvicorn's access log so bearer tokens in URLs are not logged. uvicorn's access log so bearer tokens in URLs are not logged.
@@ -182,3 +229,8 @@ uvicorn's access log so bearer tokens in URLs are not logged.
re-runs the handshake once and replays the request. re-runs the handshake once and replays the request.
* **Only DSH on `:3080` is proxied.** There is no dashboard/API-port split and * **Only DSH on `:3080` is proxied.** There is no dashboard/API-port split and
no platform API key injection. no platform API key injection.
* **SSO trusts a header.** With `AUTH_PROXY_HEADER` set, any client that can
reach the gateway Service directly can spoof the identity header. The gateway
cannot enforce the trust boundary itself: keep the Service ClusterIP and
reachable only through the authenticating proxy. There is no mTLS or signed
identity assertion.

View File

@@ -195,9 +195,7 @@ def derive_state(sandbox: dict | None) -> str:
def token_matches(claim: dict, token: str | None) -> bool: def token_matches(claim: dict, token: str | None) -> bool:
"""Timing-safe check of a user token against the stored SHA-256.""" """Timing-safe check of a user token against the stored SHA-256."""
want = (claim.get("metadata", {}).get("annotations") or {}).get( want = (claim.get("metadata", {}).get("annotations") or {}).get(TOKEN_ANNOTATION, "")
TOKEN_ANNOTATION, ""
)
if not token or not want: if not token or not want:
return False return False
got = hashlib.sha256(token.encode()).hexdigest() got = hashlib.sha256(token.encode()).hexdigest()
@@ -518,9 +516,7 @@ class Kubernetes:
async def get_pod(self, name: str) -> dict | None: async def get_pod(self, name: str) -> dict | None:
try: try:
pod = await self._call( pod = await self._call("core", "read_namespaced_pod", name, self.namespace)
"core", "read_namespaced_pod", name, self.namespace
)
except ApiException as exc: except ApiException as exc:
if exc.status == 404: if exc.status == 404:
return None return None
@@ -530,9 +526,7 @@ class Kubernetes:
# so callers can keep using ``pod.get(...)``. # so callers can keep using ``pod.get(...)``.
return k8s_client.ApiClient().sanitize_for_serialization(pod) return k8s_client.ApiClient().sanitize_for_serialization(pod)
async def read_pod_logs( async def read_pod_logs(self, name: str, container: str, tail_lines: int) -> str:
self, name: str, container: str, tail_lines: int
) -> str:
result = await self._call( result = await self._call(
"core", "core",
"read_namespaced_pod_log", "read_namespaced_pod_log",
@@ -616,9 +610,7 @@ class Gateway:
return False # a session for another user never authorizes this one return False # a session for another user never authorizes this one
return token_matches(claim, token) return token_matches(claim, token)
async def _sso_session( async def _sso_session(self, remote: str, headers, cookies, response: Response) -> dict | None:
self, remote: str, headers, cookies, response: Response
) -> dict | None:
"""Provision ``remote`` idempotently and mint/refresh a session token. """Provision ``remote`` idempotently and mint/refresh a session token.
The authenticating proxy has already verified ``remote``, so it is The authenticating proxy has already verified ``remote``, so it is
@@ -682,9 +674,7 @@ class Gateway:
while True: while True:
async with self.state.lock: async with self.state.lock:
if not self.state.maintenance.get(user): if not self.state.maintenance.get(user):
self.state.in_flight[user] = ( self.state.in_flight[user] = self.state.in_flight.get(user, 0) + 1
self.state.in_flight.get(user, 0) + 1
)
self.state.last_activity[user] = time.time() self.state.last_activity[user] = time.time()
return True return True
if time.monotonic() > deadline: if time.monotonic() > deadline:
@@ -693,9 +683,7 @@ class Gateway:
async def _release(self, user: str) -> None: async def _release(self, user: str) -> None:
async with self.state.lock: async with self.state.lock:
self.state.in_flight[user] = max( self.state.in_flight[user] = max(self.state.in_flight.get(user, 1) - 1, 0)
self.state.in_flight.get(user, 1) - 1, 0
)
self.state.last_activity[user] = time.time() self.state.last_activity[user] = time.time()
# -- sandbox / handshake helpers -------------------------------------- # -- sandbox / handshake helpers --------------------------------------
@@ -724,9 +712,7 @@ class Gateway:
# Wake-on-connect: safe in ANY state, including mid-suspension. # Wake-on-connect: safe in ANY state, including mid-suspension.
self._clear_handshake(user) self._clear_handshake(user)
try: try:
await self.k8s.patch_operating_mode( await self.k8s.patch_operating_mode(sandbox["metadata"]["name"], "Running")
sandbox["metadata"]["name"], "Running"
)
except Exception as exc: except Exception as exc:
raise SandboxUnavailable("could not wake the sandbox") from exc raise SandboxUnavailable("could not wake the sandbox") from exc
sandbox = await self.wait_ready(user, self.settings.wake_timeout) sandbox = await self.wait_ready(user, self.settings.wake_timeout)
@@ -740,10 +726,7 @@ class Gateway:
def _pod_name(self, sandbox: dict) -> str: def _pod_name(self, sandbox: dict) -> str:
annotations = sandbox.get("metadata", {}).get("annotations") or {} annotations = sandbox.get("metadata", {}).get("annotations") or {}
return ( return annotations.get(POD_NAME_ANNOTATION) or sandbox["metadata"]["name"]
annotations.get(POD_NAME_ANNOTATION)
or sandbox["metadata"]["name"]
)
@staticmethod @staticmethod
def _pod_ip(sandbox: dict) -> str | None: def _pod_ip(sandbox: dict) -> str | None:
@@ -787,16 +770,11 @@ class Gateway:
self.settings.dsh_log_tail_lines, self.settings.dsh_log_tail_lines,
) )
except ApiException as exc: except ApiException as exc:
raise HandshakeError( raise HandshakeError(f"could not read logs for {pod_name}: {exc}") from exc
f"could not read logs for {pod_name}: {exc}"
) from exc
launch_token = parse_dsh_launch_token(logs) launch_token = parse_dsh_launch_token(logs)
if not launch_token: if not launch_token:
raise HandshakeError("DSH launch token not found in pod logs") raise HandshakeError("DSH launch token not found in pod logs")
url = ( url = f"http://{fqdn}:{self.settings.dsh_port}/?token={quote(launch_token, safe='')}"
f"http://{fqdn}:{self.settings.dsh_port}/"
f"?token={quote(launch_token, safe='')}"
)
headers = { headers = {
"Host": host, "Host": host,
"X-Forwarded-Host": host, "X-Forwarded-Host": host,
@@ -812,9 +790,7 @@ class Gateway:
except httpx.HTTPError as exc: except httpx.HTTPError as exc:
raise HandshakeError(f"DSH handshake failed: {exc}") from exc raise HandshakeError(f"DSH handshake failed: {exc}") from exc
if response.status_code >= 400: if response.status_code >= 400:
raise HandshakeError( raise HandshakeError(f"DSH handshake returned {response.status_code}")
f"DSH handshake returned {response.status_code}"
)
cookie = extract_dsh_cookies(response.headers.get_list("set-cookie")) cookie = extract_dsh_cookies(response.headers.get_list("set-cookie"))
if not cookie: if not cookie:
raise HandshakeError("DSH handshake set no dsh-auth cookie") raise HandshakeError("DSH handshake set no dsh-auth cookie")
@@ -856,7 +832,7 @@ class Gateway:
else: else:
status = html.escape(state) status = html.escape(state)
confirm = ( confirm = (
'onsubmit="return confirm(\'Delete this sandbox and start fresh? ' "onsubmit=\"return confirm('Delete this sandbox and start fresh? "
"Your workspace data will be wiped.')\"" "Your workspace data will be wiped.')\""
) )
body = ( body = (
@@ -887,9 +863,7 @@ class Gateway:
# session cookie minted by _sso_session; the real page is rendered # session cookie minted by _sso_session; the real page is rendered
# once the sandbox state is known. # once the sandbox state is known.
response = HTMLResponse() response = HTMLResponse()
claim = await self._sso_session( claim = await self._sso_session(remote, request.headers, request.cookies, response)
remote, request.headers, request.cookies, response
)
sandbox = await self._sandbox_of(claim) if claim else None sandbox = await self._sandbox_of(claim) if claim else None
page = HTMLResponse(self._sso_page(remote, derive_state(sandbox))) page = HTMLResponse(self._sso_page(remote, derive_state(sandbox)))
if "set-cookie" in response.headers: if "set-cookie" in response.headers:
@@ -953,14 +927,10 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
@staticmethod @staticmethod
def _wants_form(request: Request) -> bool: def _wants_form(request: Request) -> bool:
ctype = request.headers.get("content-type", "").lower() ctype = request.headers.get("content-type", "").lower()
return ctype.startswith( return ctype.startswith(("application/x-www-form-urlencoded", "multipart/form-data"))
("application/x-www-form-urlencoded", "multipart/form-data")
)
async def _read_payload(self, request: Request) -> dict: async def _read_payload(self, request: Request) -> dict:
if request.headers.get("content-type", "").lower().startswith( if request.headers.get("content-type", "").lower().startswith("application/json"):
"application/json"
):
data = await self._read_body(request) data = await self._read_body(request)
try: try:
payload = json.loads(data) payload = json.loads(data)
@@ -997,9 +967,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
return self._error(413, "request body too large") return self._error(413, "request body too large")
user = payload.get("user", "") user = payload.get("user", "")
if not valid_user(user): if not valid_user(user):
return self._error( return self._error(400, "body must be {'user': '<dns-1123 label>'}")
400, "body must be {'user': '<dns-1123 label>'}"
)
if await self.k8s.get_claim(user) is not None: if await self.k8s.get_claim(user) is not None:
return self._error(409, "user exists") return self._error(409, "user exists")
token = secrets.token_urlsafe(32) token = secrets.token_urlsafe(32)
@@ -1059,9 +1027,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
if claim is None: if claim is None:
return self._error(404, "not found") return self._error(404, "not found")
token, cookie_user = self.credentials(request.headers, request.cookies) token, cookie_user = self.credentials(request.headers, request.cookies)
if not self._authorized( if not self._authorized(claim, token, cookie_user, user, self.remote_user(request.headers)):
claim, token, cookie_user, user, self.remote_user(request.headers)
):
return self._error(401, "unauthorized") return self._error(401, "unauthorized")
sandbox = await self._sandbox_of(claim) sandbox = await self._sandbox_of(claim)
return JSONResponse( return JSONResponse(
@@ -1080,9 +1046,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
if claim is None: if claim is None:
return self._error(404, "not found") return self._error(404, "not found")
token, cookie_user = self.credentials(request.headers, request.cookies) token, cookie_user = self.credentials(request.headers, request.cookies)
if not self._authorized( if not self._authorized(claim, token, cookie_user, user, self.remote_user(request.headers)):
claim, token, cookie_user, user, self.remote_user(request.headers)
):
return self._error(401, "unauthorized") return self._error(401, "unauthorized")
if self._mutation_from_cookie(request.headers, "DELETE", cookie_user): if self._mutation_from_cookie(request.headers, "DELETE", cookie_user):
return self._error(403, "cross-origin request rejected") return self._error(403, "cross-origin request rejected")
@@ -1091,9 +1055,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
async with self.state.lock: async with self.state.lock:
self.state.last_activity.pop(user, None) self.state.last_activity.pop(user, None)
self.state.in_flight.pop(user, None) self.state.in_flight.pop(user, None)
return JSONResponse( return JSONResponse({"user": user, "note": "claim deleted; sandbox and PVC cascade"})
{"user": user, "note": "claim deleted; sandbox and PVC cascade"}
)
async def recreate(self, request: Request) -> Response: async def recreate(self, request: Request) -> Response:
"""Delete and re-provision the SSO user's sandbox for a clean start. """Delete and re-provision the SSO user's sandbox for a clean start.
@@ -1122,9 +1084,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
break break
await asyncio.sleep(1) await asyncio.sleep(1)
response = RedirectResponse(f"/u/{remote}/", status_code=303) response = RedirectResponse(f"/u/{remote}/", status_code=303)
await self._sso_session( await self._sso_session(remote, request.headers, request.cookies, response)
remote, request.headers, request.cookies, response
)
return response return response
# -- HTTP proxy -------------------------------------------------------- # -- HTTP proxy --------------------------------------------------------
@@ -1141,9 +1101,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
return RedirectResponse("/", status_code=303) return RedirectResponse("/", status_code=303)
return self._error(404, "unknown user") return self._error(404, "unknown user")
token, cookie_user = self.credentials(request.headers, request.cookies) token, cookie_user = self.credentials(request.headers, request.cookies)
if remote == user and not ( if remote == user and not (cookie_user == user and token_matches(claim, token)):
cookie_user == user and token_matches(claim, token)
):
# SSO user without a valid session cookie: bootstrap, then continue. # SSO user without a valid session cookie: bootstrap, then continue.
return RedirectResponse("/", status_code=303) return RedirectResponse("/", status_code=303)
if not self._authorized(claim, token, cookie_user, user, remote): if not self._authorized(claim, token, cookie_user, user, remote):
@@ -1152,31 +1110,21 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
return self._error(403, "cross-origin request rejected") return self._error(403, "cross-origin request rejected")
if not await self._acquire(user): if not await self._acquire(user):
return self._error( return self._error(503, "gateway busy, retry shortly", {"Retry-After": "5"})
503, "gateway busy, retry shortly", {"Retry-After": "5"}
)
streaming = False streaming = False
try: try:
sandbox, fqdn = await self._resolve_sandbox(user, claim) sandbox, fqdn = await self._resolve_sandbox(user, claim)
host = request.headers.get("host", "") host = request.headers.get("host", "")
cookie, _, was_cached = await self.dsh_cookie( cookie, _, was_cached = await self.dsh_cookie(user, sandbox, fqdn, host)
user, sandbox, fqdn, host
)
body = await self._read_body(request) body = await self._read_body(request)
upstream = await self._send_upstream( upstream = await self._send_upstream(request, user, fqdn, cookie, body)
request, user, fqdn, cookie, body
)
if upstream.status_code == 401 and was_cached: if upstream.status_code == 401 and was_cached:
# The cached DSH cookie expired (pod restart, logout, ...): # The cached DSH cookie expired (pod restart, logout, ...):
# handshake once more and replay the (already buffered) request. # handshake once more and replay the (already buffered) request.
await upstream.aclose() await upstream.aclose()
cookie, _, _ = await self.dsh_cookie( cookie, _, _ = await self.dsh_cookie(user, sandbox, fqdn, host, force=True)
user, sandbox, fqdn, host, force=True upstream = await self._send_upstream(request, user, fqdn, cookie, body)
)
upstream = await self._send_upstream(
request, user, fqdn, cookie, body
)
response = StreamingResponse( response = StreamingResponse(
self._relay(user, upstream), self._relay(user, upstream),
@@ -1216,8 +1164,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
headers = { headers = {
key: value key: value
for key, value in request.headers.items() for key, value in request.headers.items()
if key.lower() not in OWNED_REQUEST_HEADERS if key.lower() not in OWNED_REQUEST_HEADERS and key.lower() != "content-length"
and key.lower() != "content-length"
} }
host = request.headers.get("host", "") host = request.headers.get("host", "")
headers["host"] = host headers["host"] = host
@@ -1229,9 +1176,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
else: else:
headers["x-forwarded-for"] = "" headers["x-forwarded-for"] = ""
port = host.rsplit(":", 1)[1] if ":" in host else "" port = host.rsplit(":", 1)[1] if ":" in host else ""
headers["x-forwarded-port"] = port or ( headers["x-forwarded-port"] = port or ("443" if request.url.scheme == "https" else "80")
"443" if request.url.scheme == "https" else "80"
)
last_error: Exception | None = None last_error: Exception | None = None
for attempt in range(self.settings.upstream_attempts): for attempt in range(self.settings.upstream_attempts):
@@ -1341,9 +1286,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
if ":" in upstream_host and not upstream_host.startswith("["): if ":" in upstream_host and not upstream_host.startswith("["):
upstream_host = f"[{upstream_host}]" # IPv6 literal upstream_host = f"[{upstream_host}]" # IPv6 literal
authority = f"{upstream_host}:{self.settings.dsh_port}" authority = f"{upstream_host}:{self.settings.dsh_port}"
cookie, _, _ = await self.dsh_cookie( cookie, _, _ = await self.dsh_cookie(user, sandbox, fqdn, authority)
user, sandbox, fqdn, authority
)
url = f"ws://{authority}/{path}" url = f"ws://{authority}/{path}"
query = filtered_query(websocket.url.query) query = filtered_query(websocket.url.query)
if query: if query:
@@ -1362,9 +1305,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
headers["x-forwarded-for"] = websocket.client.host headers["x-forwarded-for"] = websocket.client.host
subprotocols = [ subprotocols = [
part.strip() part.strip()
for part in websocket.headers.get( for part in websocket.headers.get("sec-websocket-protocol", "").split(",")
"sec-websocket-protocol", ""
).split(",")
if part.strip() if part.strip()
] ]
upstream_cm = self.ws_connect( upstream_cm = self.ws_connect(
@@ -1380,16 +1321,12 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
return await self._ws_reject(websocket, 1011) return await self._ws_reject(websocket, 1011)
try: try:
await websocket.accept( await websocket.accept(subprotocol=getattr(upstream, "subprotocol", None))
subprotocol=getattr(upstream, "subprotocol", None)
)
except RuntimeError: except RuntimeError:
pass pass
await self._pump(websocket, upstream) await self._pump(websocket, upstream)
with contextlib.suppress(Exception): with contextlib.suppress(Exception):
await websocket.close( await websocket.close(code=self._upstream_close_code(upstream))
code=self._upstream_close_code(upstream)
)
finally: finally:
if upstream_cm is not None: if upstream_cm is not None:
with contextlib.suppress(Exception): with contextlib.suppress(Exception):
@@ -1429,9 +1366,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
asyncio.create_task(client_to_upstream()), asyncio.create_task(client_to_upstream()),
asyncio.create_task(upstream_to_client()), asyncio.create_task(upstream_to_client()),
] ]
done, pending = await asyncio.wait( done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
tasks, return_when=asyncio.FIRST_COMPLETED
)
for task in pending: for task in pending:
task.cancel() task.cancel()
await asyncio.gather(*done, *pending, return_exceptions=True) await asyncio.gather(*done, *pending, return_exceptions=True)
@@ -1458,17 +1393,13 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
continue continue
if self.state.in_flight.get(user, 0) > 0: if self.state.in_flight.get(user, 0) > 0:
continue # requests in flight => the user is active continue # requests in flight => the user is active
idle = time.time() - self.state.last_activity.setdefault( idle = time.time() - self.state.last_activity.setdefault(user, time.time())
user, time.time()
)
if idle < self.settings.idle_timeout: if idle < self.settings.idle_timeout:
continue continue
self.state.maintenance[user] = True self.state.maintenance[user] = True
try: try:
print(f"suspending {user} (idle {int(idle)}s)", flush=True) print(f"suspending {user} (idle {int(idle)}s)", flush=True)
await self.k8s.patch_operating_mode( await self.k8s.patch_operating_mode(sandbox["metadata"]["name"], "Suspended")
sandbox["metadata"]["name"], "Suspended"
)
self._clear_handshake(user) self._clear_handshake(user)
finally: finally:
async with self.state.lock: async with self.state.lock:
@@ -1519,12 +1450,8 @@ def create_app(
k8s = k8s if k8s is not None else Kubernetes(settings.namespace) k8s = k8s if k8s is not None else Kubernetes(settings.namespace)
owns_http_client = http_client is None owns_http_client = http_client is None
if http_client is None: if http_client is None:
http_client = httpx.AsyncClient( http_client = httpx.AsyncClient(follow_redirects=False, timeout=settings.http_timeout)
follow_redirects=False, timeout=settings.http_timeout gateway = Gateway(settings, k8s, http_client, ws_connect or websockets.connect)
)
gateway = Gateway(
settings, k8s, http_client, ws_connect or websockets.connect
)
routes = [ routes = [
Route("/healthz", gateway.healthz), Route("/healthz", gateway.healthz),
Route("/", gateway.landing), Route("/", gateway.landing),

View File

@@ -1,4 +1,5 @@
-r requirements.txt -r requirements.txt
pytest==9.1.1 pytest==9.1.1
pytest-asyncio==1.4.0 pytest-asyncio==1.4.0
pytest-cov==7.1.0
ruff==0.17.1 ruff==0.17.1

View File

@@ -45,9 +45,7 @@ FQDN = "dsh-alice-sbx.test-ns.svc.cluster.local"
def dsh_response(status=200, content=b"", headers=None): def dsh_response(status=200, content=b"", headers=None):
"""A MockTransport response that is a real stream (``aiter_raw`` works).""" """A MockTransport response that is a real stream (``aiter_raw`` works)."""
return httpx.Response( return httpx.Response(status, headers=headers, stream=httpx.ByteStream(content))
status, headers=headers, stream=httpx.ByteStream(content)
)
# --- fakes ----------------------------------------------------------------- # --- fakes -----------------------------------------------------------------
@@ -99,9 +97,7 @@ class FakeK8s:
self.token_patches.append((user, token_hash)) self.token_patches.append((user, token_hash))
claim = self.claims.get(user) claim = self.claims.get(user)
if claim is not None: if claim is not None:
annotations = claim.setdefault("metadata", {}).setdefault( annotations = claim.setdefault("metadata", {}).setdefault("annotations", {})
"annotations", {}
)
annotations["aaas.example.com/token-sha256"] = token_hash annotations["aaas.example.com/token-sha256"] = token_hash
async def get_pod(self, name): async def get_pod(self, name):
@@ -118,9 +114,7 @@ class FakeUpstream:
self.requests = [] self.requests = []
self.handshake_requests = [] self.handshake_requests = []
self.handshake_status = 303 self.handshake_status = 303
self.handshake_cookies = [ self.handshake_cookies = ["dsh-auth-a=one; Path=/; HttpOnly; SameSite=Strict"]
"dsh-auth-a=one; Path=/; HttpOnly; SameSite=Strict"
]
self.proxy_handler = None self.proxy_handler = None
self.proxy_requests = [] self.proxy_requests = []
@@ -140,9 +134,7 @@ class FakeUpstream:
self.proxy_requests.append(request) self.proxy_requests.append(request)
if self.proxy_handler is not None: if self.proxy_handler is not None:
return self.proxy_handler(request) return self.proxy_handler(request)
return dsh_response( return dsh_response(200, b"hello from DSH", {"content-type": "text/plain"})
200, b"hello from DSH", {"content-type": "text/plain"}
)
class FakeWS: class FakeWS:
@@ -207,9 +199,7 @@ def add_user(k8s, user="alice", token=TOKEN, ready=True, pod_name=None, suffix="
"metadata": { "metadata": {
"name": claim_name(user), "name": claim_name(user),
"annotations": { "annotations": {
"aaas.example.com/token-sha256": hashlib.sha256( "aaas.example.com/token-sha256": hashlib.sha256(token.encode()).hexdigest()
token.encode()
).hexdigest()
}, },
}, },
"status": {"sandbox": {"name": sandbox_name}}, "status": {"sandbox": {"name": sandbox_name}},
@@ -224,10 +214,7 @@ def add_user(k8s, user="alice", token=TOKEN, ready=True, pod_name=None, suffix="
}, },
} }
k8s.pods[pod_name] = {"metadata": {"name": pod_name, "uid": POD_UID}} k8s.pods[pod_name] = {"metadata": {"name": pod_name, "uid": POD_UID}}
k8s.logs[pod_name] = ( k8s.logs[pod_name] = f"booting\ndsh web: http://10.0.0.7:3080/?token={TOKEN}-launch\n"
"booting\n"
f"dsh web: http://10.0.0.7:3080/?token={TOKEN}-launch\n"
)
return sandbox_name, pod_name return sandbox_name, pod_name
@@ -243,9 +230,7 @@ def env():
) )
k8s = FakeK8s() k8s = FakeK8s()
upstream = FakeUpstream() upstream = FakeUpstream()
http_client = httpx.AsyncClient( http_client = httpx.AsyncClient(transport=httpx.MockTransport(upstream), timeout=5)
transport=httpx.MockTransport(upstream), timeout=5
)
ws_connect = FakeWSConnect() ws_connect = FakeWSConnect()
app = create_app( app = create_app(
settings=settings, settings=settings,
@@ -275,9 +260,7 @@ def asgi_client(env, cookies=None):
def stale_handshake(env): def stale_handshake(env):
env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = ( env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = "dsh-auth-a=stale"
"dsh-auth-a=stale"
)
# --- pure helpers ---------------------------------------------------------- # --- pure helpers ----------------------------------------------------------
@@ -419,9 +402,7 @@ def test_token_matches_is_timing_safe_hash_check():
claim = { claim = {
"metadata": { "metadata": {
"annotations": { "annotations": {
"aaas.example.com/token-sha256": hashlib.sha256( "aaas.example.com/token-sha256": hashlib.sha256(TOKEN.encode()).hexdigest()
TOKEN.encode()
).hexdigest()
} }
} }
} }
@@ -486,14 +467,9 @@ async def test_kubernetes_create_claim_body(monkeypatch):
body = args[4] body = args[4]
assert body["metadata"]["name"] == "dsh-alice" assert body["metadata"]["name"] == "dsh-alice"
assert body["metadata"]["annotations"]["aaas.example.com/token-sha256"] == "hash" assert body["metadata"]["annotations"]["aaas.example.com/token-sha256"] == "hash"
assert ( assert body["metadata"]["annotations"]["aaas.example.com/managed-by"] == "dsh-gateway"
body["metadata"]["annotations"]["aaas.example.com/managed-by"]
== "dsh-gateway"
)
assert body["spec"]["warmPoolRef"]["name"] == "test-pool" assert body["spec"]["warmPoolRef"]["name"] == "test-pool"
assert body["spec"]["additionalPodMetadata"]["labels"][ assert body["spec"]["additionalPodMetadata"]["labels"]["sandbox.users.io/dsh-user"] == "alice"
"sandbox.users.io/dsh-user"
] == "alice"
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -625,9 +601,7 @@ async def test_landing_page_has_signup_and_login_forms(env):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_landing_page_shows_session_when_cookie_present(env): async def test_landing_page_shows_session_when_cookie_present(env):
async with asgi_client( async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client:
env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}
) as client:
response = await client.get("/") response = await client.get("/")
assert "Signed in as" in response.text assert "Signed in as" in response.text
assert "/u/alice/" in response.text assert "/u/alice/" in response.text
@@ -645,9 +619,7 @@ async def test_workspace_redirect_adds_trailing_slash(env):
@pytest.mark.asyncio @pytest.mark.asyncio
@pytest.mark.parametrize( @pytest.mark.parametrize("bad_user", ["Alice", "-alice", "alice-", "", "a_b", 123, None, "a" * 60])
"bad_user", ["Alice", "-alice", "alice-", "", "a_b", 123, None, "a" * 60]
)
async def test_signup_rejects_invalid_dns1123_label(env, bad_user): async def test_signup_rejects_invalid_dns1123_label(env, bad_user):
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.post("/users", json={"user": bad_user}) response = await client.post("/users", json={"user": bad_user})
@@ -781,9 +753,7 @@ async def test_login_json_and_failures(env):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_logout_clears_cookie(env): async def test_logout_clears_cookie(env):
async with asgi_client( async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client:
env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}
) as client:
response = await client.post("/logout") response = await client.post("/logout")
assert response.status_code == 303 assert response.status_code == 303
assert response.headers["location"] == "/" assert response.headers["location"] == "/"
@@ -798,12 +768,8 @@ async def test_logout_clears_cookie(env):
async def test_get_user_requires_valid_bearer_or_cookie(env): async def test_get_user_requires_valid_bearer_or_cookie(env):
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
forbidden = await client.get( forbidden = await client.get("/users/alice", headers={"Authorization": "Bearer wrong"})
"/users/alice", headers={"Authorization": "Bearer wrong"} bearer = await client.get("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"})
)
bearer = await client.get(
"/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}
)
cookie = await client.get( cookie = await client.get(
"/users/alice", "/users/alice",
headers={"Cookie": f"dsh_gw_session=alice:{TOKEN}"}, headers={"Cookie": f"dsh_gw_session=alice:{TOKEN}"},
@@ -839,9 +805,7 @@ async def test_invalid_user_in_path_is_404(env, path):
async def test_delete_user_requires_auth(env): async def test_delete_user_requires_auth(env):
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.delete( response = await client.delete("/users/alice", headers={"Authorization": "Bearer wrong"})
"/users/alice", headers={"Authorization": "Bearer wrong"}
)
assert response.status_code == 401 assert response.status_code == 401
assert env.k8s.deleted == [] assert env.k8s.deleted == []
@@ -853,12 +817,8 @@ async def test_delete_with_cookie_requires_same_origin(env):
cookies = {"dsh_gw_session": f"alice:{TOKEN}"} cookies = {"dsh_gw_session": f"alice:{TOKEN}"}
async with asgi_client(env, cookies=cookies) as client: async with asgi_client(env, cookies=cookies) as client:
missing = await client.delete("/users/alice") missing = await client.delete("/users/alice")
cross = await client.delete( cross = await client.delete("/users/alice", headers={"Origin": "http://evil.test"})
"/users/alice", headers={"Origin": "http://evil.test"} good = await client.delete("/users/alice", headers={"Origin": "http://gateway.test"})
)
good = await client.delete(
"/users/alice", headers={"Origin": "http://gateway.test"}
)
assert missing.status_code == 403 assert missing.status_code == 403
assert cross.status_code == 403 assert cross.status_code == 403
assert good.status_code == 200 assert good.status_code == 200
@@ -870,9 +830,7 @@ async def test_delete_with_cookie_requires_same_origin(env):
async def test_delete_with_bearer_does_not_need_origin(env): async def test_delete_with_bearer_does_not_need_origin(env):
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.delete( response = await client.delete("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"})
"/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}
)
assert response.status_code == 200 assert response.status_code == 200
@@ -881,9 +839,7 @@ async def test_delete_missing_claim_is_404(env):
add_user(env.k8s) add_user(env.k8s)
env.k8s.claims.clear() env.k8s.claims.clear()
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.delete( response = await client.delete("/users/alice", headers={"Authorization": f"Bearer {TOKEN}"})
"/users/alice", headers={"Authorization": f"Bearer {TOKEN}"}
)
assert response.status_code == 404 assert response.status_code == 404
@@ -958,9 +914,9 @@ async def test_missing_cookie_in_handshake_is_503(env):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_pod_annotation_overrides_sandbox_name(env): async def test_pod_annotation_overrides_sandbox_name(env):
sandbox_name, pod_name = add_user(env.k8s) sandbox_name, pod_name = add_user(env.k8s)
env.k8s.sandboxes[sandbox_name]["metadata"]["annotations"][ env.k8s.sandboxes[sandbox_name]["metadata"]["annotations"]["agents.x-k8s.io/pod-name"] = (
"agents.x-k8s.io/pod-name" "custom-pod"
] = "custom-pod" )
env.k8s.pods["custom-pod"] = {"metadata": {"uid": "other-uid"}} env.k8s.pods["custom-pod"] = {"metadata": {"uid": "other-uid"}}
env.k8s.logs["custom-pod"] = env.k8s.logs[pod_name] env.k8s.logs["custom-pod"] = env.k8s.logs[pod_name]
async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client: async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client:
@@ -1082,9 +1038,7 @@ async def test_proxy_upstream_connection_error_is_502(env):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_proxy_body_limit_returns_413(env): async def test_proxy_body_limit_returns_413(env):
add_user(env.k8s) add_user(env.k8s)
env.gateway.settings = Settings( env.gateway.settings = Settings(**{**env.settings.__dict__, "max_body_bytes": 4})
**{**env.settings.__dict__, "max_body_bytes": 4}
)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.post( response = await client.post(
"/u/alice/api/echo", "/u/alice/api/echo",
@@ -1194,9 +1148,7 @@ async def test_proxied_requests_use_the_stream_timeout(env, monkeypatch):
"server": ("gateway.test", 80), "server": ("gateway.test", 80),
"path_params": {"user": "alice", "path": "events"}, "path_params": {"user": "alice", "path": "events"},
} }
response = await env.gateway._send_upstream( response = await env.gateway._send_upstream(Request(scope), "alice", FQDN, "dsh-auth-a=1", b"")
Request(scope), "alice", FQDN, "dsh-auth-a=1", b""
)
await response.aclose() await response.aclose()
assert captured["timeout"].read is None assert captured["timeout"].read is None
@@ -1359,9 +1311,7 @@ async def test_sweeper_skips_claim_it_does_not_own(env):
assert "alice" not in env.gateway.state.last_activity assert "alice" not in env.gateway.state.last_activity
# ...and does not suspend it even when it looks long-idle. # ...and does not suspend it even when it looks long-idle.
env.gateway.state.last_activity["alice"] = ( env.gateway.state.last_activity["alice"] = time.time() - env.settings.idle_timeout - 10
time.time() - env.settings.idle_timeout - 10
)
await env.gateway.sweep_once() await env.gateway.sweep_once()
assert env.k8s.patches == [] assert env.k8s.patches == []
@@ -1370,9 +1320,7 @@ async def test_sweeper_skips_claim_it_does_not_own(env):
async def test_sweeper_suspends_idle_ready_sandbox(env): async def test_sweeper_suspends_idle_ready_sandbox(env):
env.k8s.claims["alice"] = ready_claim() env.k8s.claims["alice"] = ready_claim()
env.k8s.sandboxes["dsh-alice-sbx"] = ready_sandbox() env.k8s.sandboxes["dsh-alice-sbx"] = ready_sandbox()
env.gateway.state.last_activity["alice"] = ( env.gateway.state.last_activity["alice"] = time.time() - env.settings.idle_timeout - 10
time.time() - env.settings.idle_timeout - 10
)
env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = "dsh-auth-a=x" env.gateway.state.handshake[("alice", POD_UID, "gateway.test")] = "dsh-auth-a=x"
await env.gateway.sweep_once() await env.gateway.sweep_once()
@@ -1478,9 +1426,7 @@ async def test_sso_landing_new_user_shows_provisioning(env):
async def test_sso_landing_ready_connects_and_offers_recreate_only(env): async def test_sso_landing_ready_connects_and_offers_recreate_only(env):
enable_sso(env) enable_sso(env)
add_user(env.k8s) add_user(env.k8s)
async with asgi_client( async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client:
env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}
) as client:
response = await client.get("/", headers={"X-Forwarded-User": "alice"}) response = await client.get("/", headers={"X-Forwarded-User": "alice"})
assert response.status_code == 200 assert response.status_code == 200
assert "Signed in as <strong>alice</strong>" in response.text assert "Signed in as <strong>alice</strong>" in response.text
@@ -1512,9 +1458,7 @@ async def test_recreate_deletes_and_reprovisions(env):
enable_sso(env) enable_sso(env)
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.post( response = await client.post("/recreate", headers={"X-Forwarded-User": "alice"})
"/recreate", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 303 assert response.status_code == 303
assert response.headers["location"] == "/u/alice/" assert response.headers["location"] == "/u/alice/"
assert env.k8s.deleted == ["alice"] assert env.k8s.deleted == ["alice"]
@@ -1526,9 +1470,7 @@ async def test_recreate_deletes_and_reprovisions(env):
async def test_recreate_requires_sso(env): async def test_recreate_requires_sso(env):
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.post( response = await client.post("/recreate", headers={"X-Forwarded-User": "alice"})
"/recreate", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 401 assert response.status_code == 401
assert env.k8s.deleted == [] assert env.k8s.deleted == []
@@ -1537,9 +1479,7 @@ async def test_recreate_requires_sso(env):
async def test_recreate_rejects_cross_origin(env): async def test_recreate_rejects_cross_origin(env):
enable_sso(env) enable_sso(env)
add_user(env.k8s) add_user(env.k8s)
async with asgi_client( async with asgi_client(env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}) as client:
env, cookies={"dsh_gw_session": f"alice:{TOKEN}"}
) as client:
response = await client.post( response = await client.post(
"/recreate", "/recreate",
headers={"X-Forwarded-User": "alice", "Origin": "http://evil.test"}, headers={"X-Forwarded-User": "alice", "Origin": "http://evil.test"},
@@ -1553,9 +1493,7 @@ async def test_sso_get_user_authorizes_via_header(env):
enable_sso(env) enable_sso(env)
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.get( response = await client.get("/users/alice", headers={"X-Forwarded-User": "alice"})
"/users/alice", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 200 assert response.status_code == 200
assert response.json()["user"] == "alice" assert response.json()["user"] == "alice"
@@ -1565,9 +1503,7 @@ async def test_sso_proxy_redirects_to_bootstrap_without_session(env):
enable_sso(env) enable_sso(env)
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.get( response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"})
"/u/alice/", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 303 assert response.status_code == 303
assert response.headers["location"] == "/" assert response.headers["location"] == "/"
@@ -1576,9 +1512,7 @@ async def test_sso_proxy_redirects_to_bootstrap_without_session(env):
async def test_sso_proxy_redirects_to_bootstrap_when_claim_missing(env): async def test_sso_proxy_redirects_to_bootstrap_when_claim_missing(env):
enable_sso(env) enable_sso(env)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.get( response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"})
"/u/alice/", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 303 assert response.status_code == 303
assert response.headers["location"] == "/" assert response.headers["location"] == "/"
@@ -1587,7 +1521,5 @@ async def test_sso_proxy_redirects_to_bootstrap_when_claim_missing(env):
async def test_spoofed_header_ignored_when_sso_disabled(env): async def test_spoofed_header_ignored_when_sso_disabled(env):
add_user(env.k8s) add_user(env.k8s)
async with asgi_client(env) as client: async with asgi_client(env) as client:
response = await client.get( response = await client.get("/u/alice/", headers={"X-Forwarded-User": "alice"})
"/u/alice/", headers={"X-Forwarded-User": "alice"}
)
assert response.status_code == 401 assert response.status_code == 401

View File

@@ -5,7 +5,7 @@
# on the SandboxTemplate, stamped before any user exists — a warm-pool # on the SandboxTemplate, stamped before any user exists — a warm-pool
# constraint), so a real key committed here would be shared by every install # constraint), so a real key committed here would be shared by every install
# of this example. Generate a fresh value per install (README step 1; # of this example. Generate a fresh value per install (README step 1;
# run-test-kind.sh uses a throwaway): # scripts/e2e-kind.sh uses a throwaway):
# #
# kubectl -n dsh-demo create secret generic dsh-provider-keys \ # kubectl -n dsh-demo create secret generic dsh-provider-keys \
# --from-literal=DEEPSEEK_API_KEY="sk-..." # --from-literal=DEEPSEEK_API_KEY="sk-..."

View File

@@ -2,6 +2,9 @@
# exists, so per-user secrets cannot live in pod env; the shared provider key # exists, so per-user secrets cannot live in pod env; the shared provider key
# is injected via envFrom, and claims that would bypass the warm pool are # is injected via envFrom, and claims that would bypass the warm pool are
# rejected outright (see 40-claim-rejected.yaml). # rejected outright (see 40-claim-rejected.yaml).
#
# A stripped-down single-tenant variant of this template (no warm pool, no
# policy enforcement) lives in examples/minimal-single-tenant/.
apiVersion: extensions.agents.x-k8s.io/v1beta1 apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxTemplate kind: SandboxTemplate
metadata: metadata:
@@ -18,8 +21,8 @@ spec:
# The gateway routes each user to their DSH Web instance by this name. # The gateway routes each user to their DSH Web instance by this name.
service: true service: true
# No gateway pod scopes ingress in this standalone demo, so the # The example gateway does not manage NetworkPolicy, so it is left
# NetworkPolicy is left unmanaged. A production platform would use # unmanaged here. A production platform would use
# networkPolicyManagement: Managed and restrict ingress to its gateway on # networkPolicyManagement: Managed and restrict ingress to its gateway on
# :3080 — with the shared provider key, that NetworkPolicy is the per-user # :3080 — with the shared provider key, that NetworkPolicy is the per-user
# isolation boundary. # isolation boundary.
@@ -41,11 +44,12 @@ spec:
# with 0660 and crash-loops before the gateway can proxy to it. # with 0660 and crash-loops before the gateway can proxy to it.
# OnRootMismatch skips that walk once the volume root already carries # OnRootMismatch skips that walk once the volume root already carries
# the fsGroup ownership/mode applied on the first mount. # the fsGroup ownership/mode applied on the first mount.
# Keep in sync with deploy/k8s/sandbox-template.yaml. # Keep in sync with the minimal variant:
# examples/minimal-single-tenant/manifests/sandbox-template.yaml.
fsGroupChangePolicy: OnRootMismatch fsGroupChangePolicy: OnRootMismatch
containers: containers:
- name: dsh-web - name: dsh-web
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
ports: ports:
- name: web - name: web
@@ -61,8 +65,10 @@ spec:
value: "" value: ""
# DSH Host/Origin-fences /api. The gateway forwards the browser's # DSH Host/Origin-fences /api. The gateway forwards the browser's
# Host, so DSH must trust the public gateway host. Empty is right # Host, so DSH must trust the public gateway host. Empty is right
# for port-forward/localhost; for the optional Ingress set this to # for port-forward/localhost; for either exposure variant set this
# the same host as 60-ingress.yaml (e.g. dsh.example.com). # to the public host (e.g. dsh.example.com for
# manifests/exposure/ingress/60-ingress.yaml, or
# dsh-demo.example.com for the gateway-api-sso/ HTTPRoute).
- name: DSH_TRUSTED_HOSTS - name: DSH_TRUSTED_HOSTS
value: "" value: ""
# Probes target /favicon.svg: DSH gates / behind its launch-token # Probes target /favicon.svg: DSH gates / behind its launch-token

View File

@@ -6,11 +6,19 @@
# POST /users, POST /login, ... session cookie auth # POST /users, POST /login, ... session cookie auth
# /u/<user>/... HTTP + WebSocket proxy to the user's DSH # /u/<user>/... HTTP + WebSocket proxy to the user's DSH
# #
# Image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway # Image: the release image published for this example, pinned by tag.
# For local kind development you can build and load a local tag instead, then # imagePullPolicy: IfNotPresent lets a local kind build loaded under the same
# change `image` below to it: # tag be used without a registry round-trip (scripts/e2e-kind.sh builds
# docker build -t dsh-aaas-gateway:demo gateway/ # dsh-aaas-gateway:e2e and overrides this image).
# kind load docker-image dsh-aaas-gateway:demo #
# SSO IS OPT-IN: this manifest runs the gateway with its own bearer/session
# auth only, so AUTH_PROXY_HEADER is intentionally NOT set (see the env
# block). To put GitHub OAuth in front of it, use
# manifests/exposure/gateway-api-sso/.
#
# This Service is ClusterIP and is the only intended entry point: expose it
# through exactly ONE of the documented variants in manifests/exposure/
# (ingress/ or gateway-api-sso/), never both and never directly.
apiVersion: v1 apiVersion: v1
kind: ServiceAccount kind: ServiceAccount
metadata: metadata:
@@ -84,8 +92,8 @@ spec:
type: RuntimeDefault type: RuntimeDefault
containers: containers:
- name: gateway - name: gateway
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2
imagePullPolicy: Always imagePullPolicy: IfNotPresent
securityContext: securityContext:
allowPrivilegeEscalation: false allowPrivilegeEscalation: false
readOnlyRootFilesystem: true readOnlyRootFilesystem: true
@@ -105,6 +113,10 @@ spec:
# gateway created are swept: a claim applied by hand from the # gateway created are swept: a claim applied by hand from the
# resource-layer manifests carries no aaas.example.com/managed-by # resource-layer manifests carries no aaas.example.com/managed-by
# annotation and is never suspended. # annotation and is never suspended.
#
# These are the demo values (360/240). scripts/e2e-kind.sh
# overrides both to 60/120 with `kubectl set env` so the smoke
# test does not wait six minutes for the sweeper.
- name: IDLE_TIMEOUT - name: IDLE_TIMEOUT
value: "360" value: "360"
- name: WAKE_TIMEOUT - name: WAKE_TIMEOUT
@@ -113,19 +125,24 @@ spec:
# the DSH launch token. # the DSH launch token.
- name: DSH_CONTAINER - name: DSH_CONTAINER
value: dsh-web value: dsh-web
# Cookie dsh_gw_session is HttpOnly + SameSite=Lax. Set to "true" # Cookie dsh_gw_session is HttpOnly + SameSite=Lax. "false" keeps
# when this Service is reached over HTTPS (e.g. the optional # the plain-HTTP demo usable through port-forward: a Secure cookie
# 60-ingress.yaml with TLS); "false" keeps the plain-HTTP demo # is never sent over HTTP, so login would silently fail.
# usable via port-forward. # HTTPS deployments (the ingress/ variant with TLS, or
# gateway-api-sso/, which terminates TLS at the Gateway) MUST set
# this to "true".
- name: COOKIE_SECURE - name: COOKIE_SECURE
value: "true" value: "false"
# Trusted reverse-proxy identity header for SSO. oauth2-proxy sets # SSO IS OPT-IN AND DISABLED HERE. With AUTH_PROXY_HEADER unset the
# X-Forwarded-User to the authenticated GitHub login on every request; # gateway trusts only its own bearer/session auth and ignores every
# the gateway trusts it (and auto-provisions the user) only when this # identity header. Enable it ONLY behind
# is set. Remove/unset to disable SSO and fall back to bearer/session # manifests/exposure/gateway-api-sso/, where oauth2-proxy is the
# auth only. # sole route to this Service: set it to X-Forwarded-User, keep the
- name: AUTH_PROXY_HEADER # Service ClusterIP and reachable only through the proxy, and
value: X-Forwarded-User # terminate TLS in front of the gateway. Otherwise anyone who can
# reach :8080 can impersonate any user by sending that header.
# - name: AUTH_PROXY_HEADER
# value: X-Forwarded-User
# Reject proxied request bodies larger than this (413). # Reject proxied request bodies larger than this (413).
- name: MAX_BODY_MB - name: MAX_BODY_MB
value: "32" value: "32"

View File

@@ -0,0 +1,135 @@
# Exposing the gateway
The base manifests deploy the gateway as a ClusterIP Service
(`dsh-gateway:8080`) and expose nothing outside the cluster. WebSocket
upgrades on `/u/<user>/**` are part of the contract, so the entry point must
forward `Upgrade`/`Connection` headers (Traefik does by default).
There are two **mutually exclusive** exposure variants. Pick exactly one:
| Variant | Entry point | Authentication | TLS |
| --- | --- | --- | --- |
| [`ingress/60-ingress.yaml`](ingress/60-ingress.yaml) | Traefik `Ingress` | the gateway's own signup/session cookie | optional, recommended |
| [`gateway-api-sso/`](gateway-api-sso/72-oauth2-proxy.yaml) | Gateway API `Gateway` + `HTTPRoute` | GitHub OAuth via oauth2-proxy, then the gateway cookie | required (the Gateway listener terminates it) |
Applying both gives the gateway two entry points, and the SSO variant is only
safe when the gateway is reachable **exclusively** through oauth2-proxy. Do not
mix them.
## Applying a variant
Run these commands from the example root
(`examples/dsh-agents-as-a-service/`), the same working directory the example
README uses.
`kubectl apply -f manifests/` is **not recursive**: it applies only the
top-level numbered files and does not descend into `exposure/`. Apply the base
resources first, then the chosen variant explicitly:
```bash
# Base resources: namespace, template, warm pool, claims, gateway.
kubectl apply -f manifests/
# Plain Ingress exposure:
kubectl apply -f manifests/exposure/ingress/
# ... or SSO exposure (create the two Secrets and replace the example.com
# placeholders first — see below):
kubectl apply -f manifests/exposure/gateway-api-sso/
```
**Never use `kubectl apply -R`** (recursive): it would apply both variants at
once and hand out a second, unauthenticated path to the same gateway. Apply the
variant directory on purpose, or the individual files by name.
`kubectl` only reads `.yaml`, `.yml` and `.json` files from a directory, so
the `*.yaml.example` secret templates are skipped automatically; create the real
Secrets imperatively (commands below) before applying the directory.
## `ingress/`: plain Ingress, no SSO
`60-ingress.yaml` routes a host through the cluster's Traefik
(`ingressClassName: traefik`; k3s ships one) to the gateway Service.
1. Replace the `dsh.example.com` host and the `dsh-example-tls` Secret name
with real values, or delete the `tls:` block to serve plain HTTP.
2. Set `DSH_TRUSTED_HOSTS` in the SandboxTemplate to the same public host:
the gateway forwards the browser's `Host`, and DSH rejects unknown hosts on
`/api` (see `../10-sandboxtemplate.yaml`).
3. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS:
`kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`. The
base manifest keeps `false` so the plain-HTTP port-forward demo works; a
`Secure` cookie is never sent over HTTP, which silently breaks login.
4. Do not attach buffering or compression middlewares to this router: they
break WebSocket upgrades and streaming.
Verify: `curl -sI https://<host>/healthz` returns `200`.
## `gateway-api-sso/`: TLS + GitHub OAuth
`70-gateway.yaml` defines the TLS `Gateway`, `71-httproute.yaml` sends every
request to oauth2-proxy, and `72-oauth2-proxy.yaml` authenticates the browser
against GitHub and proxies to the gateway with `X-Forwarded-User` set.
Prerequisites:
- Gateway API CRDs and a Gateway implementation installed;
`gatewayClassName: traefik` matches the Traefik Gateway provider. The CRD
version is pinned in `versions.env` (`GATEWAY_API_VERSION`).
- The OAuth App credentials and a cookie secret, from
`73-oauth2-proxy-secret.yaml.example`.
- A TLS certificate/key for the public host, from
`74-tls-secret.yaml.example`.
- oauth2-proxy is pinned in the manifest to
`quay.io/oauth2-proxy/oauth2-proxy:v7.15.5` (`versions.env`:
`OAUTH2_PROXY_IMAGE`).
Steps:
```bash
# 1. Real Secrets (the committed files are *.yaml.example templates).
kubectl -n dsh-demo create secret generic oauth2-proxy \
--from-literal=client-id='<GitHub OAuth App client ID>' \
--from-literal=client-secret='<GitHub OAuth App client secret>' \
--from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"
kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key
# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72) with
# the real public host and allowed GitHub organization.
# 3. Enable SSO on the gateway and trust the public host in the template.
kubectl -n dsh-demo set env deploy/dsh-gateway \
AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true
# then set DSH_TRUSTED_HOSTS=dsh-demo.example.com in the SandboxTemplate
# (see ../10-sandboxtemplate.yaml) and re-apply it.
# 4. Apply the variant.
kubectl apply -f manifests/exposure/gateway-api-sso/
```
### SSO trust boundary
`AUTH_PROXY_HEADER` is opt-in and deliberately unset in the base gateway
manifest: the gateway ignores identity headers until it is set. Enabling SSO
takes **both** sides:
1. `AUTH_PROXY_HEADER=X-Forwarded-User` on `deploy/dsh-gateway` (step 3
above). Without it, oauth2-proxy's login has no effect on the gateway and
users are still asked for the gateway's own token.
2. The gateway Service must be reachable **only through oauth2-proxy**: keep it
ClusterIP, route all browser traffic through the `HTTPRoute` →
oauth2-proxy → `dsh-gateway` chain, and never apply `ingress/` (or any
other path to the Service) while SSO is enabled. A client that can reach
`dsh-gateway:8080` directly can set `X-Forwarded-User` itself and
impersonate any user.
Also terminate TLS in front of the gateway (the Gateway listener does) and
leave `COOKIE_SECURE=true` so the session cookie never travels in clear text.
## Switching variants
Delete the variant you had before applying the other one, e.g.
`kubectl delete -f manifests/exposure/ingress/`, and revert
`AUTH_PROXY_HEADER`/`COOKIE_SECURE` on the gateway. Verify with
`kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to
the gateway remains.

View File

@@ -1,3 +1,8 @@
# Browser entry point (TLS) for the gateway-api-sso variant: terminates TLS
# for dsh-demo.example.com and routes through 71-httproute.yaml to
# oauth2-proxy, which authenticates the browser before the gateway.
# Mutually exclusive with ../../ingress/ (see ../../README.md). The
# "dsh-demo-tls" Secret comes from 74-tls-secret.yaml.example.
apiVersion: gateway.networking.k8s.io/v1 apiVersion: gateway.networking.k8s.io/v1
kind: Gateway kind: Gateway
metadata: metadata:

View File

@@ -17,5 +17,7 @@ spec:
- backendRefs: - backendRefs:
# Route through oauth2-proxy (GitHub OAuth) first; it authenticates the # Route through oauth2-proxy (GitHub OAuth) first; it authenticates the
# browser and forwards to dsh-gateway:8080 with X-Forwarded-User set. # browser and forwards to dsh-gateway:8080 with X-Forwarded-User set.
# The gateway only trusts that header when AUTH_PROXY_HEADER is set on
# its Deployment — see 72-oauth2-proxy.yaml and ../README.md.
- name: oauth2-proxy - name: oauth2-proxy
port: 4180 port: 4180

View File

@@ -4,10 +4,25 @@
# (restricted to a configured organization), then forwards the request to # (restricted to a configured organization), then forwards the request to
# the dsh-gateway Service with the authenticated identity in X-Forwarded-User # the dsh-gateway Service with the authenticated identity in X-Forwarded-User
# (plus X-Auth-Request-* via --set-xauthrequest). The gateway trusts that # (plus X-Auth-Request-* via --set-xauthrequest). The gateway trusts that
# header (AUTH_PROXY_HEADER) to inherit the username (SSO). # header only when AUTH_PROXY_HEADER is set on its Deployment.
# #
# Credentials come from the "oauth2-proxy" Secret (see # ENABLING SSO TAKES BOTH SIDES (see ../../../exposure/README.md):
# oauth2-proxy-secret.yaml.example); this manifest never carries secrets. # 1. Set AUTH_PROXY_HEADER=X-Forwarded-User on deploy/dsh-gateway, otherwise
# the gateway ignores the header and the login is cosmetic:
# kubectl -n dsh-demo set env deploy/dsh-gateway \
# AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true
# 2. Make dsh-gateway reachable ONLY through this proxy: keep the Service
# ClusterIP, route browser traffic through 71-httproute.yaml, and do not
# apply ../ingress/ (or anything else that reaches the gateway). A
# client that can reach dsh-gateway:8080 can spoof X-Forwarded-User and
# impersonate any user.
# TLS terminates at the Gateway listener (70-gateway.yaml); keep
# --cookie-secure=true and COOKIE_SECURE=true on the gateway.
#
# The image is pinned to quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
# (versions.env: OAUTH2_PROXY_IMAGE). Credentials come from the
# "oauth2-proxy" Secret (see 73-oauth2-proxy-secret.yaml.example); this
# manifest never carries secrets.
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:

View File

@@ -2,7 +2,7 @@
# #
# Copy to tls-secret.yaml and populate with real base64-encoded PEM material, # Copy to tls-secret.yaml and populate with real base64-encoded PEM material,
# then apply. The secret must live in the dsh-demo namespace and be named # then apply. The secret must live in the dsh-demo namespace and be named
# "dsh-demo-tls" (referenced by gateway.yaml). # "dsh-demo-tls" (referenced by 70-gateway.yaml).
# #
# kubectl -n dsh-demo create secret tls dsh-demo-tls \ # kubectl -n dsh-demo create secret tls dsh-demo-tls \
# --cert=tls.crt --key=tls.key --dry-run=client -o yaml > tls-secret.yaml # --cert=tls.crt --key=tls.key --dry-run=client -o yaml > tls-secret.yaml

View File

@@ -1,15 +1,21 @@
# OPTIONAL browser entry point: expose the gateway through Traefik. # OPTIONAL browser entry point: expose the gateway through Traefik.
# #
# Requires 50-gateway.yaml (Service dsh-gateway:8080) and a Traefik install # This is the "no SSO" exposure variant and is MUTUALLY EXCLUSIVE with
# with an IngressClass named "traefik" (k3s ships one). Replace the placeholder # ../gateway-api-sso/ — apply exactly one of them (see ../README.md).
# host below with a name that resolves to your ingress address, and either #
# supply a real TLS Secret or delete the `tls:` block for plain HTTP. # Requires the base manifests (Service dsh-gateway:8080 from
# manifests/50-gateway.yaml) and a Traefik install with an IngressClass named
# "traefik" (k3s ships one). Replace the placeholder host below with a name
# that resolves to your ingress address, and either supply a real TLS Secret or
# delete the `tls:` block for plain HTTP.
# #
# Two things must match this host: # Two things must match this host:
# 1. SandboxTemplate (10-sandboxtemplate.yaml): set DSH_TRUSTED_HOSTS to # 1. SandboxTemplate (manifests/10-sandboxtemplate.yaml): set
# this host so DSH accepts the forwarded Host/Origin on /api. # DSH_TRUSTED_HOSTS to this host so DSH accepts the forwarded Host/Origin
# 2. Gateway Deployment (50-gateway.yaml): set COOKIE_SECURE to "true" # on /api.
# once this Ingress serves HTTPS, so the session cookie is Secure. # 2. Gateway Deployment (manifests/50-gateway.yaml): set COOKIE_SECURE to
# "true" once this Ingress serves HTTPS, so the session cookie is Secure.
# The base manifest sets "false" for the plain-HTTP port-forward demo.
# #
# WebSocket support is REQUIRED: the browser talks to /u/<user>/... and DSH # WebSocket support is REQUIRED: the browser talks to /u/<user>/... and DSH
# uses WebSockets for live session traffic. Traefik upgrades WS/SSE # uses WebSockets for live session traffic. Traefik upgrades WS/SSE

View File

@@ -1,13 +1,37 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Smoke test for the dsh-agents-as-a-service example: runs the full README # End-to-end smoke test for the dsh-agents-as-a-service example: runs the
# walkthrough non-interactively against the current kubectl context. # README walkthrough non-interactively against the current kubectl context.
# Expects agent-sandbox (with extensions) already installed. #
# Prerequisites:
# - A kind cluster whose name matches the current kubectl context, or set
# KIND_CLUSTER=<name>.
# - agent-sandbox with extensions installed (API v1beta1). The release
# manifest is pinned by AGENT_SANDBOX_VERSION in versions.env:
# . ../../../versions.env
# kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${AGENT_SANDBOX_VERSION}/sandbox-with-extensions.yaml
# - docker, to build the gateway image (skipped with SKIP_GATEWAY=1).
#
# Environment:
# SKIP_GATEWAY=1 run only the resource-layer checks (no gateway build/load,
# no signup/proxy/WebSocket/idle/wake checks)
# KIND_CLUSTER kind cluster name for "kind load docker-image"
# (default: derived from the current kubectl context)
# GW_LOCAL_IMAGE local gateway image tag
# (default: dsh-aaas-gateway:e2e)
#
# The gateway is built from the working tree (gateway/) and loaded into kind,
# so this tests the example as checked out, not the published image.
#
# The script is destructive only inside the dsh-demo namespace: it creates the
# namespace (and a throwaway provider key) and DELETES it on exit via the
# cleanup trap.
# #
# NOTE: the only local port bound is the gateway port-forward on 18080. # NOTE: the only local port bound is the gateway port-forward on 18080.
# The DSH Web container port (3080) is in-cluster only and is never bound # The DSH Web container port (3080) is in-cluster only and is never bound
# on the local host. # on the local host.
set -euo pipefail set -euo pipefail
cd "$(dirname "$0")" # Example root: paths below are manifests/... and gateway/... from here.
cd "$(dirname "$0")/.."
NS=dsh-demo NS=dsh-demo
pass() { echo "PASS: $1"; } pass() { echo "PASS: $1"; }
@@ -20,12 +44,12 @@ cleanup() {
trap cleanup EXIT trap cleanup EXIT
echo "=== 1. install prereqs (throwaway key) + template + warm pool" echo "=== 1. install prereqs (throwaway key) + template + warm pool"
kubectl apply -f 00-prereqs.yaml kubectl apply -f manifests/00-prereqs.yaml
# The provider key is never committed; generate a throwaway value. # The provider key is never committed; generate a throwaway value.
kubectl -n "$NS" delete secret dsh-provider-keys --ignore-not-found >/dev/null kubectl -n "$NS" delete secret dsh-provider-keys --ignore-not-found >/dev/null
kubectl -n "$NS" create secret generic dsh-provider-keys \ kubectl -n "$NS" create secret generic dsh-provider-keys \
--from-literal=DEEPSEEK_API_KEY="sk-test-$(openssl rand -hex 16)" --from-literal=DEEPSEEK_API_KEY="sk-test-$(openssl rand -hex 16)"
kubectl apply -f 10-sandboxtemplate.yaml -f 20-sandboxwarmpool.yaml kubectl apply -f manifests/10-sandboxtemplate.yaml -f manifests/20-sandboxwarmpool.yaml
for _ in $(seq 1 120); do for _ in $(seq 1 120); do
READY=$(kubectl -n "$NS" get sandboxes -l agents.x-k8s.io/warm-pool-sandbox \ READY=$(kubectl -n "$NS" get sandboxes -l agents.x-k8s.io/warm-pool-sandbox \
-o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' | grep -c True || true) -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' | grep -c True || true)
@@ -37,7 +61,7 @@ pass "warm pool: 2 spares Ready"
echo "=== 2. signup via claim (warm adoption)" echo "=== 2. signup via claim (warm adoption)"
T0=$(date +%s) T0=$(date +%s)
kubectl apply -f 30-claim-alice.yaml kubectl apply -f manifests/30-claim-alice.yaml
kubectl -n "$NS" wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s kubectl -n "$NS" wait sandboxclaim dsh-alice --for=condition=Ready --timeout=60s
ELAPSED=$(( $(date +%s) - T0 )) ELAPSED=$(( $(date +%s) - T0 ))
SB=$(kubectl -n "$NS" get sandboxclaim dsh-alice -o jsonpath='{.status.sandbox.name}') SB=$(kubectl -n "$NS" get sandboxclaim dsh-alice -o jsonpath='{.status.sandbox.name}')
@@ -65,7 +89,7 @@ MARKER=$(kubectl -n "$NS" exec "$POD" -- cat /home/node/.dsh/marker.txt)
pass "resumed: state survived on the reattached PVC" pass "resumed: state survived on the reattached PVC"
echo "=== 6. injection policy rejects env-carrying claims" echo "=== 6. injection policy rejects env-carrying claims"
kubectl apply -f 40-claim-rejected.yaml kubectl apply -f manifests/40-claim-rejected.yaml
# Wait for the POSITIVE rejection signal, not merely "not bound yet". # Wait for the POSITIVE rejection signal, not merely "not bound yet".
REASON="" REASON=""
for _ in $(seq 1 12); do for _ in $(seq 1 12); do
@@ -78,19 +102,25 @@ done
|| fail "claim was not rejected (Ready reason='$REASON')" || fail "claim was not rejected (Ready reason='$REASON')"
BOUND=$(kubectl -n "$NS" get sandboxclaim dsh-mallory -o jsonpath='{.status.sandbox.name}' 2>/dev/null || true) BOUND=$(kubectl -n "$NS" get sandboxclaim dsh-mallory -o jsonpath='{.status.sandbox.name}' 2>/dev/null || true)
[ -z "$BOUND" ] || fail "env-carrying claim unexpectedly bound sandbox '$BOUND'" [ -z "$BOUND" ] || fail "env-carrying claim unexpectedly bound sandbox '$BOUND'"
kubectl delete -f 40-claim-rejected.yaml >/dev/null kubectl delete -f manifests/40-claim-rejected.yaml >/dev/null
pass "claim rejected with EnvVarsInjectionRejected and never bound" pass "claim rejected with EnvVarsInjectionRejected and never bound"
echo "=== 7. gateway: browser signup, proxy, idle-suspend, wake-on-connect" echo "=== 7. gateway: browser signup, proxy, idle-suspend, wake-on-connect"
if command -v docker >/dev/null && command -v kind >/dev/null; then if command -v docker >/dev/null && command -v kind >/dev/null; then
KIND_CLUSTER="${KIND_CLUSTER:-$(kubectl config current-context | sed 's/^kind-//')}" KIND_CLUSTER="${KIND_CLUSTER:-$(kubectl config current-context | sed 's/^kind-//')}"
# Tag exactly what 50-gateway.yaml references so the smoke test exercises GW_LOCAL_IMAGE="${GW_LOCAL_IMAGE:-dsh-aaas-gateway:e2e}"
# the shipped manifest unchanged (imagePullPolicy: IfNotPresent uses the # Test the working tree, not the published image: build the gateway from
# image loaded into kind). # gateway/, load it into kind, then point the Deployment at that tag. The
GW_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway # shipped manifest pins the release image; imagePullPolicy: IfNotPresent
docker build -q -t "$GW_IMAGE" gateway/ >/dev/null # makes the node use the loaded one.
kind load docker-image "$GW_IMAGE" --name "$KIND_CLUSTER" >/dev/null docker build -q -t "$GW_LOCAL_IMAGE" gateway/ >/dev/null
kubectl apply -f 50-gateway.yaml kind load docker-image "$GW_LOCAL_IMAGE" --name "$KIND_CLUSTER" >/dev/null
kubectl apply -f manifests/50-gateway.yaml
kubectl -n "$NS" set image deploy/dsh-gateway gateway="$GW_LOCAL_IMAGE"
# Override the demo timeouts from manifests/50-gateway.yaml (360/240) with the
# gateway's working-tree defaults, so idle-suspend and wake-on-connect are
# exercised in seconds instead of minutes.
kubectl -n "$NS" set env deploy/dsh-gateway IDLE_TIMEOUT=60 WAKE_TIMEOUT=120
kubectl -n "$NS" rollout status deploy/dsh-gateway --timeout=120s kubectl -n "$NS" rollout status deploy/dsh-gateway --timeout=120s
# Local port-forward uses 18080 (NOT 3080, which stays in-cluster). # Local port-forward uses 18080 (NOT 3080, which stays in-cluster).
kubectl -n "$NS" port-forward svc/dsh-gateway 18080:8080 >/dev/null 2>&1 & kubectl -n "$NS" port-forward svc/dsh-gateway 18080:8080 >/dev/null 2>&1 &

View File

@@ -0,0 +1,127 @@
# Minimal single-tenant DSH Web
The smallest Kubernetes deployment of DeepSeek Harness (DSH) Web: one
`SandboxTemplate`, a one-replica warm pool and one claim. It is the "just one
DSH Web instance" quickstart.
For the full platform — warm pool sizing, per-user claims, a browser-facing
gateway with cookie sessions, idle-suspend and wake-on-connect — see
[`examples/dsh-agents-as-a-service/`](../dsh-agents-as-a-service/README.md).
This example deliberately has no gateway: you reach DSH through a
`kubectl port-forward` and its launch-token URL.
## Prerequisites
- A Kubernetes cluster with a default StorageClass (kind and GKE both work).
- The agent-sandbox controller with extensions (API `v1beta1`) installed.
The version is pinned as `AGENT_SANDBOX_VERSION` in
[`versions.env`](../../versions.env):
```bash
# from the repository root:
. versions.env
kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${AGENT_SANDBOX_VERSION}/sandbox-with-extensions.yaml"
kubectl -n agent-sandbox-system rollout status deployment --timeout=180s
```
- A DeepSeek API key for real model calls. The pod starts without one; only
model requests fail.
## Quickstart
Run these from this example's directory (`examples/minimal-single-tenant/`).
```bash
# 1. Namespace + shared provider key. The key is injected into the sandbox via
# envFrom and is never committed.
kubectl create namespace dsh-demo
kubectl -n dsh-demo create secret generic dsh-provider-keys \
--from-literal=DEEPSEEK_API_KEY="sk-..."
# 2. The template itself (image pinned in the manifest).
kubectl apply -f manifests/sandbox-template.yaml
# 3. A template only describes pods; a warm pool turns it into running
# Sandboxes and a claim adopts one. One replica is enough here.
kubectl apply -f - <<'EOF'
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
name: dsh-pool
namespace: dsh-demo
spec:
replicas: 1
sandboxTemplateRef:
name: dsh-web-template
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
name: dsh
namespace: dsh-demo
spec:
warmPoolRef:
name: dsh-pool
EOF
kubectl -n dsh-demo wait sandboxclaim/dsh --for=condition=Ready --timeout=180s
```
## Talk to the instance
DSH gates `/` behind a launch token it prints only in its startup logs. Open
the printed URL once; DSH then sets its own auth cookie and the UI loads
normally.
```bash
SB=$(kubectl -n dsh-demo get sandboxclaim dsh -o jsonpath='{.status.sandbox.name}')
kubectl -n dsh-demo logs "$SB" | grep -m1 '?token='
# dsh web: http://<pod-ip>:3080/?token=...
kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
# open http://localhost:18081/?token=... from the log line
```
> **Port note:** DSH Web listens on `3080` in-cluster. If `3080` is already in
> use on your workstation, forward to another local port (here `18081`), never
> `3080`. `localhost` is always in DSH's trusted-host list, so the forwarded
> request is accepted.
## Probes and trusted hosts
Two settings in `manifests/sandbox-template.yaml` are easy to get wrong:
- **Probe path.** `/` returns `401` without the launch-token cookie, so a probe
there crash-loops the pod. The template probes `/favicon.svg` (served
unauthenticated) and gates liveness/readiness behind a `startupProbe`, which
gives a cold start (plugin activation, or the first resume) up to 120s before
a restart.
- **Host allow-list.** DSH rejects unknown `Host` headers. `localhost` and
`127.0.0.1` are always trusted; if you later front the sandbox with an
Ingress or proxy, set `DSH_TRUSTED_HOSTS` to that public host (comma or space
separated). The full example explains both exposure variants in
[`manifests/exposure/README.md`](../dsh-agents-as-a-service/manifests/exposure/README.md).
The template also sets `fsGroupChangePolicy: OnRootMismatch`. DSH's
credentials plugin requires `.credentials.yaml` to be `0600`, while the
kubelet's default fsGroup policy rewrites it to `0660` on every pod start —
without this policy a recreated or resumed sandbox crash-loops. The rationale
is spelled out in
[`10-sandboxtemplate.yaml`](../dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml).
## Suspend, resume and clean up
```bash
# Suspend: the pod is deleted, the PVC (state) and Service are kept.
kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Suspended"}}'
# Resume: a new pod reattaches the same PVC; conversations survive.
kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'
kubectl -n dsh-demo wait sandbox "$SB" --for=condition=Ready --timeout=180s
# Delete everything this quickstart created.
kubectl -n dsh-demo delete sandboxclaim dsh
kubectl delete namespace dsh-demo
```
Deleting the claim cascades to the sandbox and its PVC.

View File

@@ -4,8 +4,9 @@
# multi-tenant platform (warm pool, per-user claims, gateway, idle-suspend), # multi-tenant platform (warm pool, per-user claims, gateway, idle-suspend),
# see examples/dsh-agents-as-a-service/. # see examples/dsh-agents-as-a-service/.
# #
# Prerequisites (create these first, or apply 00-prereqs.yaml from the # Claims reference this template directly (spec.sandboxTemplateRef): there is
# examples directory): # no warm pool and no per-user platform here. See the README for the full
# quickstart; in short the prerequisites are:
# - Namespace dsh-demo. # - Namespace dsh-demo.
# - Secret dsh-provider-keys in dsh-demo with key DEEPSEEK_API_KEY # - Secret dsh-provider-keys in dsh-demo with key DEEPSEEK_API_KEY
# (injected via envFrom; the pod starts without it, only model calls fail). # (injected via envFrom; the pod starts without it, only model calls fail).
@@ -26,16 +27,15 @@ spec:
fsGroup: 1000 fsGroup: 1000
# DSH's credentials plugin refuses to start when .credentials.yaml has # DSH's credentials plugin refuses to start when .credentials.yaml has
# any group/other permission bit (it creates the file 0600). The # any group/other permission bit (it creates the file 0600). The
# default kubelet fsGroup policy ("Always") recursively ORs 0660 into # default kubelet fsGroup policy ("Always") rewrites it to 0660 on every
# every file each time the pod starts, so a recreated/resumed sandbox # pod start, so a recreated or resumed sandbox crash-loops; OnRootMismatch
# comes back with 0660 and crash-loops. OnRootMismatch skips that walk # avoids that walk once the volume root already carries the ownership.
# once the volume root already has the fsGroup ownership/mode from the # Rationale in full at
# first mount. Keep in sync with # examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml.
# examples/dsh-agents-as-a-service/10-sandboxtemplate.yaml.
fsGroupChangePolicy: OnRootMismatch fsGroupChangePolicy: OnRootMismatch
containers: containers:
- name: dsh-web - name: dsh-web
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2 image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
ports: ports:
- name: web - name: web
@@ -50,8 +50,10 @@ spec:
value: "" value: ""
# DSH Host/Origin-fences /api. Empty trusts only localhost, which # DSH Host/Origin-fences /api. Empty trusts only localhost, which
# is right for `kubectl port-forward`. Set this to the public host # is right for `kubectl port-forward`. Set this to the public host
# (e.g. dsh.example.com) if you expose DSH behind an Ingress, so # (e.g. dsh.example.com) if you front DSH with a proxy or Ingress,
# requests forwarded with the browser's Host are accepted. # so requests forwarded with the browser's Host are accepted. The
# full example documents both exposure variants in
# examples/dsh-agents-as-a-service/manifests/exposure/README.md.
- name: DSH_TRUSTED_HOSTS - name: DSH_TRUSTED_HOSTS
value: "" value: ""
volumeMounts: volumeMounts:

View File

@@ -1,91 +0,0 @@
# Gateway API — external access for `dsh-demo/dsh-gateway` (GitHub SSO)
Exposes the `dsh-gateway` service (`dsh-demo`, port `8080`) outside the cluster
using the Kubernetes Gateway API (Traefik Gateway controller), fronted by
**oauth2-proxy** so access requires a GitHub login restricted to a configured
GitHub organization. The gateway inherits the authenticated GitHub username via
the `X-Forwarded-User` header (SSO).
## Request flow
```
Browser ──▶ Gateway dsh-gateway (HTTPS, TLS-terminate)
──▶ HTTPRoute <your-hostname>
──▶ oauth2-proxy:4180 (GitHub OAuth, org-restricted)
──▶ dsh-gateway:8080 (X-Forwarded-User = GitHub login → SSO)
```
## Resources
| File | Kind | Purpose |
|---|---|---|
| `gateway.yaml` | `Gateway` | HTTPS listener with TLS termination |
| `httproute.yaml` | `HTTPRoute` | Routes the hostname → `oauth2-proxy:4180` |
| `oauth2-proxy.yaml` | `Deployment`/`Service` | GitHub OAuth reverse proxy |
| `oauth2-proxy-secret.yaml.example` | `Secret` | Example OAuth credentials (not applied as-is) |
| `tls-secret.yaml.example` | `Secret` | Example TLS cert secret (not applied as-is) |
## Prerequisites
1. **GitHub OAuth App** — create one at
`https://github.com/settings/developers` with:
- Homepage URL: `https://dsh-demo.example.com`
- Authorization callback URL: `https://dsh-demo.example.com/oauth2/callback`
(replace `dsh-demo.example.com` with the actual public FQDN). Note the client
ID and client secret.
2. **oauth2-proxy Secret** — `kubectl -n dsh-demo create secret generic oauth2-proxy`
with `client-id`, `client-secret`, and a `cookie-secret` (URL-safe, unpadded
base64 of 32 bytes: `python3 -c 'import secrets; print(secrets.token_urlsafe(32))'`).
See `oauth2-proxy-secret.yaml.example`.
3. **TLS secret** — create `dsh-demo-tls` in `dsh-demo` (see `tls-secret.yaml.example`).
4. **Gateway app env** — the `dsh-gateway` Deployment must run the SSO-capable
image with `AUTH_PROXY_HEADER=X-Forwarded-User` and `COOKIE_SECURE=true`
(see `examples/dsh-agents-as-a-service/50-gateway.yaml`).
5. **Ingress and DNS** — configure the upstream ingress and public DNS for the
hostname.
## Apply
```bash
kubectl diff -f gateway-api/dsh-demo/
kubectl apply -f gateway-api/dsh-demo/ --dry-run=server
kubectl apply -f gateway-api/dsh-demo/
```
> ⚠️ A write-enabled kubeconfig is required to create Gateways, HTTPRoutes,
> Deployments, and Secrets.
## Verify
```bash
kubectl get gateway dsh-gateway -n dsh-demo -o yaml # expect Programmed=True
kubectl get httproute -n dsh-demo -o wide # backend oauth2-proxy:4180, Accepted
kubectl get deploy oauth2-proxy -n dsh-demo # 1/1 Ready
kubectl get endpoints oauth2-proxy -n dsh-demo # backend present
kubectl -n dsh-demo logs deploy/oauth2-proxy --tail=50 # no config errors
```
Browser check: open `https://<your-hostname>/` — you should be redirected
to GitHub, and after signing in as a member of the allowed organization your
sandbox opens with the session showing your GitHub login. A non-member is denied
before reaching the gateway.
## Rollback
```bash
kubectl delete -f gateway-api/dsh-demo/
```
## Notes
- **SSO is opt-in and additive.** The gateway honors `X-Forwarded-User` only when
`AUTH_PROXY_HEADER` is set; unset it to restore bearer-token/session-only auth.
- **Trust boundary.** After this change the `dsh-gateway` Service must be reachable
**only** through oauth2-proxy (it is a ClusterIP Service). A second Route/Ingress
that exposes `dsh-gateway:8080` directly would let clients spoof the header.
- **GitHub login → DSH user.** GitHub usernames are lowercase alphanumerics and
hyphens, so they map 1:1 onto the gateway's dns-1123 `user` validation.
- **Sign-out.** The gateway `/logout` clears `dsh_gw_session`; oauth2-proxy's own
session can be cleared at `/oauth2/sign_out` if desired.
- No `ReferenceGrant` is required: the Gateway, HTTPRoute, oauth2-proxy, and the
backend Service are all in `dsh-demo`.

56
scripts/check-examples.sh Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# Enforce the example contract documented in docs/examples.md.
# Run by "make check" locally and by the CI lint job.
set -euo pipefail
cd "$(dirname "$0")/.."
index=examples/README.md
fail=0
err() { echo "check-examples: $*" >&2; fail=1; }
[ -f "$index" ] || err "missing $index"
for dir in examples/*/; do
name="$(basename "$dir")"
[ "$name" = "_template" ] && continue
[ -f "${dir}README.md" ] || err "$name: missing README.md"
[ -d "${dir}manifests" ] || err "$name: missing manifests/"
if [ -d "${dir}manifests" ]; then
count=$(find "${dir}manifests" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ')
[ "$count" -gt 0 ] || err "$name: manifests/ has no top-level *.yaml files"
fi
stray=$(find "$dir" -maxdepth 1 -type f -name '*.yaml' | wc -l | tr -d ' ')
[ "$stray" -eq 0 ] || err "$name: manifests belong under manifests/, not at the example root"
if [ -d "${dir}manifests/exposure" ] && [ ! -f "${dir}manifests/exposure/README.md" ]; then
err "$name: manifests/exposure/ requires a README.md describing the variants"
fi
for component in "$dir"*/; do
[ -d "$component" ] || continue
case "$(basename "$component")" in
manifests|scripts|docs) continue ;;
esac
if [ -f "${component}Dockerfile" ] && [ ! -f "${component}README.md" ]; then
err "$name/$(basename "$component"): component with a Dockerfile needs a README.md"
fi
done
if ! grep -Fq "examples/$name/" "$index"; then
err "$name: not listed in examples/README.md"
fi
for script in "$dir"scripts/*.sh; do
[ -e "$script" ] || continue
[ -x "$script" ] || err "$script: must be executable"
done
done
if [ "$fail" -ne 0 ]; then
echo "check-examples: FAILED" >&2
exit 1
fi
echo "check-examples: ok"

View File

@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""Parse every example manifest and assert the minimal Kubernetes shape."""
from __future__ import annotations
import sys
from pathlib import Path
try:
import yaml
except ModuleNotFoundError: # pragma: no cover - environment guard
sys.exit("check-manifests: PyYAML is required (python -m pip install pyyaml)")
ROOT = Path(__file__).resolve().parent.parent
FILES = sorted(ROOT.glob("examples/*/manifests/**/*.yaml"))
if not FILES:
sys.exit("check-manifests: no manifests found under examples/*/manifests/")
errors: list[str] = []
documents = 0
for path in FILES:
relative = path.relative_to(ROOT)
try:
loaded = list(yaml.safe_load_all(path.read_text(encoding="utf-8")))
except yaml.YAMLError as exc:
errors.append(f"{relative}: invalid YAML: {exc}")
continue
for position, document in enumerate(loaded, start=1):
if document is None:
continue
documents += 1
if not isinstance(document, dict):
errors.append(f"{relative} document {position}: not a mapping")
continue
missing = [key for key in ("apiVersion", "kind", "metadata") if key not in document]
if missing:
errors.append(f"{relative} document {position}: missing {', '.join(missing)}")
continue
metadata = document["metadata"]
if not isinstance(metadata, dict) or not metadata.get("name"):
errors.append(f"{relative} document {position}: metadata.name is required")
if errors:
print("\n".join(errors), file=sys.stderr)
sys.exit(1)
print(f"check-manifests: ok ({len(FILES)} files, {documents} documents)")

70
scripts/check-versions.sh Executable file
View File

@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# Verify that every version pin in the repository agrees with versions.env.
# Run by "make check" locally and by the CI lint job.
set -euo pipefail
cd "$(dirname "$0")/.."
# shellcheck source=/dev/null
. ./versions.env
IDENTITY="$DSH_VERSION"
if [ -n "${RELEASE_REVISION:-}" ]; then
IDENTITY="$DSH_VERSION-$RELEASE_REVISION"
fi
fail=0
err() { echo "check-versions: $*" >&2; fail=1; }
expect() { # <file> <literal>
if ! grep -Fq -- "$2" "$1"; then
err "$1: expected to contain: $2"
fi
}
# Main Dockerfile build args
expect Dockerfile "ARG DSH_VERSION=$DSH_VERSION"
expect Dockerfile "ARG PNPM_VERSION=$PNPM_VERSION"
expect Dockerfile "ARG RELEASE_REVISION=$RELEASE_REVISION"
# Local configuration mirrored for humans
expect .env.sample "DSH_VERSION=$DSH_VERSION"
expect .env.sample "PNPM_VERSION=$PNPM_VERSION"
expect .env.sample "RELEASE_REVISION=$RELEASE_REVISION"
# Image references in the example manifests
for f in \
examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml \
examples/minimal-single-tenant/manifests/sandbox-template.yaml; do
expect "$f" "image: $GHCR_IMAGE:$IDENTITY"
done
expect examples/dsh-agents-as-a-service/manifests/50-gateway.yaml "image: $GATEWAY_IMAGE:$IDENTITY"
# The gateway image must live below the repository image (one package per
# example component, no unrelated top-level packages).
case "$GATEWAY_IMAGE" in
"$GHCR_IMAGE"/*) ;;
*) err "GATEWAY_IMAGE ($GATEWAY_IMAGE) must be nested below GHCR_IMAGE ($GHCR_IMAGE)" ;;
esac
# Build tooling must read versions.env instead of hard-coding a version.
# (scripts/ is excluded: set-version.sh documents usage examples in comments.)
while IFS= read -r hit; do
[ -n "$hit" ] || continue
err "hard-coded version outside versions.env: $hit"
done < <(grep -rnE 'dsh-v[0-9]' .github/workflows Makefile 2>/dev/null || true)
# Prose may mention versions, but only the current identity or its base tag.
while IFS= read -r token; do
[ -n "$token" ] || continue
case "$token" in
"$IDENTITY"|"$DSH_VERSION") ;;
*) err "stale version reference in docs: $token" ;;
esac
done < <(grep -rhoE 'dsh-v[0-9][A-Za-z0-9._-]*' \
--include='*.md' --exclude-dir=.git --exclude-dir=.venv --exclude-dir=.uvcache \
--exclude=CHANGELOG.md . | sort -u)
if [ "$fail" -ne 0 ]; then
echo "check-versions: FAILED" >&2
exit 1
fi
echo "check-versions: ok ($IDENTITY)"

64
scripts/set-version.sh Executable file
View File

@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# Rewrite every version pin in one pass, then leave the tree ready for
# "make check".
#
# ./scripts/set-version.sh dsh-v0.2.2-rc.1 # new DSH ref, no revision
# ./scripts/set-version.sh dsh-v0.2.1-rc.2 r3 # same ref, new repo revision
set -euo pipefail
cd "$(dirname "$0")/.."
new_version="${1:-}"
new_revision="${2:-}"
case "$new_version" in
dsh-v[0-9]*) ;;
*) echo "usage: $0 <dsh-vX.Y.Z> [rN]" >&2; exit 2 ;;
esac
if [ -n "$new_revision" ]; then
case "$new_revision" in
r[2-9]|r[1-9][0-9]*) ;;
*) echo "revision must look like r2, r3, ..." >&2; exit 2 ;;
esac
fi
# shellcheck source=/dev/null
old_version="$(. ./versions.env; printf '%s' "$DSH_VERSION")"
# shellcheck source=/dev/null
old_revision="$(. ./versions.env; printf '%s' "${RELEASE_REVISION:-}")"
old_identity="$old_version"
[ -z "$old_revision" ] || old_identity="$old_version-$old_revision"
new_identity="$new_version"
[ -z "$new_revision" ] || new_identity="$new_version-$new_revision"
if [ "$old_identity" = "$new_identity" ]; then
echo "set-version: already at $new_identity"
exit 0
fi
# Replace the identity first (it contains the base version), then the base
# version itself. CHANGELOG.md is generated from tags, so it is left alone.
while IFS= read -r file; do
[ -f "$file" ] || continue
case "$file" in
.env.sample|Dockerfile|Makefile|*.md|*.yaml|*.yml|*.sh|*.py|*.toml|*.env) ;;
*) continue ;;
esac
sed -i "s/$old_identity/$new_identity/g; s/$old_version/$new_version/g" "$file"
done < <(git ls-files)
# versions.env is authoritative: write both keys explicitly so an empty
# revision is stored as an empty value.
sed -i "s/^DSH_VERSION=.*/DSH_VERSION=$new_version/" versions.env
sed -i "s/^RELEASE_REVISION=.*/RELEASE_REVISION=$new_revision/" versions.env
# Values stored separately from the full identity: Dockerfile ARG defaults and
# the human-facing sample cannot be reached by the identity replacement above.
sed -i "s/^ARG DSH_VERSION=.*/ARG DSH_VERSION=$new_version/" Dockerfile
sed -i "s/^ARG RELEASE_REVISION=.*/ARG RELEASE_REVISION=$new_revision/" Dockerfile
sed -i "s/^DSH_VERSION=.*/DSH_VERSION=$new_version/" .env.sample
sed -i "s/^RELEASE_REVISION=.*/RELEASE_REVISION=$new_revision/" .env.sample
echo "set-version: $old_identity -> $new_identity"
echo "set-version: run 'make check' and review the diff"

26
versions.env Normal file
View File

@@ -0,0 +1,26 @@
# Single source of truth for every pinned version and image name in this
# repository. Consumed by the Makefile, build.sh, scripts/*.sh and the GitHub
# Actions workflows. Never hard-code these values elsewhere: run "make check"
# (scripts/check-versions.sh) to verify every pin.
# Upstream DeepSeek Harness source, and the ref this repository packages.
DSH_REPO=https://github.com/deepseek-ai/deepseek-harness.git
DSH_VERSION=dsh-v0.2.1-rc.2
PNPM_VERSION=11.28.5
# Release identity = DSH_VERSION plus an optional repository revision.
# An empty RELEASE_REVISION is the first release at this DSH ref; rN (N >= 2)
# marks a repository-only re-release at the same DSH ref. The dsh-v0.2.1-rc.2
# image tag is already published on GHCR and tags are immutable, so the first
# release under the current scheme is dsh-v0.2.1-rc.2-r2.
RELEASE_REVISION=r2
# Control-plane versions used by the examples and their manifests.
AGENT_SANDBOX_VERSION=v1.0.7
GATEWAY_API_VERSION=v1.2.1
OAUTH2_PROXY_IMAGE=quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
# Registry layout: the main image is the repository itself, and every example
# component is a nested package below it (<registry>/<owner>/<repo>/<example>/<component>).
GHCR_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox
GATEWAY_IMAGE=ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway