nicolas-georger/deepseek-kubernetes-sandbox
1
0
Fork 0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00
Code Projects Releases Activity

feat(repo)!: repo reorganization (#16)

Browse Source 
* feat(repo)!: restructure around the dsh-harness image and one example per implementation

- the root Dockerfile is the primary artifact; versions.env is the single source of truth
- merge gateway-api/dsh-demo into examples/dsh-agents-as-a-service (manifests/, exposure variants)
- move the single-tenant template into examples/minimal-single-tenant
- publish the gateway as a nested package under the repository image
- mirror the DSH ref for release identities (first release dsh-v0.2.1-rc.2-r2)
- CI: lint/python/manifests/gateway-image, reusable multi-arch build, tag-driven release, weekly e2e
- docs: README, docs/{main-image,examples,gitflow,versioning}, example READMEs, AGENTS.md, governance
- SSO is opt-in again; gateway pull policy and cookie defaults match the documented demo

BREAKING CHANGE: example manifest paths, the gateway image name, and the removal
of deploy/ and gateway-api/ affect existing deployments; see the migration table
in README.md.

* docs(changelog): generate initial changelog from history

* fix(scripts): make set-version update Dockerfile and sample revision pins

* build(hadolint): document intentional lint exceptions for the main image

---------

Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
This commit is contained in:
Nicolás Georger
2026-10-25 08:37:43 -03:00
committed by GitHub
parent 53e55fc851
commit 3b70f179ac
69 changed files with 3774 additions and 1643 deletions
Download Patch File Download Diff File Expand all files Collapse all files

324
README.md
View File

@@ -1,180 +1,214 @@
# deepseek-kubernetes-sandbox
Packages [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web (`dsh web`) into
a multi-arch, Kubernetes-ready container image, and ships the
[agents-as-a-service](examples/dsh-agents-as-a-service/) platform pattern — running a personal
DSH Web instance per user on [kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox)
behind a small browser-usable gateway.
**DeepSeek Harness Web, packaged as a container.** This repository's primary
artifact is a slim, non-root, multi-arch image of
[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web
(`dsh web`), ready for Docker and Kubernetes. Everything else exists to support
that image or to demonstrate an implementation on top of it.
## What's in this repo
The flagship implementation is
[`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/): the
"agents-as-a-service" platform pattern with one DSH Web instance per user,
provisioned from a warm pool on
[kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox),
suspended when idle, and fronted by a small browser-usable gateway.
| Path | Purpose |
| --- | --- |
| [`Dockerfile`](Dockerfile) | Multi-stage build that clones DSH, `pnpm pack`s the CLI, and produces a slim runtime image (`:3080`) |
| [`build.sh`](build.sh), [`Makefile`](Makefile) | Local image builds: DSH Web and gateway, plus test/lint targets |
| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | The full multi-tenant platform pattern (warm pool, per-user claims, gateway, smoke test) |
| [`examples/dsh-agents-as-a-service/gateway/`](examples/dsh-agents-as-a-service/gateway/) | Starlette ASGI gateway: signup, cookie sessions, `/u/{user}/**` HTTP + WebSocket proxy, idle sweeper |
| [`deploy/k8s/sandbox-template.yaml`](deploy/k8s/sandbox-template.yaml) | Minimal single-tenant `SandboxTemplate` |
| [`docs/architecture.md`](docs/architecture.md) | Container and sequence diagrams, design decisions, security notes |
| [`.github/workflows/`](.github/workflows/) | CI: manifests dry-run, gateway tests/lint, multi-arch image builds |
## Published images
Everything is published to GitHub Container Registry as
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox`:
| Tag | Contents | Port |
| --- | --- | --- |
| `dsh-v0.2.1-rc.2` (also `main`, `latest`) | DSH Web image | `3080` |
| `gateway` | Starlette gateway image | `8080` |
The DSH Web tag matches the upstream DSH git tag it was built from.
## Quick start (DSH Web image)
## Quick start
```bash
docker run -d --name dsh-web \
-p 3080:3080 \
-v dsh-home:/home/node/.dsh \
-e DEEPSEEK_API_KEY \
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
```
**DSH Web is not usable at a bare `/`.** Without its launch-token cookie it answers `401`, and
the token is only printed in the startup logs:
DSH Web is **not usable at a bare `/`**. Without its launch-token cookie it
answers `401`, and the token is printed only in the startup logs:
```bash
docker logs dsh-web 2>&1 | grep -m1 '?token='
# dsh web: http://localhost:3080/?token=...
```
Open the printed `?token=...` URL once — DSH sets its auth cookie and the UI loads normally
afterwards. Hitting `/` directly (or after the cookie expires) returns `401` by design; that is
why Kubernetes probes use `/favicon.svg` (see below) and why the gateway performs this
handshake for the browser.
Open the printed `?token=...` URL once: DSH sets its auth cookie and the UI loads
normally afterwards. Hitting `/` directly (or after the cookie expires) returns
`401` by design.
> **Different host port?** If `3080` is already taken on your workstation, map another port and
> set the public URL so the printed token URL matches: `-p 18081:3080 -e
> DSH_PUBLIC_URL=http://localhost:18081`, then open `http://localhost:18081/?token=...`.
### Host port and public URL
## Quick start (Kubernetes)
If `3080` is taken locally, map another port and set the public URL so the quoted
token URL matches:
```bash
kubectl create namespace dsh-demo --dry-run=client -o yaml | kubectl apply -f -
kubectl -n dsh-demo create secret generic dsh-provider-keys \
--from-literal=DEEPSEEK_API_KEY="sk-..."
kubectl apply -f deploy/k8s/sandbox-template.yaml
# A template needs a warm pool and a claim to become a running Sandbox.
kubectl apply -f - <<'EOF'
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
name: dsh-pool
namespace: dsh-demo
spec:
replicas: 1
sandboxTemplateRef:
name: dsh-web-template
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
name: dsh
namespace: dsh-demo
spec:
warmPoolRef:
name: dsh-pool
EOF
kubectl -n dsh-demo wait sandboxclaim/dsh --for=condition=Ready --timeout=180s
docker run -d --name dsh-web -p 18081:3080 \
-e DSH_PUBLIC_URL=http://localhost:18081 \
-v dsh-home:/home/node/.dsh \
-e DEEPSEEK_API_KEY \
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
# then open http://localhost:18081/?token=...
```
Then reach the instance through its launch-token URL:
### Hosts, probes and the 401
- **Probes must not hit `/`.** It returns `401` without the launch token, so a
readiness/liveness probe on `/` crash-loops the pod. Probe `/favicon.svg`
instead: it is unauthenticated. The shipped manifests use it.
- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and
`127.0.0.1` are always trusted; add any other public hostname to
`DSH_TRUSTED_HOSTS` (comma or space separated). A gateway or Ingress that
forwards the browser's `Host` needs its hostname listed there.
- **State lives in `/home/node/.dsh`.** Mount a volume (Docker) or PVC
(Kubernetes) to keep conversations, sessions and config across restarts.
### Runtime environment
| Variable | Default | Purpose |
| --- | --- | --- |
| `DEEPSEEK_API_KEY` | empty | Provider credential consumed by DSH. Inject it; never bake it into the image or commit it. |
| `DSH_PORT` | `3080` | Port DSH Web listens on inside the container. |
| `DSH_HOST` | auto-detected pod/container IP | Bind address. Leave empty in Kubernetes. |
| `DSH_PUBLIC_URL` | `http://localhost:${DSH_PORT}` | Public base URL used in generated links and the launch-token URL in the logs. |
| `DSH_TRUSTED_HOSTS` | empty | Extra trusted `Host`/`Origin` entries, comma/space separated. |
| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (mount a volume here). |
Full contract, build args, probes, volumes and signals:
[docs/main-image.md](docs/main-image.md).
## Published images
Published to GitHub Container Registry. The main image is the repository image;
example components are nested packages below it:
```
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox # DSH Web (main image)
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway # example gateway
```
| Tag | Contents | Meaning |
| --- | --- | --- |
| `dsh-v0.2.1-rc.2-r2` | DSH Web, port `3080` | **Current release identity.** Immutable: once published it never moves. |
| `latest` | DSH Web | Most recent release; same digest as the current identity. |
| `main` | DSH Web | Development channel rebuilt from every push to `main`. Not a release; not for production. |
| `dsh-v0.2.1-rc.2` | DSH Web | **Legacy tag** published before the release-identity scheme. It is not updated. New deployments use `dsh-v0.2.1-rc.2-r2`. |
The example gateway is published under its nested path with the same release
identity, e.g.
`ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2`.
Release identities, channels and rollback rules are explained in
[docs/versioning.md](docs/versioning.md). Production deployments should pin a
digest:
```bash
SB=$(kubectl -n dsh-demo get sandboxclaim dsh -o jsonpath='{.status.sandbox.name}')
kubectl -n dsh-demo logs "$SB" | grep -m1 '?token='
kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
# open http://localhost:18081/?token=... from the log line
docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
# image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest>
```
Two things bite first-time setups:
- **Probes must not hit `/`.** DSH returns `401` there without the launch token, so a
readiness/liveness probe on `/` crash-loops the pod. The shipped templates use
`/favicon.svg`, which is unauthenticated; keep that path in custom templates.
- **Host allow-listing.** DSH rejects unknown `Host` headers. `localhost` and `127.0.0.1` are
always trusted; for any other hostname add it to the template's `DSH_TRUSTED_HOSTS` (comma or
space separated). The gateway forwards the browser's `Host`, so the public gateway hostname
must be listed there.
## The multi-user platform pattern
`examples/dsh-agents-as-a-service/` is a port of the upstream
[hermes-agents-as-a-service](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/hermes-agents-as-a-service)
example to DSH Web. It demonstrates the platform shape — one `SandboxTemplate`, one warm pool,
one tiny `SandboxClaim` per user, and `operatingMode` as the cost dial — plus a gateway that
turns it into a usable service.
**Browser flow:** open the gateway landing page, sign up, and get an `HttpOnly` session cookie.
The browser then browses `http://<gateway>/u/<user>/`, which proxies HTTP **and WebSocket** to
that user's DSH Web. The gateway reads the DSH launch token from the pod logs, performs the
token handshake server-side, and injects the resulting DSH cookie on every proxied request — the
browser never sees the token. Suspended instances are woken transparently on the next request;
idle ones are suspended again after `IDLE_TIMEOUT`.
See the [example README](examples/dsh-agents-as-a-service/README.md) for the full walkthrough
and [docs/architecture.md](docs/architecture.md) for the diagrams and trade-offs.
### Mechanism mapping: Hermes → DSH
| Hermes example mechanism | Role | Adaptation for DSH |
| :--- | :--- | :--- |
| `SandboxTemplate` | Reusable Pod blueprint | Image `dsh-web`, port `3080`, probes on `/favicon.svg` |
| `volumeClaimTemplates` | Auto-create a PVC per user | Mount `/home/node/.dsh` (DSH state) |
| `SandboxWarmPool` | Pre-warmed instances | Keep as-is; size for expected concurrency |
| `SandboxClaim` + `warmPoolRef` | "Registration" adopts a pooled instance | Keep exactly as-is; one claim per user |
| `service: true` | Stable DNS per Sandbox | Enabled, so the gateway routes to each user's DSH Web |
| `envVarsInjectionPolicy: Disallowed` | Reject claims that would cold-start | `DEEPSEEK_API_KEY` injected via `envFrom` on the template (shared platform key) |
| `operatingMode` | Cost switch: Running / Suspended | Reused directly — suspend idle instances to save cost |
### Key differences from Hermes
- **Multi-arch** — the DSH image is built for amd64 and arm64 (no amd64 nodeSelector).
- **Single port** — DSH Web serves everything on `3080`; no API/dashboard port split.
- **One secret** — only `DEEPSEEK_API_KEY`; no dashboard/session/api-server credentials.
- **Host forwarding** — the gateway forwards `Host` + `X-Forwarded-*` (and keeps
`DSH_PUBLIC_URL` empty) so DSH generates reachable links; add the public hostname to
`DSH_TRUSTED_HOSTS`.
- **Browser sessions + WebSocket** — the gateway is Starlette ASGI, so it proxies WebSocket
upgrades as well as HTTP/SSE, and authenticates browsers with an `HttpOnly` cookie instead of
exposing per-user bearer tokens in URLs.
## Build it yourself
## Build from source
```bash
./build.sh # build the DSH Web image, then run it interactively
./build.sh web # build the DSH Web image only
./build.sh gateway # build the gateway image
make image # same as ./build.sh web
make gateway # gateway image
make test # gateway unit tests (pytest)
make lint # gateway lint (ruff)
make image # build the main DSH Web image (clones and builds upstream DSH)
make run # build it, then run it interactively (needs DEEPSEEK_API_KEY)
```
Or directly:
The pins (`DSH_VERSION`, `RELEASE_REVISION`, `PNPM_VERSION`) come from
[versions.env](versions.env) — never hard-code them
(`make check` verifies). [`build.sh`](build.sh) is a deprecated shim that
forwards to `make`; use the Makefile targets.
```bash
docker build \
--build-arg DSH_VERSION=dsh-v0.2.1-rc.2 \
--build-arg PNPM_VERSION=11.28.5 \
-t dsh-web:0.2.1-rc.2 .
## Examples
docker build -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:gateway \
examples/dsh-agents-as-a-service/gateway/
```
Self-contained implementations that consume the image. Index and contract:
[`examples/README.md`](examples/README.md) and
[docs/examples.md](docs/examples.md).
See [`Dockerfile`](Dockerfile) for build args (`DSH_REPO`, `DSH_VERSION`, `PNPM_VERSION`) and
the runtime image contract (non-root `node` user, `DSH_HOME=/home/node/.dsh`, `EXPOSE 3080`).
`.env.sample` documents the runtime variables for both images.
| Example | What it demonstrates |
| --- | --- |
| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Full multi-tenant platform: SandboxTemplate with per-user PVCs, warm pool, one claim per user, suspend-when-idle, wake-on-return, browser gateway with signup + `HttpOnly` sessions + HTTP/WebSocket proxy, Ingress or Gateway API + GitHub SSO exposure. Start with the [walkthrough](examples/dsh-agents-as-a-service/README.md) and the [architecture](examples/dsh-agents-as-a-service/docs/architecture.md). |
| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | The smallest useful setup: one `SandboxTemplate` and one `SandboxClaim`, reached through `kubectl port-forward` + the launch-token URL. Manifest: [`manifests/sandbox-template.yaml`](examples/minimal-single-tenant/manifests/sandbox-template.yaml). |
The gateway is a component of the multi-tenant example and has its own
[README](examples/dsh-agents-as-a-service/gateway/README.md) (API, configuration,
auth/SSO, limitations).
## Repository layout
| Path | Purpose |
| --- | --- |
| [`Dockerfile`](Dockerfile) | Primary artifact: the multi-stage DSH Web image (port `3080`). |
| [`versions.env`](versions.env) | Single source of truth for the DSH ref, repository revision, pnpm/control-plane versions and image names. |
| [`Makefile`](Makefile), [`build.sh`](build.sh) | Canonical local entrypoint; `build.sh` is a deprecated shim. |
| [`docs/`](docs/README.md) | Repository docs: image contract, example contract, gitflow, versioning. |
| [`examples/`](examples/README.md) | One self-contained implementation per directory. |
| [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Multi-tenant platform example: `manifests/`, `gateway/`, `docs/`, `scripts/`. |
| [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | Single-sandbox quickstart. |
| [`scripts/`](scripts/) | Repository checks: versions, example layout, manifest YAML, version bump. |
| [`.github/`](.github/) | CI, release automation, dependency updates, PR/issue templates. |
| [`.env.sample`](.env.sample) | Documented local configuration (copy to `.env`; never commit it). |
## Development
The Makefile is the canonical entrypoint. No cluster is needed for `check`,
`test`, `lint`, `gateway` or `image`:
| Target | What it does |
| --- | --- |
| `make check` | Version pins, example layout and manifest YAML checks. Run before every commit. |
| `make test` | Gateway unit tests (`pytest`). |
| `make lint` | `ruff` + `shellcheck` (+ `yamllint`/`actionlint` when installed). |
| `make image` | Build the main DSH Web image. |
| `make gateway` | Build the example gateway image locally (`dsh-aaas-gateway:dev`). |
| `make e2e` | Full kind walkthrough of the multi-tenant example (needs a cluster with agent-sandbox). |
| `make changelog` | Regenerate `CHANGELOG.md` with git-cliff (release only). |
| `make set-version` | Bump pins in `versions.env`; e.g. `make set-version DSH_VERSION=<ref> RELEASE_REVISION=rN`. |
Local setup and the PR workflow: [CONTRIBUTING.md](CONTRIBUTING.md); branches,
required checks and releases: [docs/gitflow.md](docs/gitflow.md).
## Versioning and releases
The release identity is the upstream DSH ref plus an optional repository
revision: `DSH_VERSION` and `RELEASE_REVISION` in [versions.env](versions.env).
The current identity is **`dsh-v0.2.1-rc.2-r2`** — the first release under the
mirrored-DSH scheme at the same upstream DSH ref as the legacy
`dsh-v0.2.1-rc.2` tag. Tags are immutable; a broken release is fixed by
publishing `-r(N+1)`, never by re-tagging.
`CHANGELOG.md` is generated by [git-cliff](https://git-cliff.org) from
Conventional Commits and is never edited by hand. See
[`CHANGELOG.md`](CHANGELOG.md), [docs/versioning.md](docs/versioning.md) and
[cliff.toml](cliff.toml).
## Migrating from the previous layout
If you are coming from an older checkout, these paths moved. Nothing else changed
semantically; the content was split into the example it belongs to.
| Old path | New path |
| --- | --- |
| `gateway-api/dsh-demo/` (Gateway API + oauth2-proxy / GitHub SSO) | `examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/` (now `70-gateway.yaml` ... `74-tls-secret.yaml.example`) |
| `examples/dsh-agents-as-a-service/60-ingress.yaml` | `examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml` |
| `examples/dsh-agents-as-a-service/NN-*.yaml` (`00-` ... `50-`) | `examples/dsh-agents-as-a-service/manifests/NN-*.yaml` |
| `examples/dsh-agents-as-a-service/run-test-kind.sh` | `examples/dsh-agents-as-a-service/scripts/e2e-kind.sh` |
| `deploy/k8s/sandbox-template.yaml` | `examples/minimal-single-tenant/manifests/sandbox-template.yaml` |
| `docs/architecture.md` | `examples/dsh-agents-as-a-service/docs/architecture.md` |
The SSO walkthrough that lived in `gateway-api/dsh-demo/README.md` is merged
into the [multi-tenant example README](examples/dsh-agents-as-a-service/README.md)
(step 9) and the [exposure README](examples/dsh-agents-as-a-service/manifests/exposure/README.md).
## Contributing
- [CONTRIBUTING.md](CONTRIBUTING.md) — local setup, PR workflow, what a change needs.
- [AGENTS.md](AGENTS.md) — repository conventions for AI coding agents (and humans) and the CI commands.
- [SECURITY.md](SECURITY.md) — private vulnerability reporting and project threat notes.
- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — community standards.
- [docs/gitflow.md](docs/gitflow.md) — branches, protected `main`, required checks, release process.
Do not commit secrets; secret-shaped YAML lives in `*.yaml.example` files, and
version pins change only through `versions.env` / `make set-version`.
## License
[MIT](LICENSE) © Nicolás Georger.