1
0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00
Files
deepseek-kubernetes-sandbox/docs/main-image.md
SREDevOps.org Agent a09d4ac340 fix: ship bubblewrap for DSH's command sandbox, keep agents usable without it (#35)
* fix(example): keep agents usable when DSH's inner sandbox has no backend

DSH confines bash commands itself (bubblewrap, else Landlock) and fails closed
with SANDBOX_UNAVAILABLE when no runner is usable. In this hardened pod -
non-root, capabilities dropped, seccomp RuntimeDefault - bubblewrap cannot
start (it needs CAP_SYS_ADMIN) and the pod exposes no Landlock runner, so the
shipped workspace-write default made every command fail:

  sandbox mode "workspace-write" is requested but no sandbox backend is usable

The DSH base bundle at the pinned ref documents DSH_PERMISSION_MODE as the
explicit deployment override, so set it to danger-full-access in both sandbox
templates. The per-user pod is the isolation boundary in this example (its own
pod, PVC and Service; non-root; all capabilities dropped), so a working agent
is worth more than an inner file policy that cannot be enforced. Note that
danger-full-access also turns approval prompts off.

Trade-off documented rather than hidden: an agent command can now read its own
DSH_HOME (including .credentials.yaml) and write anywhere in the container.
Hosts where a runner works should keep workspace-write or read-only.

- main-image docs: DSH_PERMISSION_MODE row plus a note on the runners and the
  fail-closed behavior
- example README: limits bullet and a troubleshooting row with the in-pod
  backend check and how to restore a confined mode
- architecture: tenant-isolation bullet extended
- skill runbook/troubleshooting: operation guidance; never grant CAP_SYS_ADMIN
  or seccompProfile: Unconfined to "fix" this without an explicit decision

Verified: make check (14 manifests), make lint, YAML assertions on both
templates, and the override expression at the pinned dsh-v0.2.1-rc.5 base
bundle (mode: !!js process.env.DSH_PERMISSION_MODE ?? 'workspace-write').

* fix(image): install bubblewrap for DSH's command sandbox

DSH's Linux sandbox runner chain is bubblewrap, then Landlock, and it fails
closed with SANDBOX_UNAVAILABLE when neither works. The image shipped neither,
so a sandboxed agent had no way to execute confined commands.

Add bubblewrap to the runtime stage. Debian ships it unsetuid (0.12.0), so it
still needs unprivileged user namespaces plus the namespace/mount syscalls: in a
hardened pod (all capabilities dropped, RuntimeDefault seccomp) it usually
cannot create one, and DSH then falls back to Landlock or, failing that, to the
DSH_PERMISSION_MODE=danger-full-access override the templates already set.
Where the host allows an unprivileged user namespace, the first-choice runner
now exists, so the confined modes can be restored.

Docs corrected to match: bubblewrap is no longer "absent"; the blocker is the
unsetuid binary's namespace requirement, not a missing CAP_SYS_ADMIN grant.

Verified: make check, make lint; the package exists in the pinned base
(node:24-trixie-slim, bubblewrap 0.12.0-1~deb13u1); a throwaway container with
the pod's flags (--user 1000 --cap-drop ALL --security-opt no-new-privileges)
still cannot create a namespace, which is why the override stays. No local image
build was run.

* ci(image): build and smoke-test the main image on PRs that touch it

The Dockerfile and the pins in versions.env could only be validated by a merge
to main or by running `make image` by hand; neither gives a PR-time signal.

Add ci-image.yaml: on pull requests that change Dockerfile, .dockerignore or
versions.env (and on manual dispatch) it builds linux/amd64 natively with the
pins sourced from versions.env - no registry login, no push - loads the image
locally and smoke-tests it:

- the packaged CLI runs and reports the pinned version
- the container is the non-root node user (1000) and owns /workspace + DSH_HOME
- the bubblewrap command-sandbox runner is present
- the OCI version label matches versions.env
- a container serves /favicon.svg (the same unauthenticated probe the shipped
  manifests use), with logs filtered so the launch token is never printed

Path-filtered like the other on-demand checks, so it is not a required status
check; the published multi-arch build stays in build-main.yaml and release.yaml.

Docs updated in the same change: AGENTS.md repository map and testing table,
docs/gitflow.md checks section, docs/main-image.md build section.

Verified: actionlint 1.7.12 and yamllint 1.38.0 (CI versions) clean over the
whole repository, plus make check and make lint.

---------

Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
2026-10-30 08:36:21 -03:00

10 KiB

The main DSH Web image

This repository's primary artifact is a container image that packages DeepSeek Harness Web (dsh web) for Kubernetes and plain Docker. The Dockerfile at the repository root builds it; every version pin it uses lives in versions.env.

The image is image-first: it is the product, and the examples under examples/ are self-contained implementations that consume it.

What the build does

The Dockerfile is a two-stage build:

  1. Build stage (node:24-trixie) clones DSH at the pinned git ref, installs dependencies with pnpm, runs the production build, packages the @deepseek-ai/dsh CLI with pnpm pack, and installs its production dependencies with npm.
  2. Runtime stage (node:24-trixie-slim) copies /opt/dsh to /app, adds the CLI tools DSH agents commonly use (git, ripgrep, jq, yq, curl, wget, gh, tini, uv), installs bubblewrap as DSH's preferred command-sandbox runner, and starts dsh web through a small entrypoint.

Build it

make image          # canonical: builds the DSH Web image, no push

make image reads versions.env and passes the pins as build args. Pull requests that touch Dockerfile, .dockerignore or versions.env get the same build for linux/amd64 from the ci-image.yaml workflow — no registry login, no push — followed by a container smoke test. To reproduce the equivalent by hand:

docker build \
  --build-arg DSH_VERSION=dsh-v0.2.1-rc.5 \
  --build-arg RELEASE_REVISION= \
  --build-arg PNPM_VERSION=11.28.5 \
  --build-arg VCS_REF="$(git rev-parse --short HEAD)" \
  -t ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.5 .

Build args

Arg Default / source Purpose
DSH_REPO https://github.com/deepseek-ai/deepseek-harness.git Upstream DSH repository to clone.
DSH_VERSION DSH_VERSION in versions.env Upstream DSH git tag/ref to package; recorded as org.opencontainers.image.version.
RELEASE_REVISION RELEASE_REVISION in versions.env Repository revision suffix (rN); the release identity is DSH_VERSION-RELEASE_REVISION, recorded in the io.sredevopsorg.release.revision label. See versioning.
PNPM_VERSION PNPM_VERSION in versions.env pnpm version activated with corepack in both stages.
VCS_REF git rev-parse --short HEAD (via make image) Source revision recorded for traceability.
NODE_BUILD_IMAGE docker.io/node:24-trixie Build-stage base image.
NODE_RUNTIME_IMAGE docker.io/node:24-trixie-slim Runtime-stage base image.

Never hard-code a version outside versions.env: ./scripts/check-versions.sh (and make check) fails when a pin drifts.

Runtime contract

Entrypoint

The entrypoint is tini -- /usr/local/bin/dsh-entrypoint:

  1. tini runs as PID 1 and reaps children/forwards signals.
  2. The shell entrypoint applies defaults, derives a bind address when DSH_HOST is empty (the pod/container's first non-loopback IPv4), builds the dsh web command line, and execs it so signals reach the Node process.
  3. localhost and 127.0.0.1 are always added to DSH's trusted-host list; each DSH_TRUSTED_HOSTS entry is added as an extra --trusted-host.

Environment variables

Variable Default Purpose
DSH_PORT 3080 Port DSH Web listens on inside the container.
DSH_HOST auto-detected non-loopback IPv4 Bind address. Leave empty in Kubernetes so the pod IP is used automatically.
DSH_PUBLIC_URL http://localhost:${DSH_PORT} Public base URL used to build links and the launch-token URL printed in the logs. Set it when the container is published on a different host port or behind a proxy.
DSH_TRUSTED_HOSTS empty Extra Host/Origin allow-list entries, comma- or space-separated. localhost/127.0.0.1 are always trusted. Required when a proxy forwards the browser's Host.
DEEPSEEK_API_KEY empty Provider credential read by DSH. Inject it with -e/envFrom; never bake it into the image. Only model calls fail when it is missing.
DSH_HOME /home/node/.dsh DSH state directory (conversations, sessions, config, credentials). Mount a volume here to persist it.
DSH_PERMISSION_MODE workspace-write Deployment default for DSH's file policy: read-only, workspace-write, or danger-full-access (which also turns approval prompts off). The confined modes need bubblewrap or Landlock inside the container; where neither is usable every bash command fails closed with SANDBOX_UNAVAILABLE. See the note below.

The image also sets NODE_ENV=production, HOME=/home/node, and a PATH that includes /app/node_modules/.bin.

DSH confines bash commands itself (a second layer inside whatever container sandbox you already run). Its Linux runners are bubblewrap and then Landlock, and it fails closed rather than running unconfined. The image ships bubblewrap, but Debian's binary is unsetuid: it needs unprivileged user namespaces plus the namespace/mount syscalls. A hardened pod — non-root, capabilities.drop: ["ALL"], seccompProfile: RuntimeDefault — usually cannot create one, and it may expose no Landlock runner either, in which case every command fails with SANDBOX_UNAVAILABLE. Set DSH_PERMISSION_MODE=danger-full-access only when the surrounding pod or VM is the isolation boundary you rely on; keep the confined modes when the host provides a working runner (check grep -o landlock /sys/kernel/security/lsm and bwrap --ro-bind / / true inside the container first).

Provider credentials

DSH resolves provider API keys from the environment when no stored credential exists. The packaged provider catalog maps each provider to its standard environment name. Commonly used names:

DEEPSEEK_API_KEY, OPENROUTER_API_KEY, GEMINI_API_KEY, NVIDIA_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, GROQ_API_KEY, XAI_API_KEY, MISTRAL_API_KEY, TOGETHER_API_KEY, CEREBRAS_API_KEY, AI_GATEWAY_API_KEY, HF_TOKEN.

Use GEMINI_API_KEY for Google Gemini (not GOOGLE_API_KEY), HF_TOKEN for Hugging Face, and AI_GATEWAY_API_KEY for the Vercel AI Gateway. Inject keys with -e/--env-file in Docker or a Secret + envFrom in Kubernetes; never bake them into the image.

A key saved through DSH's credential store ($DSH_HOME/.credentials.yaml) wins over the environment. The agents-as-a-service example documents the optional dsh-provider-keys-extra Secret and the pod-refresh procedure.

Ports, volumes and signals

Item Value Notes
Exposed port 3080/tcp EXPOSE 3080; one port serves the UI, assets, API, SSE and WebSocket traffic.
State volume /home/node/.dsh Declared with VOLUME; use a named volume (Docker) or PVC (Kubernetes).
Working directory /workspace The agent workspace. Mount a volume here if agents should keep files; the agents-as-a-service example mounts the same PVC's workspace subPath here.
Stop signal SIGTERM tini forwards it to dsh web; allow a graceful shutdown period.
User node, UID/GID 1000 The process is non-root. A PVC mounted at DSH_HOME must be writable by UID 1000.

Health probes

/ is not a health endpoint: DSH returns 401 until a browser completes the launch-token handshake. Probe /favicon.svg instead — it is unauthenticated and served by the web layer, so it proves the HTTP server is up without triggering auth.

startupProbe:
  httpGet: {path: /favicon.svg, port: 3080}
  periodSeconds: 5
  failureThreshold: 24
readinessProbe:
  httpGet: {path: /favicon.svg, port: 3080}
  periodSeconds: 5
livenessProbe:
  httpGet: {path: /favicon.svg, port: 3080}
  periodSeconds: 15

The shipped templates use exactly this shape; see examples/minimal-single-tenant/manifests/sandbox-template.yaml. A startupProbe is important because plugin activation (or the first start of a large sandbox) can exceed a default liveness budget.

Launch token

DSH prints its one-time launch URL to the logs:

dsh web: http://<host>:${DSH_PORT}/?token=...

Open that URL once to obtain DSH's auth cookie. After that the UI works at the bare URL; without the cookie / answers 401 by design. The agents-as-a-service gateway performs this handshake server-side so browsers never see the token.

Published image names and tags

Registry layout: the main image is the repository itself, and every example component is a nested package below it:

ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox                        # main DSH Web image
ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway   # example component

The main image is published multi-arch (linux/amd64, linux/arm64) to GitHub Container Registry:

Tag Meaning
dsh-v0.2.1-rc.5 Current release identity. Immutable: once published it never moves.
latest Most recent release; points at the same digest as the current identity.
main Development channel rebuilt from every push to main; not a release, not for production.

Tags published before the release-identity scheme remain published but are not updated; new deployments use the current identity.

Release identities, channels and rollback rules are defined in versioning. Tags are immutable, so production deployments should pin a digest:

docker buildx imagetools inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.5
# ... pin the digest in your deployment manifests:
#   image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox@sha256:<digest>

Verifying an image

docker inspect ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.5 \
  --format '{{json .Config.Labels}}'
# org.opencontainers.image.title / .version / .source

Release builds also publish SBOM and provenance attestations from the release workflow (see SECURITY.md and gitflow).