mirror of
https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git
synced 2026-11-03 09:23:25 +00:00
a09d4ac340
* fix(example): keep agents usable when DSH's inner sandbox has no backend DSH confines bash commands itself (bubblewrap, else Landlock) and fails closed with SANDBOX_UNAVAILABLE when no runner is usable. In this hardened pod - non-root, capabilities dropped, seccomp RuntimeDefault - bubblewrap cannot start (it needs CAP_SYS_ADMIN) and the pod exposes no Landlock runner, so the shipped workspace-write default made every command fail: sandbox mode "workspace-write" is requested but no sandbox backend is usable The DSH base bundle at the pinned ref documents DSH_PERMISSION_MODE as the explicit deployment override, so set it to danger-full-access in both sandbox templates. The per-user pod is the isolation boundary in this example (its own pod, PVC and Service; non-root; all capabilities dropped), so a working agent is worth more than an inner file policy that cannot be enforced. Note that danger-full-access also turns approval prompts off. Trade-off documented rather than hidden: an agent command can now read its own DSH_HOME (including .credentials.yaml) and write anywhere in the container. Hosts where a runner works should keep workspace-write or read-only. - main-image docs: DSH_PERMISSION_MODE row plus a note on the runners and the fail-closed behavior - example README: limits bullet and a troubleshooting row with the in-pod backend check and how to restore a confined mode - architecture: tenant-isolation bullet extended - skill runbook/troubleshooting: operation guidance; never grant CAP_SYS_ADMIN or seccompProfile: Unconfined to "fix" this without an explicit decision Verified: make check (14 manifests), make lint, YAML assertions on both templates, and the override expression at the pinned dsh-v0.2.1-rc.5 base bundle (mode: !!js process.env.DSH_PERMISSION_MODE ?? 'workspace-write'). * fix(image): install bubblewrap for DSH's command sandbox DSH's Linux sandbox runner chain is bubblewrap, then Landlock, and it fails closed with SANDBOX_UNAVAILABLE when neither works. The image shipped neither, so a sandboxed agent had no way to execute confined commands. Add bubblewrap to the runtime stage. Debian ships it unsetuid (0.12.0), so it still needs unprivileged user namespaces plus the namespace/mount syscalls: in a hardened pod (all capabilities dropped, RuntimeDefault seccomp) it usually cannot create one, and DSH then falls back to Landlock or, failing that, to the DSH_PERMISSION_MODE=danger-full-access override the templates already set. Where the host allows an unprivileged user namespace, the first-choice runner now exists, so the confined modes can be restored. Docs corrected to match: bubblewrap is no longer "absent"; the blocker is the unsetuid binary's namespace requirement, not a missing CAP_SYS_ADMIN grant. Verified: make check, make lint; the package exists in the pinned base (node:24-trixie-slim, bubblewrap 0.12.0-1~deb13u1); a throwaway container with the pod's flags (--user 1000 --cap-drop ALL --security-opt no-new-privileges) still cannot create a namespace, which is why the override stays. No local image build was run. * ci(image): build and smoke-test the main image on PRs that touch it The Dockerfile and the pins in versions.env could only be validated by a merge to main or by running `make image` by hand; neither gives a PR-time signal. Add ci-image.yaml: on pull requests that change Dockerfile, .dockerignore or versions.env (and on manual dispatch) it builds linux/amd64 natively with the pins sourced from versions.env - no registry login, no push - loads the image locally and smoke-tests it: - the packaged CLI runs and reports the pinned version - the container is the non-root node user (1000) and owns /workspace + DSH_HOME - the bubblewrap command-sandbox runner is present - the OCI version label matches versions.env - a container serves /favicon.svg (the same unauthenticated probe the shipped manifests use), with logs filtered so the launch token is never printed Path-filtered like the other on-demand checks, so it is not a required status check; the published multi-arch build stays in build-main.yaml and release.yaml. Docs updated in the same change: AGENTS.md repository map and testing table, docs/gitflow.md checks section, docs/main-image.md build section. Verified: actionlint 1.7.12 and yamllint 1.38.0 (CI versions) clean over the whole repository, plus make check and make lint. --------- Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
132 lines
5.9 KiB
YAML
132 lines
5.9 KiB
YAML
# Minimal single-tenant SandboxTemplate for DeepSeek Harness Web.
|
|
#
|
|
# This is the "just one DSH Web" quick-start template. For the full
|
|
# multi-tenant platform (warm pool, per-user claims, gateway, idle-suspend),
|
|
# see examples/dsh-agents-as-a-service/.
|
|
#
|
|
# Claims reference this template directly (spec.sandboxTemplateRef): there is
|
|
# no warm pool and no per-user platform here. See the README for the full
|
|
# quickstart; in short the prerequisites are:
|
|
# - Namespace dsh-demo.
|
|
# - Secret dsh-provider-keys in dsh-demo with key DEEPSEEK_API_KEY
|
|
# (injected via envFrom; the pod starts without it, only model calls fail).
|
|
# - Optional: Secret dsh-provider-keys-extra with additional provider keys
|
|
# (OPENROUTER_API_KEY, GEMINI_API_KEY, NVIDIA_API_KEY, ...). It is
|
|
# referenced with `optional: true`, so it may be absent.
|
|
# The agent-sandbox controller (v1.0.8, API v1beta1) must be installed.
|
|
apiVersion: extensions.agents.x-k8s.io/v1beta1
|
|
kind: SandboxTemplate
|
|
metadata:
|
|
name: dsh-web-template
|
|
namespace: dsh-demo
|
|
spec:
|
|
service: true # headless Service per Sandbox -> stable DNS (status.serviceFQDN)
|
|
# networkPolicyManagement is omitted, so the controller uses its Managed
|
|
# secure default (ingress only from the sandbox-router, public egress). This
|
|
# quickstart reaches the sandbox through `kubectl port-forward`, which is
|
|
# local to the pod. The multi-tenant example documents opt-in policies in
|
|
# examples/dsh-agents-as-a-service/docs/optional-hardening.md.
|
|
#
|
|
# OPTIONAL: recreate a Failed sandbox Pod instead of retaining it
|
|
# (agent-sandbox v1.0.8+). Documented with the multi-tenant example.
|
|
# failedPodPolicy: Recreate
|
|
podTemplate:
|
|
spec:
|
|
# OPTIONAL: run under gVisor. gVisor breaks `kubectl port-forward`, so
|
|
# this quickstart's only access path will not work; use the multi-tenant
|
|
# gateway example or a Service/NodePort instead.
|
|
# runtimeClassName: gvisor
|
|
#
|
|
# OPTIONAL: pin to a dedicated node pool. Replace the placeholder with a
|
|
# node label; add a matching toleration if the pool is tainted.
|
|
# nodeSelector:
|
|
# dsh.example.com/node-pool: dsh
|
|
securityContext:
|
|
runAsNonRoot: true
|
|
runAsUser: 1000 # Matches the UID of the 'node' user
|
|
runAsGroup: 1000
|
|
fsGroup: 1000
|
|
# DSH's credentials plugin refuses to start when .credentials.yaml has
|
|
# any group/other permission bit (it creates the file 0600). The
|
|
# default kubelet fsGroup policy ("Always") rewrites it to 0660 on every
|
|
# pod start, so a recreated or resumed sandbox crash-loops; OnRootMismatch
|
|
# avoids that walk once the volume root already carries the ownership.
|
|
# Rationale in full at
|
|
# examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml.
|
|
fsGroupChangePolicy: OnRootMismatch
|
|
containers:
|
|
- name: dsh-web
|
|
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.5
|
|
imagePullPolicy: IfNotPresent
|
|
ports:
|
|
- name: web
|
|
containerPort: 3080
|
|
envFrom:
|
|
- secretRef:
|
|
name: dsh-provider-keys # Secret with key DEEPSEEK_API_KEY
|
|
# Optional extra provider keys; an absent Secret is safe.
|
|
- secretRef:
|
|
name: dsh-provider-keys-extra
|
|
optional: true
|
|
env:
|
|
# Empty => DSH derives links from the request Host. Keep it empty
|
|
# when fronting the sandbox with a proxy/gateway.
|
|
- name: DSH_PUBLIC_URL
|
|
value: ""
|
|
# DSH Host/Origin-fences /api. Empty trusts only localhost, which
|
|
# is right for `kubectl port-forward`. Set this to the public host
|
|
# (e.g. dsh.example.com) if you front DSH with a proxy or Ingress,
|
|
# so requests forwarded with the browser's Host are accepted. The
|
|
# full example documents both exposure variants in
|
|
# examples/dsh-agents-as-a-service/manifests/exposure/README.md.
|
|
- name: DSH_TRUSTED_HOSTS
|
|
value: ""
|
|
# DSH's own bash file sandbox (bubblewrap, else Landlock) cannot be
|
|
# enforced in this hardened pod - bubblewrap is installed but is
|
|
# unsetuid and needs an unprivileged user namespace - and fails
|
|
# closed with SANDBOX_UNAVAILABLE. The pod is the boundary here;
|
|
# keep the deployment override in sync with the multi-tenant example
|
|
# and see its README troubleshooting row to restore a confined mode.
|
|
- name: DSH_PERMISSION_MODE
|
|
value: danger-full-access
|
|
volumeMounts:
|
|
- name: dsh-home
|
|
mountPath: /home/node/.dsh # DSH_HOME
|
|
# / is behind DSH's launch-token cookie (401 without it); use an
|
|
# unauthenticated path for probes. The startup probe gives a cold
|
|
# start (plugin activation on a small node, or the first resume of a
|
|
# suspended sandbox) up to 120s before liveness can restart it.
|
|
startupProbe:
|
|
httpGet:
|
|
path: /favicon.svg
|
|
port: 3080
|
|
periodSeconds: 5
|
|
failureThreshold: 24
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /favicon.svg
|
|
port: 3080
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 5
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /favicon.svg
|
|
port: 3080
|
|
initialDelaySeconds: 15
|
|
periodSeconds: 15
|
|
securityContext:
|
|
allowPrivilegeEscalation: false
|
|
capabilities:
|
|
drop: ["ALL"]
|
|
seccompProfile:
|
|
type: RuntimeDefault
|
|
volumeClaimTemplates:
|
|
- metadata:
|
|
name: dsh-home
|
|
spec:
|
|
accessModes: ["ReadWriteOnce"]
|
|
resources:
|
|
requests:
|
|
storage: 5Gi
|
|
# storageClassName omitted => cluster default
|