mirror of
https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git
synced 2026-11-03 01:13:24 +00:00
feat(example): add optional gVisor, nodeSelector, NetworkPolicy and provider keys (#21)
Add opt-in hardening to the agents-as-a-service example: - commented runtimeClassName/nodeSelector fields for sandboxes and Deployments - optional sandbox NetworkPolicy plus exposure-variant policies - optional dsh-provider-keys-extra Secret for extra provider keys - docs/optional-hardening.md with verification, caveats and rollback The base walkthrough stays portable; no gateway code, routes or auth changed. Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
This commit is contained in:
committed by
GitHub
parent
bf24306421
commit
3e9ba38fc1
@@ -16,6 +16,19 @@ Applying both gives the gateway two entry points, and the SSO variant is only
|
||||
safe when the gateway is reachable **exclusively** through oauth2-proxy. Do not
|
||||
mix them.
|
||||
|
||||
Each variant also ships a NetworkPolicy, applied by the same directory apply:
|
||||
|
||||
- `ingress/61-networkpolicy.yaml` allows only Traefik pods to reach the gateway
|
||||
on `8080`.
|
||||
- `gateway-api-sso/75-networkpolicies.yaml` allows only oauth2-proxy to reach
|
||||
the gateway, only the Gateway implementation (Traefik) to reach oauth2-proxy,
|
||||
and limits oauth2-proxy egress to DNS, public HTTPS (GitHub) and the gateway.
|
||||
|
||||
The policies assume Traefik's `app.kubernetes.io/name: traefik` pod label.
|
||||
Adjust the selector for another implementation or a `hostNetwork` install; see
|
||||
[optional hardening](../../docs/optional-hardening.md) for the probe caveat and
|
||||
verification commands.
|
||||
|
||||
## Applying a variant
|
||||
|
||||
Run these commands from the example root
|
||||
@@ -62,6 +75,9 @@ Secrets imperatively (commands below) before applying the directory.
|
||||
`Secure` cookie is never sent over HTTP, which silently breaks login.
|
||||
4. Do not attach buffering or compression middlewares to this router: they
|
||||
break WebSocket upgrades and streaming.
|
||||
5. `61-networkpolicy.yaml` is part of this variant: it allows only Traefik pods
|
||||
to reach the gateway on `8080`. Adjust its selector for another ingress
|
||||
controller or `hostNetwork`; if kubelet probes fail, see the hardening doc.
|
||||
|
||||
Verify: `curl -sI https://<host>/healthz` returns `200`.
|
||||
|
||||
@@ -70,6 +86,7 @@ Verify: `curl -sI https://<host>/healthz` returns `200`.
|
||||
`70-gateway.yaml` defines the TLS `Gateway`, `71-httproute.yaml` sends every
|
||||
request to oauth2-proxy, and `72-oauth2-proxy.yaml` authenticates the browser
|
||||
against GitHub and proxies to the gateway with `X-Forwarded-User` set.
|
||||
`75-networkpolicies.yaml` makes this the only network path to the gateway.
|
||||
|
||||
Prerequisites:
|
||||
|
||||
@@ -111,7 +128,7 @@ kubectl apply -f manifests/exposure/gateway-api-sso/
|
||||
|
||||
`AUTH_PROXY_HEADER` is opt-in and deliberately unset in the base gateway
|
||||
manifest: the gateway ignores identity headers until it is set. Enabling SSO
|
||||
takes **both** sides:
|
||||
takes all of these:
|
||||
|
||||
1. `AUTH_PROXY_HEADER=X-Forwarded-User` on `deploy/dsh-gateway` (step 3
|
||||
above). Without it, oauth2-proxy's login has no effect on the gateway and
|
||||
@@ -122,6 +139,10 @@ takes **both** sides:
|
||||
other path to the Service) while SSO is enabled. A client that can reach
|
||||
`dsh-gateway:8080` directly can set `X-Forwarded-User` itself and
|
||||
impersonate any user.
|
||||
3. Keep `75-networkpolicies.yaml` with the variant. It enforces the boundary in
|
||||
the network layer (gateway ← oauth2-proxy ← Traefik, and limited oauth2-proxy
|
||||
egress). A custom exposure without an equivalent policy has no network
|
||||
enforcement.
|
||||
|
||||
Also terminate TLS in front of the gateway (the Gateway listener does) and
|
||||
leave `COOKIE_SECURE=true` so the session cookie never travels in clear text.
|
||||
|
||||
@@ -16,6 +16,12 @@
|
||||
# apply ../ingress/ (or anything else that reaches the gateway). A
|
||||
# client that can reach dsh-gateway:8080 can spoof X-Forwarded-User and
|
||||
# impersonate any user.
|
||||
#
|
||||
# 75-networkpolicies.yaml enforces that boundary at the network layer: the
|
||||
# gateway accepts traffic only from oauth2-proxy pods, oauth2-proxy only from
|
||||
# the Gateway implementation, and oauth2-proxy egress is limited to DNS,
|
||||
# public HTTPS and the gateway. Apply it with this variant.
|
||||
#
|
||||
# TLS terminates at the Gateway listener (70-gateway.yaml); keep
|
||||
# --cookie-secure=true and COOKIE_SECURE=true on the gateway.
|
||||
#
|
||||
@@ -42,6 +48,11 @@ spec:
|
||||
app: oauth2-proxy
|
||||
app.kubernetes.io/part-of: dsh-demo
|
||||
spec:
|
||||
# OPTIONAL: run the proxy under gVisor and/or pin it to a node pool.
|
||||
# See ../../../docs/optional-hardening.md.
|
||||
# runtimeClassName: gvisor
|
||||
# nodeSelector:
|
||||
# dsh.example.com/node-pool: dsh
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 2000
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# OPTIONAL with the SSO exposure variant: enforce the trust boundary at the
|
||||
# network layer.
|
||||
#
|
||||
# The chain is Traefik (Gateway API) -> oauth2-proxy -> dsh-gateway. These
|
||||
# policies make it the ONLY chain:
|
||||
# - the gateway accepts traffic only from oauth2-proxy pods;
|
||||
# - oauth2-proxy accepts traffic only from the Gateway implementation;
|
||||
# - oauth2-proxy egress is limited to DNS, public HTTPS (GitHub) and the
|
||||
# gateway.
|
||||
# The gateway's own egress (Kubernetes API, sandbox pods) is intentionally not
|
||||
# restricted here: the API-server endpoint is cluster-specific. See
|
||||
# ../../../docs/optional-hardening.md for that recipe.
|
||||
#
|
||||
# This manifest assumes Traefik, matching gatewayClassName: traefik in
|
||||
# 70-gateway.yaml. Adjust the app.kubernetes.io/name selector for another
|
||||
# implementation. NetworkPolicy enforcement of kubelet probes is CNI-dependent;
|
||||
# if the gateway or proxy fails its probes, allow the node CIDR on the probe
|
||||
# port or remove the ingress policy.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: dsh-gateway-ingress
|
||||
namespace: dsh-demo
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: dsh-gateway
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app: oauth2-proxy
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8080
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: oauth2-proxy-ingress
|
||||
namespace: dsh-demo
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: oauth2-proxy
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector: {}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: traefik
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 4180
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: oauth2-proxy-egress
|
||||
namespace: dsh-demo
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: oauth2-proxy
|
||||
policyTypes:
|
||||
- Egress
|
||||
egress:
|
||||
# Cluster DNS.
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
podSelector:
|
||||
matchLabels:
|
||||
k8s-app: kube-dns
|
||||
ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
# NodeLocal DNSCache.
|
||||
- to:
|
||||
- ipBlock:
|
||||
cidr: 169.254.20.10/32
|
||||
ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
# The gateway backend.
|
||||
- to:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app: dsh-gateway
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8080
|
||||
# GitHub OAuth endpoints are public HTTPS. Private ranges and the metadata
|
||||
# server stay blocked.
|
||||
- to:
|
||||
- ipBlock:
|
||||
cidr: 0.0.0.0/0
|
||||
except:
|
||||
- 10.0.0.0/8
|
||||
- 172.16.0.0/12
|
||||
- 192.168.0.0/16
|
||||
- 169.254.0.0/16
|
||||
- 100.64.0.0/10
|
||||
- ipBlock:
|
||||
cidr: "::/0"
|
||||
except:
|
||||
- "fc00::/7"
|
||||
- "fe80::/10"
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 443
|
||||
@@ -23,6 +23,10 @@
|
||||
# middlewares (e.g. compress, buffering) to this router — they break streaming
|
||||
# and upgrades. `path: /` Prefix covers the landing page and every per-user
|
||||
# proxy path, so no per-sandbox Ingress is needed.
|
||||
#
|
||||
# 61-networkpolicy.yaml is part of this variant: it restricts the gateway's
|
||||
# ingress to Traefik pods. Adjust its selector if your ingress controller uses
|
||||
# a different label or hostNetwork.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# OPTIONAL with the Ingress exposure variant: restrict the gateway's ingress
|
||||
# to the ingress controller.
|
||||
#
|
||||
# This manifest assumes Traefik, matching this variant's
|
||||
# `ingressClassName: traefik`. Traefik's Helm chart labels its pods
|
||||
# app.kubernetes.io/name=traefik; adjust the selector for another controller or
|
||||
# for a hostNetwork install (where the source is the node IP, not a pod IP).
|
||||
#
|
||||
# NetworkPolicy enforcement of kubelet probe traffic is CNI-dependent. If the
|
||||
# gateway starts failing its /healthz probes, allow the node CIDR on port 8080
|
||||
# or remove this policy. See ../../../docs/optional-hardening.md.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: dsh-gateway-ingress
|
||||
namespace: dsh-demo
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: dsh-gateway
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector: {}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: traefik
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8080
|
||||
Reference in New Issue
Block a user