Skip to content

Security: JoshWheeler08/local-llm-gateway

Security

docs/security.md

Security Guide

Overview

Security features:

  • Service mesh integration (mTLS)
  • Non-root containers
  • Network policies (optional)
  • RBAC configurations
  • No external API keys stored

Service Mesh (Recommended)

Istio

serviceMesh:
  istio: true

Provides:

  • ✅ Automatic mTLS
  • ✅ Traffic encryption
  • ✅ Zero-trust networking

Linkerd

serviceMesh:
  linkerd: true

Provides:

  • ✅ Automatic mTLS
  • ✅ Lightweight
  • ✅ Simple setup

Pod Security

Security Context

gateway:
  podSecurityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 1000
    seccompProfile:
      type: RuntimeDefault

Container Security

gateway:
  securityContext:
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: true
    runAsNonRoot: true
    capabilities:
      drop:
        - ALL

Network Policies

Enable Network Policies

networkPolicies:
  enabled: true

Restricts:

  • Only gateway → runners traffic
  • Only ingress → gateway traffic
  • Deny all other traffic

Example Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: local-llm-gateway
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: local-llm-gateway
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
  egress:
    - to:
        - podSelector:
            matchLabels:
              local-llm-gateway.io/runner: llama3-8b

RBAC

Minimal Permissions

Gateway only needs:

  • Read ConfigMaps
  • Read Secrets (if used)
rbac:
  create: true

serviceAccount:
  create: true
  name: local-llm-gateway

Secrets Management

Environment Variables

gateway:
  env:
    - name: API_KEY
      valueFrom:
        secretKeyRef:
          name: local-llm-gateway-secrets
          key: api-key

External Secrets Operator

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: local-llm-gateway-secrets
spec:
  secretStoreRef:
    name: aws-secrets-manager
  target:
    name: local-llm-gateway-secrets
  data:
    - secretKey: api-key
      remoteRef:
        key: local-llm-gateway/api-key

Best Practices

✅ Do

  • Use service mesh for mTLS
  • Run as non-root user
  • Enable network policies
  • Use RBAC
  • Regular security updates (Dependabot)
  • Scan images with Trivy

❌ Don't

  • Store API keys in ConfigMaps
  • Run as root
  • Allow privilege escalation
  • Expose metrics publicly
  • Use latest image tag

Compliance

Pod Security Standards

Restricted profile:

podSecurityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

Image Scanning

GitHub Actions runs Trivy scan:

  • On every PR
  • On every release
  • Results in Security tab

Audit Logging

Enable service mesh audit logs:

Istio:

meshConfig:
  accessLogFile: /dev/stdout

Linkerd:

proxy:
  logLevel: info

Updates

  • Enable Dependabot (automatic PRs)
  • Monitor security advisories
  • Update regularly
# Check for updates
helm search repo local-llm-gateway

# Upgrade
helm upgrade local-llm-gateway oci://ghcr.io/.../charts/local-llm-gateway --version 0.2.0

There aren't any published security advisories