Skip to content

Clusters

A LanguageCluster is the top-level organizational unit in Language Operator. It creates a managed Kubernetes namespace and deploys the shared infrastructure that all agents, models, and tools in that namespace depend on: a LiteLLM gateway, NetworkPolicies, and optional external ingress.

Namespace Mapping

Each LanguageCluster maps 1:1 to a namespace. The namespace is created and owned by the cluster resource — deleting the LanguageCluster deletes the namespace and everything in it.

apiVersion: langop.io/v1alpha1
kind: LanguageCluster
metadata:
  name: production-agents   # → namespace: production-agents
spec: {}

All namespace-scoped resources (LanguageAgent, LanguageModel, LanguageTool, LanguagePersona) are deployed into this namespace.

Shared Gateway

Every cluster runs exactly one LiteLLM proxy. It aggregates all LanguageModel resources in the namespace into a single OpenAI-compatible endpoint. Agents connect to it via the MODEL_ENDPOINT environment variable:

MODEL_ENDPOINT=http://gateway.<namespace>.svc.cluster.local:8000

Credentials never leave the gateway pod. Agents send model names and prompts; the gateway holds the API keys and routes to the correct upstream provider.

When the model list changes, the gateway restarts with the updated configuration. No agent redeploy is required.

Network Isolation

By default, agents in a cluster can communicate with each other and with the shared gateway, but not with arbitrary external hosts. Additional ingress and egress rules are configured on the cluster and applied as built-in rules to every agent's NetworkPolicy.

Allow HTTPS egress to upstream APIs:

spec:
  networkPolicies:
    egress:
      - to:
          - cidr: "0.0.0.0/0"
        ports:
          - port: 443

Allow egress by DNS name (requires Cilium):

spec:
  networkPolicies:
    egress:
      - to:
          - dns:
              - "api.anthropic.com"
              - "api.openai.com"
        ports:
          - port: 443

Restrict ingress to agents in a named group:

spec:
  networkPolicies:
    ingress:
      - from:
          - group: data-pipeline
        ports:
          - port: 8080

See LanguageCluster API Reference for the full NetworkPeer field reference.

Capacity Limits

Enforce hard limits on how many resources can be created in the namespace:

spec:
  capacity:
    maxAgents: 10
    maxModels: 5
    maxCPU: "8"
    maxMemory: 16Gi

The operator creates a ResourceQuota named langop-quota in the namespace. Current usage is reflected in status.capacity.

Gateway Customization

spec:
  gateway:
    deployment:
      replicas: 2
      resources:
        requests:
          cpu: 100m
          memory: 256Mi
        limits:
          cpu: 500m
          memory: 512Mi

All standard Kubernetes deployment fields are supported: nodeSelector, affinity, tolerations, topologySpreadConstraints, env, envFrom, volumes, and volumeMounts.