Before You Install
Read this page before you install Monad on your own Kubernetes cluster. It covers the operators the chart depends on, the permissions the installing account needs, the choices you cannot change after the first install, and the Secrets that pods wait for. Once everything here is in place, continue with Install.
Merge snippets into your values file
Each values snippet gives full paths from the top-level key. The chart's example files already have some of these keys, such as nats in every example, postgresql and victoriametrics in private-cluster.yaml, and api and ui in local-auth-external-postgres.yaml. Add a snippet's settings under the key you already have: a second copy of a top-level key silently replaces the first. See Install.
Cluster
- Monad supports the Kubernetes minor versions that the Kubernetes project currently maintains, listed on Releases.
- Size the cluster with Cluster Sizing.
- Nodes pull Monad's images from Docker Hub with the credentials Monad support gives you. If your cluster cannot reach public registries, see Images and Registries.
- The chart's volumes use the cluster's default StorageClass unless you name one. See Storage.
- Clients reach Monad through your Gateway or ingress controller, which needs an address they can resolve. Networking and TLS covers the listeners and certificates.
Operators and controllers
Install these before Monad. The chart creates their resources, so helm install fails while their CRDs are missing, and the resources do nothing until the operator runs.
| Component | Needed | Version | What Monad uses it for |
|---|---|---|---|
| CloudNativePG | Unless you bring your own Postgres (postgresql.cnpg.enabled: false, see Database and Cache) | 1.25 or later. With the chart's backups on, 1.30 or earlier | Runs Monad's Postgres as a Cluster. Local authentication on this Postgres also creates a Database, a resource added in 1.25. The chart's backups use CloudNativePG's native Barman Cloud support, which CloudNativePG has announced it removes in 1.31; see Backup and Restore. |
| VictoriaMetrics operator | Yes | v0.75.2 or later | Runs the VMCluster and VMAlert that hold Monad's pipeline metrics. It does not replace your monitoring stack; every component also exposes metrics for scraping, see Metrics & Monitoring. |
| A Gateway API implementation, such as Envoy Gateway, Istio, kgateway or Traefik | Yes, or an ingress controller | Gateway API CRDs that serve HTTPRoute and TCPRoute: the standard channel from v1.6, or the experimental channel before it | Routes web, API and ingest traffic (HTTPRoute) and syslog (TCPRoute). An implementation without TCPRoute can carry syslog over a TLSRoute instead; see Syslog through a TLSRoute. |
| An ingress controller | Instead of Gateway API | Any | Publishes the HTTP paths only. Syslog and OTLP ingest need another way in; see What an Ingress cannot carry. |
| cert-manager | Optional | Any | Issues the TLS certificates Monad needs. The chart creates no certificates itself. |
| External Secrets Operator | Optional | Any | Syncs the required Secrets from your secrets manager. |
| Prometheus Operator CRDs | Optional | Any | Only for postgresql.cnpg.monitoring.enabled and victoriametrics.monitoring.enabled, which render a PodMonitor and ServiceMonitor. The render fails if either is on and its CRD is missing. |
Cluster permissions
The account that runs helm install needs:
- Cluster-wide: permission to create the chart's three CustomResourceDefinitions (
organizations,pipelinesandpipelinenodesinpipeline.monad.com) and two PriorityClasses. - In the release namespace: permission to create every resource the chart renders, Roles and RoleBindings included. Kubernetes lets an account grant only permissions it already holds, so the account also needs every permission those Roles grant, such as managing Deployments, Secrets, Jobs and the
pipeline.monad.comresources.
Without permission to create PriorityClasses
The chart creates one PriorityClass for application pods and a higher one for NATS, Postgres and vmstorage, so that application pods cannot preempt the datastores. To use classes you already have instead, set global.monad.priorityClasses.create: false and name your classes. Priority classes has the settings and lists which workloads are in each tier.
Without permission to create CRDs
Set crds.create: false and have a cluster administrator apply the CRDs from the same chart version, before the first install and again before every upgrade. The chart is private, so they first log in with the token from Monad support, as in Log in to the registry:
Code
Choose an authentication provider
Choose the provider now: it sets which keys go in the api and ui Secrets below, and whether you also provide a Secret for Ory Kratos.
| Provider | Where its settings go |
|---|---|
| Auth0 (the default) | Keys in the api and ui Secrets |
| Amazon Cognito | Keys in the api and ui Secrets |
| Local authentication with Ory Kratos | Your values file. Kratos keeps its data in Postgres: the chart's, or your own if you also provide the Kratos Secret yourself. Installing with Argo CD or another tool that renders with helm template also needs the Kratos Secret provided yourself. |
Authentication covers setting up each provider and lists its keys.
Storage
The chart creates three sets of PersistentVolumeClaims. Choose their StorageClasses before the first install: a volume keeps the class it was created with. With no class set, a volume uses the cluster's default StorageClass, and stays Pending if the cluster has none.
| Volume | One per | Size key (default) | StorageClass key | Can it grow after install? |
|---|---|---|---|---|
| NATS JetStream | NATS server (nats.config.cluster.replicas, 3) | nats.config.jetstream.fileStore.pvc.size (10Gi) | nats.config.jetstream.fileStore.pvc.storageClassName | Not through Helm: the size is fixed at the first install. Growing it is a manual procedure; see Grow the JetStream volume. |
| Postgres | CloudNativePG instance (postgresql.cnpg.instances, 3) | postgresql.cnpg.storage.size (5Gi) | postgresql.cnpg.storage.storageClass | Yes, where the StorageClass allows volume expansion. It cannot shrink. |
| vmstorage | vmstorage node (victoriametrics.vmstorage.replicaCount, 2) | victoriametrics.vmstorage.storage.volumeClaimTemplate.spec.resources.requests.storage (2Gi) | victoriametrics.vmstorage.storage.volumeClaimTemplate.spec.storageClassName | Yes: the VictoriaMetrics operator resizes the volumes where the StorageClass allows expansion. |
The NATS volume buffers pipeline data, so size it for your load now; Size the JetStream volume before you install suggests sizes. Every example values file already sets nats.config.jetstream.fileStore.pvc.size, so change it there and add storageClassName beside it. private-cluster.yaml also has postgresql and victoriametrics keys; add the storage settings under them. With all three set, the keys look like this:
Code
Namespace
This guide installs into monad; any namespace works. Install one Monad release per namespace, because the chart names its components' resources, such as the api Deployment, without the release name.
Code
The three CRDs are cluster-wide, and the first release in the cluster owns them. Every later release, in another namespace, sets crds.create: false: Helm refuses to install CRDs that another release owns. The CRDs then follow the first release:
- Upgrade the first release before the others, so the CRDs are at least as new as every release that uses them.
- Uninstalling the first release leaves the CRDs in place, so the other releases keep their organizations and pipelines. From then on no release upgrades the CRDs: apply them as in Without permission to create CRDs. See Uninstall.
Required Secrets
Create these in the release namespace before you install. A pod that needs a missing Secret waits for it and starts once it exists.
| Secret | Read by | Until it exists |
|---|---|---|
monad-license | api, api-grpc | Pods start, but the API refuses every request with License cannot be verified |
monad-master-encryption-key | api, api-grpc, eventhandler, operator | Pods stay in ContainerCreating with a FailedMount event for key-encryption-key |
default-pull-secret | When you pull Monad's images from Docker Hub, as minimal.yaml and local-auth-external-postgres.yaml do: every pod that pulls a Monad image, the pipeline pods included. See Image pull secret. | Image pulls fail with ErrImagePull |
api | api, api-grpc, eventhandler, http-input, l4-input | Pods stay in CreateContainerConfigError |
ui | ui | The pod stays in CreateContainerConfigError |
http-input-tls | l4-input | The pod exits at startup and restarts until the Secret holds a valid key pair |
The sections below cover each one. To sync them from your secrets manager instead, see Sync Secrets with External Secrets Operator.
License
The Monad license is a certificate, license.crt, which Monad gives you. The key must be named license.crt:
Code
Key encryption key
You are the sole custodian of the key encryption key
The key encryption key (KEK) wraps every organization's data encryption key (DEK). Losing a KEK version makes everything wrapped under it permanently unreadable, and Monad cannot recover it. Back up every version in durable storage outside the cluster, never delete or overwrite a version once it has been used, and treat a leaked KEK as a critical security incident.
Each Monad organization has its own DEK, wrapped by the active KEK before it is stored. Unwrapping needs the same KEK version that wrapped it, so the Secret keeps every version that has ever been used. The KEK never leaves your cluster.
The Secret is named monad-master-encryption-key. Each key is a version number (1, 2, 3, …) holding a base64-encoded 32-byte random key, and the highest number is the active version. Keys that are not whole numbers are ignored. When a new version appears, Monad re-wraps every organization's DEK with it; no manual re-encryption is needed.
In production, keep the KEK in the secrets manager you already run, such as HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, Azure Key Vault or 1Password, so it is backed up, audited and rotated with your other secrets. Store one item, such as monad-key-encryption-key, with one field per version, and sync it with External Secrets Operator.
Without a secrets manager, for example on an air-gapped cluster, create the Secret directly. Generate the key and back it up somewhere durable first: once it exists only in the cluster, the cluster is its only copy.
Code
To rotate, add the next version number with a new key, keeping every existing version, either with kubectl patch or by recreating the Secret with all versions.
After you install, confirm the pods that mount the KEK are running. A pod stuck in ContainerCreating with a FailedMount event for key-encryption-key means the Secret is missing or in another namespace.
Code
Image pull secret
Monad support gives you a token for Monad's images on Docker Hub. Store it as default-pull-secret, which the example values files that pull from Docker Hub list under global.imagePullSecrets. The pods the operator creates for your pipelines use the same list, and a Secret named default-pull-secret when the list is empty; Pull secrets lists what every pod gets.
Code
To pull from your own registry instead, see Images and Registries.
The api and ui Secrets
The api, api-grpc, eventhandler, http-input and l4-input load the api Secret as environment variables, and the ui loads the ui Secret. Both are required, even with local authentication, where the api Secret can be empty. Keep credentials in these Secrets rather than in a component's env values, which anyone who can read the Pod spec can see. Every other component also loads a Secret named after itself, such as operator, if one exists; none is required.
Every install needs one key:
| Key | Secret | Value |
|---|---|---|
AUTH_SECRET | ui | A random value the ui signs session cookies with |
Your provider adds more keys to both Secrets; Authentication lists them. To connect the Google Workspace or Figma inputs through OAuth, also add your OAuth app's GOOGLE_OAUTH_CLIENT_ID and GOOGLE_OAUTH_CLIENT_SECRET, or FIGMA_OAUTH_CLIENT_ID and FIGMA_OAUTH_CLIENT_SECRET, to the api Secret, which runs the sign-in flow. Input pods read the same keys from a Secret named inputs if it exists, and the Figma input needs them there to refresh its token.
To create the Secrets with kubectl:
-
Start the two env files. This writes a random
AUTH_SECRETintoui.envand an emptyapi.env:Code -
Add your provider's keys to both files, one
KEY=valueper line. Write nothing else on a line: kubectl takes everything after the first=as the value, trailing spaces and any#comment included, and it rejects a file that repeats a key. With local authentication,api.envstays empty. -
Create the Secrets:
Code
Syslog certificate
l4-input serves syslog and other TLS-over-TCP ingest with the certificate in the kubernetes.io/tls Secret http-input-tls, and runs on every install, so create the Secret even if you never publish syslog. Syslog certificate lists the names it must cover and gives a self-signed recipe for installs that do not publish syslog.
Sync Secrets with External Secrets Operator
The chart can render an ExternalSecret for the KEK and for the api and ui Secrets. This needs a SecretStore or ClusterSecretStore of your own: the chart's default store name does not exist in your cluster. Turn each ExternalSecret on under its component, not at the top level, because a top-level externalsecret block is inherited by every component.
Code
- KEK:
extractcopies every field of the item into the Secret, so adding a version in your secrets manager needs no change here. To read it from another store, setapi.externalsecrets.monad-master-encryption-key.secretStoreRef. apiandui: eachdataentry maps a key of the Secret (secretKey) to a property in your store (remoteRef). Adatalist you set replaces the chart's default list, so list every key,AUTH_SECRETincluded. With local authentication theapiSecret has no keys, so create it withkubectlinstead: the chart renders no ExternalSecret without keys.- License and pull secret: the chart renders no ExternalSecret for these. Write your own if you keep them in your secrets manager.
Optional Secrets
Some setups need more Secrets. Each is covered on its own page.
| Secret | Needed when | See |
|---|---|---|
monad-nats-creds | The message bus requires authentication. It has to be on from the first install. | NATS authentication |
monad-kratos-secrets | Local authentication with your own Postgres, or installed with a tool that renders with helm template | Provide the Kratos Secret yourself |
monad-kratos-smtp | Local authentication sends email codes | |
monad-db-app | You bring your own Postgres. With the chart's Postgres, CloudNativePG creates it. | Create the database Secret |
inputs | Input pods need OAuth app credentials, as described above | The api and ui Secrets |