# Before You Install Read this page before you install Monad on your own Kubernetes cluster. It covers the operators the chart depends on, the permissions the installing account needs, the choices you cannot change after the first install, and the Secrets that pods wait for. Once everything here is in place, continue with [Install](/docs/guides/self-managed/install). :::note[Merge snippets into your values file] Each values snippet gives full paths from the top-level key. The chart's example files already have some of these keys, such as `nats` in every example, `postgresql` and `victoriametrics` in `private-cluster.yaml`, and `api` and `ui` in `local-auth-external-postgres.yaml`. Add a snippet's settings under the key you already have: a second copy of a top-level key silently replaces the first. See [Install](/docs/guides/self-managed/install#build-your-values-file). ::: ## Cluster - Monad supports the Kubernetes minor versions that the Kubernetes project currently maintains, listed on [Releases](https://kubernetes.io/releases/). - Size the cluster with [Cluster Sizing](/docs/guides/cluster-sizing). - Nodes pull Monad's images from Docker Hub with the credentials Monad support gives you. If your cluster cannot reach public registries, see [Images and Registries](/docs/guides/self-managed/images). - The chart's volumes use the cluster's default StorageClass unless you name one. See [Storage](#storage). - Clients reach Monad through your Gateway or ingress controller, which needs an address they can resolve. [Networking and TLS](/docs/guides/self-managed/networking) covers the listeners and certificates. ## Operators and controllers Install these before Monad. The chart creates their resources, so `helm install` fails while their CRDs are missing, and the resources do nothing until the operator runs. | Component | Needed | Version | What Monad uses it for | | --- | --- | --- | --- | | [CloudNativePG](https://cloudnative-pg.io/) | Unless you bring your own Postgres (`postgresql.cnpg.enabled: false`, see [Database and Cache](/docs/guides/self-managed/database)) | 1.25 or later. With the chart's backups on, 1.30 or earlier | Runs Monad's Postgres as a `Cluster`. Local authentication on this Postgres also creates a `Database`, a resource added in 1.25. The chart's backups use CloudNativePG's native Barman Cloud support, which CloudNativePG has announced it removes in 1.31; see [Backup and Restore](/docs/guides/self-managed/backup-and-restore). | | [VictoriaMetrics operator](https://docs.victoriametrics.com/operator/) | Yes | v0.75.2 or later | Runs the `VMCluster` and `VMAlert` that hold Monad's pipeline metrics. It does not replace your monitoring stack; every component also exposes metrics for scraping, see [Metrics & Monitoring](/docs/guides/metrics). | | A [Gateway API](https://gateway-api.sigs.k8s.io/) implementation, such as Envoy Gateway, Istio, kgateway or Traefik | Yes, or an ingress controller | Gateway API CRDs that serve `HTTPRoute` and `TCPRoute`: the standard channel from v1.6, or the experimental channel before it | Routes web, API and ingest traffic (`HTTPRoute`) and syslog (`TCPRoute`). An implementation without `TCPRoute` can carry syslog over a `TLSRoute` instead; see [Syslog through a TLSRoute](/docs/guides/self-managed/networking#syslog-through-a-tlsroute). | | An ingress controller | Instead of Gateway API | Any | Publishes the HTTP paths only. Syslog and OTLP ingest need another way in; see [What an Ingress cannot carry](/docs/guides/self-managed/networking#what-an-ingress-cannot-carry). | | [cert-manager](https://cert-manager.io/) | Optional | Any | Issues the TLS certificates Monad needs. The chart creates no certificates itself. | | [External Secrets Operator](https://external-secrets.io/) | Optional | Any | Syncs the [required Secrets](#required-secrets) from your secrets manager. | | Prometheus Operator CRDs | Optional | Any | Only for `postgresql.cnpg.monitoring.enabled` and `victoriametrics.monitoring.enabled`, which render a `PodMonitor` and `ServiceMonitor`. The render fails if either is on and its CRD is missing. | ## Cluster permissions The account that runs `helm install` needs: - **Cluster-wide**: permission to create the chart's three CustomResourceDefinitions (`organizations`, `pipelines` and `pipelinenodes` in `pipeline.monad.com`) and two PriorityClasses. - **In the release namespace**: permission to create every resource the chart renders, Roles and RoleBindings included. Kubernetes lets an account grant only permissions it already holds, so the account also needs every permission those Roles grant, such as managing Deployments, Secrets, Jobs and the `pipeline.monad.com` resources. ### Without permission to create PriorityClasses The chart creates one PriorityClass for application pods and a higher one for NATS, Postgres and vmstorage, so that application pods cannot preempt the datastores. To use classes you already have instead, set `global.monad.priorityClasses.create: false` and name your classes. [Priority classes](/docs/guides/self-managed/scheduling#priority-classes) has the settings and lists which workloads are in each tier. ### Without permission to create CRDs Set `crds.create: false` and have a cluster administrator apply the CRDs from the same chart version, before the first install and again before every upgrade. The chart is private, so they first log in with the token from Monad support, as in [Log in to the registry](/docs/guides/self-managed/install#log-in-to-the-registry): ```bash helm registry login registry-1.docker.io --username monadinc helm template monad oci://registry-1.docker.io/monadinc/monad --version \ --show-only templates/crds.yaml | kubectl apply -f - ``` ## Choose an authentication provider Choose the provider now: it sets which keys go in the `api` and `ui` Secrets below, and whether you also provide a Secret for Ory Kratos. | Provider | Where its settings go | | --- | --- | | Auth0 (the default) | Keys in the `api` and `ui` Secrets | | Amazon Cognito | Keys in the `api` and `ui` Secrets | | Local authentication with Ory Kratos | Your values file. Kratos keeps its data in Postgres: the chart's, or your own if you also provide the Kratos Secret yourself. Installing with Argo CD or another tool that renders with `helm template` also needs the Kratos Secret provided yourself. | [Authentication](/docs/guides/self-managed/authentication) covers setting up each provider and lists its keys. ## Storage The chart creates three sets of PersistentVolumeClaims. Choose their StorageClasses before the first install: a volume keeps the class it was created with. With no class set, a volume uses the cluster's default StorageClass, and stays `Pending` if the cluster has none. | Volume | One per | Size key (default) | StorageClass key | Can it grow after install? | | --- | --- | --- | --- | --- | | NATS JetStream | NATS server (`nats.config.cluster.replicas`, 3) | `nats.config.jetstream.fileStore.pvc.size` (`10Gi`) | `nats.config.jetstream.fileStore.pvc.storageClassName` | Not through Helm: the size is fixed at the first install. Growing it is a manual procedure; see [Grow the JetStream volume](/docs/guides/self-managed/nats#grow-the-jetstream-volume). | | Postgres | CloudNativePG instance (`postgresql.cnpg.instances`, 3) | `postgresql.cnpg.storage.size` (`5Gi`) | `postgresql.cnpg.storage.storageClass` | Yes, where the StorageClass allows volume expansion. It cannot shrink. | | vmstorage | vmstorage node (`victoriametrics.vmstorage.replicaCount`, 2) | `victoriametrics.vmstorage.storage.volumeClaimTemplate.spec.resources.requests.storage` (`2Gi`) | `victoriametrics.vmstorage.storage.volumeClaimTemplate.spec.storageClassName` | Yes: the VictoriaMetrics operator resizes the volumes where the StorageClass allows expansion. | The NATS volume buffers pipeline data, so size it for your load now; [Size the JetStream volume before you install](/docs/guides/self-managed/nats#size-the-jetstream-volume-before-you-install) suggests sizes. Every example values file already sets `nats.config.jetstream.fileStore.pvc.size`, so change it there and add `storageClassName` beside it. `private-cluster.yaml` also has `postgresql` and `victoriametrics` keys; add the storage settings under them. With all three set, the keys look like this: ```yaml nats: config: jetstream: fileStore: pvc: size: 100Gi storageClassName: postgresql: cnpg: storage: size: 20Gi storageClass: victoriametrics: vmstorage: storage: volumeClaimTemplate: spec: storageClassName: resources: requests: storage: 10Gi ``` ## Namespace This guide installs into `monad`; any namespace works. Install one Monad release per namespace, because the chart names its components' resources, such as the `api` Deployment, without the release name. ```bash kubectl create namespace monad ``` The three CRDs are cluster-wide, and the first release in the cluster owns them. Every later release, in another namespace, sets `crds.create: false`: Helm refuses to install CRDs that another release owns. The CRDs then follow the first release: - Upgrade the first release before the others, so the CRDs are at least as new as every release that uses them. - Uninstalling the first release leaves the CRDs in place, so the other releases keep their organizations and pipelines. From then on no release upgrades the CRDs: apply them as in [Without permission to create CRDs](#without-permission-to-create-crds). See [Uninstall](/docs/guides/self-managed/install#uninstall). ## Required Secrets Create these in the release namespace before you install. A pod that needs a missing Secret waits for it and starts once it exists. | Secret | Read by | Until it exists | | --- | --- | --- | | `monad-license` | api, api-grpc | Pods start, but the API refuses every request with `License cannot be verified` | | `monad-master-encryption-key` | api, api-grpc, eventhandler, operator | Pods stay in `ContainerCreating` with a `FailedMount` event for `key-encryption-key` | | `default-pull-secret` | When you pull Monad's images from Docker Hub, as `minimal.yaml` and `local-auth-external-postgres.yaml` do: every pod that pulls a Monad image, the pipeline pods included. See [Image pull secret](#image-pull-secret). | Image pulls fail with `ErrImagePull` | | `api` | api, api-grpc, eventhandler, http-input, l4-input | Pods stay in `CreateContainerConfigError` | | `ui` | ui | The pod stays in `CreateContainerConfigError` | | `http-input-tls` | l4-input | The pod exits at startup and restarts until the Secret holds a valid key pair | The sections below cover each one. To sync them from your secrets manager instead, see [Sync Secrets with External Secrets Operator](#sync-secrets-with-external-secrets-operator). ### License The Monad license is a certificate, `license.crt`, which Monad gives you. The key must be named `license.crt`: ```bash kubectl create secret generic monad-license -n monad \ --from-file=license.crt=/path/to/license.crt ``` ### Key encryption key :::danger[You are the sole custodian of the key encryption key] The key encryption key (KEK) wraps every organization's data encryption key (DEK). Losing a KEK version makes everything wrapped under it permanently unreadable, and Monad cannot recover it. Back up every version in durable storage outside the cluster, never delete or overwrite a version once it has been used, and treat a leaked KEK as a critical security incident. ::: Each Monad organization has its own DEK, wrapped by the active KEK before it is stored. Unwrapping needs the same KEK version that wrapped it, so the Secret keeps every version that has ever been used. The KEK never leaves your cluster. The Secret is named `monad-master-encryption-key`. Each key is a version number (`1`, `2`, `3`, …) holding a base64-encoded 32-byte random key, and the highest number is the active version. Keys that are not whole numbers are ignored. When a new version appears, Monad re-wraps every organization's DEK with it; no manual re-encryption is needed. In production, keep the KEK in the secrets manager you already run, such as HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, Azure Key Vault or 1Password, so it is backed up, audited and rotated with your other secrets. Store one item, such as `monad-key-encryption-key`, with one field per version, and sync it with [External Secrets Operator](#sync-secrets-with-external-secrets-operator). Without a secrets manager, for example on an air-gapped cluster, create the Secret directly. Generate the key and back it up somewhere durable first: once it exists only in the cluster, the cluster is its only copy. ```bash kubectl create secret generic monad-master-encryption-key -n monad \ --from-literal=1="$(openssl rand 32 | base64)" ``` To rotate, add the next version number with a new key, keeping every existing version, either with `kubectl patch` or by recreating the Secret with all versions. After you install, confirm the pods that mount the KEK are running. A pod stuck in `ContainerCreating` with a `FailedMount` event for `key-encryption-key` means the Secret is missing or in another namespace. ```bash kubectl get pods -n monad \ -l 'app.kubernetes.io/component in (api,api-grpc,eventhandler,operator)' ``` ### Image pull secret Monad support gives you a token for Monad's images on Docker Hub. Store it as `default-pull-secret`, which the example values files that pull from Docker Hub list under `global.imagePullSecrets`. The pods the operator creates for your pipelines use the same list, and a Secret named `default-pull-secret` when the list is empty; [Pull secrets](/docs/guides/self-managed/images#pull-secrets) lists what every pod gets. ```bash kubectl create secret docker-registry default-pull-secret -n monad \ --docker-server=registry-1.docker.io \ --docker-username=monadinc \ --docker-password= ``` To pull from your own registry instead, see [Images and Registries](/docs/guides/self-managed/images). ### The api and ui Secrets The api, api-grpc, eventhandler, http-input and l4-input load the `api` Secret as environment variables, and the ui loads the `ui` Secret. Both are required, even with local authentication, where the `api` Secret can be empty. Keep credentials in these Secrets rather than in a component's `env` values, which anyone who can read the Pod spec can see. Every other component also loads a Secret named after itself, such as `operator`, if one exists; none is required. Every install needs one key: | Key | Secret | Value | | --- | --- | --- | | `AUTH_SECRET` | `ui` | A random value the ui signs session cookies with | Your provider adds more keys to both Secrets; [Authentication](/docs/guides/self-managed/authentication) lists them. To connect the Google Workspace or Figma inputs through OAuth, also add your OAuth app's `GOOGLE_OAUTH_CLIENT_ID` and `GOOGLE_OAUTH_CLIENT_SECRET`, or `FIGMA_OAUTH_CLIENT_ID` and `FIGMA_OAUTH_CLIENT_SECRET`, to the `api` Secret, which runs the sign-in flow. Input pods read the same keys from a Secret named `inputs` if it exists, and the Figma input needs them there to refresh its token. To create the Secrets with `kubectl`: 1. Start the two env files. This writes a random `AUTH_SECRET` into `ui.env` and an empty `api.env`: ```bash printf 'AUTH_SECRET=%s\n' "$(openssl rand -base64 32)" > ui.env touch api.env ``` 2. Add your provider's keys to both files, one `KEY=value` per line. Write nothing else on a line: kubectl takes everything after the first `=` as the value, trailing spaces and any `#` comment included, and it rejects a file that repeats a key. With local authentication, `api.env` stays empty. 3. Create the Secrets: ```bash kubectl create secret generic api -n monad --from-env-file=api.env kubectl create secret generic ui -n monad --from-env-file=ui.env ``` ### Syslog certificate `l4-input` serves syslog and other TLS-over-TCP ingest with the certificate in the `kubernetes.io/tls` Secret `http-input-tls`, and runs on every install, so create the Secret even if you never publish syslog. [Syslog certificate](/docs/guides/self-managed/networking#syslog-certificate) lists the names it must cover and gives a self-signed recipe for installs that do not publish syslog. ### Sync Secrets with External Secrets Operator The chart can render an ExternalSecret for the KEK and for the `api` and `ui` Secrets. This needs a `SecretStore` or `ClusterSecretStore` of your own: the chart's default store name does not exist in your cluster. Turn each ExternalSecret on under its component, not at the top level, because a top-level `externalsecret` block is inherited by every component. ```yaml externalsecret: secretStoreRef: kind: ClusterSecretStore name: api: externalsecrets: monad-master-encryption-key: enabled: true dataFrom: - extract: key: monad-key-encryption-key externalsecret: enabled: true data: # One entry per key your provider needs in the api Secret - secretKey: MONAD_AUTH_TYPE remoteRef: key: property: MONAD_AUTH_TYPE ui: externalsecret: enabled: true data: - secretKey: AUTH_SECRET remoteRef: key: property: AUTH_SECRET # One entry per key your provider needs in the ui Secret ``` - **KEK**: `extract` copies every field of the item into the Secret, so adding a version in your secrets manager needs no change here. To read it from another store, set `api.externalsecrets.monad-master-encryption-key.secretStoreRef`. - **`api` and `ui`**: each `data` entry maps a key of the Secret (`secretKey`) to a property in your store (`remoteRef`). A `data` list you set replaces the chart's default list, so list every key, `AUTH_SECRET` included. With local authentication the `api` Secret has no keys, so create it with `kubectl` instead: the chart renders no ExternalSecret without keys. - **License and pull secret**: the chart renders no ExternalSecret for these. Write your own if you keep them in your secrets manager. ### Optional Secrets Some setups need more Secrets. Each is covered on its own page. | Secret | Needed when | See | | --- | --- | --- | | `monad-nats-creds` | The message bus requires authentication. It has to be on from the first install. | [NATS authentication](/docs/guides/self-managed/nats#nats-authentication) | | `monad-kratos-secrets` | Local authentication with your own Postgres, or installed with a tool that renders with `helm template` | [Provide the Kratos Secret yourself](/docs/guides/self-managed/authentication#provide-the-kratos-secret-yourself) | | `monad-kratos-smtp` | Local authentication sends email codes | [Email](/docs/guides/self-managed/authentication#email) | | `monad-db-app` | You bring your own Postgres. With the chart's Postgres, CloudNativePG creates it. | [Create the database Secret](/docs/guides/self-managed/database#create-the-database-secret) | | `inputs` | Input pods need OAuth app credentials, as described above | [The api and ui Secrets](#the-api-and-ui-secrets) |