# Installation With Helm ## Overview This guide documents the process for installing Monad into a Kubernetes cluster using Helm. The primary installation method uses the Kubernetes Gateway API for ingress. If your cluster uses a traditional ingress controller instead, see [Alternative: Ingress Controller](#alternative-ingress-controller) at the end of this document. --- ## Phase 1: Prerequisites These must be installed before the Monad Helm chart can be deployed. ### Authentication Provider Monad supports local authentication, Auth0, and AWS Cognito, with Auth0 being the default. ### Required Operators/Controllers 1. **Victoria Metrics Operator** - Required for internal metrics and dashboards, deployed by Custom Resources from within the chart. This does not replace an observability stack like Prometheus, and all components expose endpoints for scraping by an observability platform. - Installation: [https://docs.victoriametrics.com/operator/](https://docs.victoriametrics.com/operator/) 2. **CloudNativePG (CNPG)** - PostgreSQL operator - Required unless you're using an [external PostgreSQL instance](#external-postgres) - Installation: [https://cloudnative-pg.io/documentation/current/installation_upgrade/](https://cloudnative-pg.io/documentation/current/installation_upgrade/) ### Gateway or Ingress Controller - Monad recommends using the Kubernetes Gateway API for ingress and will create HTTPRoute and TCPRoute resources automatically. You need a Gateway API implementation installed in your cluster (e.g., Traefik, Istio, Envoy Gateway, kgateway, or any other conformant implementation). - Your implementation must support the experimental Gateway API channel, which includes `TCPRoute`. - Installation varies by implementation. Refer to your implementation's documentation. - This guide assumes the use of Gateway API, though alternatives for using an ingress controller are provided at the [end of the document](#alternative-ingress-controller). ### Create Namespace ```bash kubectl create namespace monad ``` --- ## Phase 2: Create Required Secrets These secrets must exist before installing the Helm chart. All secrets are created in the `monad` namespace. ### 1. License Secret The Monad license is a TLS certificate. You should have received a `license.crt` file from Monad. ```bash kubectl create secret generic monad-license \ --from-file=license.crt=/path/to/license.crt \ -n monad ``` ### 2. Encryption Key Secret Used for encrypting sensitive data within Monad. This is a base64-encoded random 32-byte key. ```bash # Generate the encryption key ENCRYPTION_KEY=$(dd if=/dev/urandom bs=1 count=32 2>/dev/null | base64) # Create the secret kubectl create secret generic secret \ --from-literal=MONAD_ENCRYPTION_KEY="${ENCRYPTION_KEY}" \ -n monad ``` ### 3. Key Encryption Key (KEK) :::danger The Key Encryption Key (KEK) is the master key that wraps every organization's Data Encryption Key (DEK). **All historical KEK versions must be preserved** — older versions stay in the Secret because previously-wrapped DEKs still need them to decrypt. **Losing the KEK is unrecoverable**: Monad has no way to recover wrapped DEKs without it, and every byte of data wrapped under a lost version becomes permanently inaccessible. You are the sole custodian. ::: Each Monad organization is issued its own DEK, which is wrapped by the active KEK before being stored. Decryption requires the same KEK version that performed the wrap, so the Secret holds every version that has ever been used. The KEK never leaves the cluster. The Helm chart consumes the KEK from a Kubernetes Secret named `monad-master-encryption-key` in the `monad` namespace. Each data field is a numeric version label (`1`, `2`, `3`, …) holding a base64-encoded 32-byte random key. The highest numeric field is the active version; new versions are appended on rotation, and old versions are never deleted or overwritten. When a new version appears, Monad automatically re-wraps every organization's DEK to the new active KEK — no manual re-encryption step is required. #### Recommended: sync from your secrets manager In production, the KEK should originate in the secrets manager you already operate — HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, 1Password, etc. — and be reconciled into the cluster via External Secrets Operator (ESO), so that backups, audit, and rotation flow through the same controls as the rest of your secrets. Store an item in your secrets manager (e.g. named `monad-key-encryption-key`) with one numeric field per version, each value a base64-encoded 32-byte random key (`openssl rand 32 | base64` or any equivalent CSPRNG). Then enable the chart's ExternalSecret in your `values-override.yaml`: ```yaml api: externalsecrets: monad-master-encryption-key: enabled: true # Optional: point at a specific ClusterSecretStore. Falls back to the # global externalsecret.secretStoreRef if omitted. # secretStoreRef: # name: my-secret-store # kind: ClusterSecretStore dataFrom: - extract: key: monad-key-encryption-key ``` The `extract` directive syncs every field of the upstream item into the K8s Secret, so adding a new numeric version upstream needs no chart or values change. :::warning You are the sole custodian of the KEK. Back up every version in your secrets manager, and back up that secrets manager. Never delete or overwrite a numeric field once it has been used to wrap data. Treat KEK exfiltration as a P0 security incident. ::: #### Without an external secret store For air-gapped clusters or environments without ESO (or an equivalent bridge like SealedSecrets / SOPS), you can create the Secret directly with `kubectl`. Generate and back up the key material in something durable **before** applying it — once it lives only in the cluster, K8s becomes the source of truth, which is fragile. ```bash kubectl create secret generic monad-master-encryption-key \ -n monad \ --from-literal=1="$(openssl rand 32 | base64)" ``` Rotations are performed by `kubectl patch`-ing in a new numeric field, or by re-creating the Secret with all existing versions plus the new one. #### Verify After install, confirm the `api` and `operator` pods see the mount: ```bash kubectl exec -n monad deploy/monad-api -- ls /secrets/key-encryption-key ``` ### 4. Image Pull Secret You should have received credentials from Monad support for accessing the images in Docker Hub. Configure those here as `default-pull-secret` (required name). ```bash kubectl create secret docker-registry default-pull-secret \ --docker-server=registry-1.docker.io \ --docker-username=monadinc \ --docker-password= \ -n monad ``` ### 5. Authentication Secrets #### Using External Secrets Operator If you're using an external secrets store, you'll need to either save your secrets with the property names found in `values.yaml` or update the `values.yaml` to match your secret property names. ```yaml externalsecret: enabled: true data: - secretKey: AUTH0_DOMAIN remoteRef: key: env-secrets property: MONAD_AUTH_AUTH0_API_AUDIENCE # <-- update these to match your secret property names - secretKey: AUTH0_API_AUDIENCE remoteRef: key: env-secrets property: MONAD_AUTH_AUTH0_API_AUDIENCE ... ``` #### Not Using External Secrets Operator If you're not using an External Secrets operator, you need to create secrets for authentication backend credentials. Below is an example file of key/value environment variable pairs that you can create a secret from. **Note:** Some of these variables exist in two forms. Monad is migrating from variable names like `AUTH0_CLIENT_ID` to `MONAD_AUTH_AUTH0_CLIENT_ID`. The `MONAD_`-prefixed variables are the new names, and the old names will be deprecated in a future release. For current and future compatibility, use both until informed that the old names can be removed. ##### api.env ```bash # Required AUTH_SECRET= MONAD_AUTH_SECRET= # Required for AUTH0 MONAD_AUTH_TYPE=auth0 MONAD_AUTH_AUTH0_CLIENT_ID= MONAD_AUTH_AUTH0_API_AUDIENCE= MONAD_AUTH_AUTH0_API_CLIENT_ID= MONAD_AUTH_AUTH0_CLIENT_SECRET= MONAD_AUTH_AUTH0_DOMAIN= MONAD_AUTH_AUTH0_ISSUER= MONAD_AUTH_AUTH0_MACHINE_CLIENT_SECRET= MONAD_AUTH_AUTH0_MACHINE_CLIENT_ID= MONAD_AUTH_AUTH0_MACHINE_AUDIENCE= # Required for Cognito MONAD_AUTH_TYPE=cognito MONAD_AUTH_COGNITO_AWS_REGION= MONAD_AUTH_COGNITO_USER_POOL_ID= MONAD_AUTH_COGNITO_CLIENT_ID= MONAD_AUTH_COGNITO_DOMAIN= MONAD_AUTH_COGNITO_SECRET= MONAD_AUTH_COGNITO_ISSUER_URL= # Optional GOOGLE_OAUTH_CLIENT_ID= GOOGLE_OAUTH_CLIENT_SECRET= ZOOM_OAUTH_CLIENT_ID= ZOOM_OAUTH_CLIENT_SECRET= ``` Create the Secret `api` from the file: ```bash kubectl create secret generic api \ --from-env-file=api.env \ -n monad ``` ##### ui.env ```bash # Required AUTH_SECRET= # Required for Auth0 MONAD_AUTH_TYPE=auth0 AUTH_AUTH0_DOMAIN= AUTH_AUTH0_CLIENT_ID= AUTH_AUTH0_API_AUDIENCE= AUTH_AUTH0_CLIENT_SECRET= AUTH_SECRET= # Required for Cognito MONAD_AUTH_TYPE=cognito AUTH_COGNITO_AWS_REGION= AUTH_COGNITO_USER_POOL_ID= AUTH_COGNITO_CLIENT_ID= AUTH_COGNITO_DOMAIN= AUTH_COGNITO_SECRET= AUTH_COGNITO_ISSUER_URL= ``` Create the secret `ui` from the file: ```bash kubectl create secret generic ui \ --from-env-file=ui.env \ -n monad ``` --- ## Phase 3: Configure TLS For the remainder of this guide we will be using `monad.example.com` as our example domain. Monad requires two TLS certificates: ### 1. Gateway Certificate Used by the Gateway to terminate HTTPS for all web traffic. This is a standard certificate for your Monad hostname (e.g., `monad.example.com`). The Secret must be created in the same namespace as your Gateway resource (not the `monad` namespace). This is a Gateway API requirement: certificate secrets must be co-located with the Gateway that references them. See [TLS with cert-manager](#tls-with-cert-manager) for instructions on creating this certificate automatically. ### 2. HTTP Input TLS Certificate Monad's HTTP Input workload handles inputs such as Syslog that require direct TLS termination at the Pod level. This certificate is separate from the Gateway certificate and must be named `http-input-tls` in the `monad` namespace. Syslog and other TCP-terminated inputs use the hostname with SNI for routing, with names like `cef85707-4b6e-405a-aea9-3237d520e805.l4.monad.example.com`. It should answer to both `l4.monad.example.com` and the wildcard domain `*.l4.monad.example.com`. You will also need a DNS record pointing `*.l4.monad.example.com` to the load balancer address that handles TCP traffic into your cluster. :::tip The full FQDN that you use for L4 traffic doesn't have to be connected to the hostname that you use for Monad itself (such as `*.l4.monad.example.com` and `monad.example.com`). As long as the FQDN you choose lands on the Gateway and is routed to the correct Service, the Pod that receives it performs a handshake, retrieves the requested FQDN from SNI, and then uses the hostname portion to route to the corresponding pipeline. ::: See [TLS with cert-manager](#tls-with-cert-manager) for instructions on creating this certificate automatically. If you want each pipeline to have its own ingest hostname, this certificate needs one more name on it. See [Per-Pipeline Ingest Host](#per-pipeline-ingest-host). ### Configure Your Gateway Your Gateway resource needs an HTTPS listener that references the Gateway certificate Secret. The exact configuration depends on your implementation, but the Gateway API spec looks like: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: namespace: spec: gatewayClassName: listeners: - name: web port: 80 protocol: HTTP allowedRoutes: namespaces: from: All - name: websecure port: 443 protocol: HTTPS allowedRoutes: namespaces: from: All tls: mode: Terminate certificateRefs: - name: monad-example-com-crt namespace: - name: otel-grpc port: 4317 protocol: HTTPS allowedRoutes: namespaces: from: All tls: mode: Terminate certificateRefs: - name: monad-example-com-crt namespace: - name: otel-https port: 4318 protocol: HTTPS allowedRoutes: namespaces: from: All tls: mode: Terminate certificateRefs: - name: monad-example-com-crt namespace: - name: tcp port: 6514 protocol: TCP allowedRoutes: namespaces: from: All ``` Refer to your Gateway implementation's documentation for how to configure listeners. Per-pipeline ingest hostnames need one additional HTTPS listener. See [Per-Pipeline Ingest Host](#per-pipeline-ingest-host). --- ## Phase 4: Configure values.yaml Overrides Create a `values-override.yaml` file with the following configurations. With Gateway API enabled, Monad's chart generates `HTTPRoute` resources automatically for all components. You do not need to configure ingress per-component — setting `hostnames` and `routing` at the top level is sufficient for all components to be reachable. ```yaml # The hostname(s) at which Monad will be accessible. # All HTTPRoutes, backend URLs, and the UI origin are derived from this. hostnames: - monad.example.com # Pull images from Docker Hub instead of GHCR image: repository: registry-1.docker.io/monadinc/ imagePullSecrets: - name: default-pull-secret # Enable Gateway API routing routing: enabled: true # Point all routes at your Gateway routes: default: parentRefs: - namespace: name: sectionName: websecure otel: parentRefs: - namespace: name: sectionName: otel-grpc - namespace: name: sectionName: otel-https tcp: parentRefs: - namespace: name: sectionName: tcp # Set this to your cluster's storageClassName (even if you have a default storage class, you have to override the setting in values.yaml) nats: config: jetstream: fileStore: pvc: storageClassName: operator: env: MONAD_PIPELINE_IMAGE_REGISTRY: value: registry-1.docker.io/monadinc/ ``` :::tip The `hostnames` value replaces what previously required per-component `BACKEND_URL` environment variables and per-component ingress host configuration. Setting it once here propagates to all components automatically. ::: --- ## Phase 5: Install Monad ### Log in to OCI Registry Before you can pull the Helm chart, authenticate to the Docker registry: ```bash helm registry login registry-1.docker.io --username monadinc Password: Login Succeeded ``` **Note:** Credentials are provided by Monad support. The login persists in `~/.docker/config.json`. ### Install with Custom Values ```bash helm upgrade monad oci://registry-1.docker.io/monadinc/monad \ --install \ --namespace monad \ --values values-override.yaml \ --timeout 10m ``` :::tip You can perform upgrades of Monad using the same command above. Simply remove the `--install` directive to perform an upgrade. ::: ### Verify Installation Some pods will initially come up in an `Error` state as they wait for the database to be ready. They should all be `Running` (or `Completed`) within a few minutes. ```bash # Check all pods are running kubectl get pods -n monad # Check services kubectl get svc -n monad # Check Gateway API routes (replaces kubectl get ingress) kubectl get httproute -n monad kubectl get certificate -n monad ``` --- ## Phase 6: Post-Installation At this point, Monad should be up and running. Access it at your designated hostname. For a map of the components now running in your cluster, how data flows between them, and where to look when troubleshooting, see [Platform Architecture](/docs/guides/architecture). --- ## Alternative Installation Options ### TLS with cert-manager If you're using cert-manager for certificate management, create a `ClusterIssuer` and two `Certificate` resources as described below. #### ClusterIssuer ```yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: your-email@example.com privateKeySecretRef: name: letsencrypt-prod solvers: - http01: ingress: class: traefik ``` #### Gateway Certificate Create this in your Gateway's namespace (e.g., `kube-system` for Traefik on k3s): ```yaml apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: monad-example-com-crt namespace: spec: secretName: monad-example-com-crt issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - monad.example.com ``` #### HTTP Input TLS Certificate Create this in the `monad` namespace: ```yaml apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: http-input-tls namespace: monad spec: secretName: http-input-tls issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - l4.monad.example.com - "*.l4.monad.example.com" ``` --- ### Per-Pipeline Ingest Host Monad can serve each pipeline its own ingest hostname, `.data.monad.example.com`. Senders POST to the root of that host with no path, and the hostname itself identifies the pipeline, so there is nothing per-sender to configure beyond the URL. Syslog clients use the same name on port 6514, where the pipeline ID is read from SNI. This is optional. Without it, HTTP senders post to `https://monad.example.com/api/v2/http/send/` and syslog clients connect to `.l4.monad.example.com:6514`. {/* Add a "requires Monad or later" line here once the release carrying host-based pipeline resolution ships. */} Setting it up takes a DNS record, a certificate change, a Gateway listener, and one values override. #### 1. DNS Point a wildcard record at the load balancer that fronts your Gateway: ``` *.data.monad.example.com -> ``` HTTP ingest (443) and syslog (6514) both use this name, so it has to resolve to a load balancer that carries both listeners. #### 2. Certificates The wildcard name has to appear on two certificates, because HTTP ingest and syslog terminate TLS in different places: - **Gateway certificate.** The Gateway terminates TLS for HTTP ingest, so add `*.data.monad.example.com` to the certificate your HTTPS listener references, or issue a separate certificate for the data listener. - **`http-input-tls`.** The HTTP Input Pod terminates TLS itself for syslog, and this is the certificate a syslog client validates, so it needs the same name. With cert-manager, that is one more entry in each `dnsNames` list: ```yaml dnsNames: - l4.monad.example.com - "*.l4.monad.example.com" - "*.data.monad.example.com" ``` :::tip A wildcard SAN cannot be issued through an HTTP-01 solver. If you use Let's Encrypt, the issuer for these certificates needs a DNS-01 solver. ::: #### 3. Gateway listener Add an HTTPS listener for the wildcard name to the Gateway you configured in [Phase 3](#configure-your-gateway): ```yaml - name: data hostname: "*.data.monad.example.com" port: 443 protocol: HTTPS allowedRoutes: namespaces: from: All tls: mode: Terminate certificateRefs: - name: monad-example-com-crt namespace: ``` The `tcp` listener on 6514 needs no change. A Gateway API TCP listener carries no hostname, so it already accepts syslog connections for any name that resolves to it. #### 4. values.yaml override Add a route on the `http-input` component that claims the wildcard hostname and sends everything under it to the ingest Service: ```yaml http-input: routes: data: enabled: true type: http pathType: PathPrefix hostnames: - "*.data.monad.example.com" parentRefs: - namespace: name: sectionName: data rules: - matches: - path: value: / ``` To accept OTLP on the same hostname, add a second route for the OTLP listeners. Both ports land on the ingest Service's single OTLP port: ```yaml http-input: routes: data-otel: enabled: true type: http pathType: PathPrefix hostnames: - "*.data.monad.example.com" parentRefs: - namespace: name: sectionName: otel-grpc - namespace: name: sectionName: otel-https rules: - matches: - path: type: PathPrefix value: / backendRefs: - kind: Service name: http-input port: 4317 ``` :::warning[Define this under http-input, not at the top level] A route defined at the top level of `values-override.yaml` is inherited by every component, so each one would claim `*.data.monad.example.com` and ingest requests could land on the UI or the API instead. A new route name also inherits nothing from `routes.default`, so `enabled`, `type`, `pathType`, `hostnames`, `parentRefs`, and `rules` all have to be set. Leaving out `type: http` renders no route at all and reports no error. ::: #### 5. Verify ```bash # The new route's name ends in -data. It should be Accepted by your Gateway. kubectl get httproute -n monad # A pipeline with an HTTP input should now accept data at the bare host curl -i --request POST \ --url https://.data.monad.example.com \ --header 'authorization: ApiKey ' \ --data '{"hello":"world"}' ``` A 200 response with `{"status":"success","count":1}` means the DNS record, certificate, listener, and route are all wired up. --- ### External Postgres If you want to use an external Postgres instance, you can disable the CloudNativePG operator and provide the connection details in the `monad-db-app` Secret. Below is an example file you can create a Secret from. It uses the following values: - `dbname`: monad - `user`: monad - `password`: somereallylongandcomplexpassword - `host`: monad-db-rw.postgres / monad-db-rw.postgres.svc.cluster.local (default CNPG service structure for a database in the `postgres` namespace) - `port`: 5432 ```bash # db.env dbname=monad fqdn-jdbc-uri=jdbc:postgresql://monad-db-rw.postgres.svc.cluster.local:5432/monad?password=somereallylongandcomplexpassword&user=monad fqdn-url=postgresql://monad:somereallylongandcomplexpassword@monad-db-rw.postgres.svc.cluster.local:5432/monad host=monad-db-rw.postgres jdbc-uri=jdbc:postgresql://monad-db-rw.postgres:5432/monad?password=somereallylongandcomplexpassword&user=monad password=somereallylongandcomplexpassword pgpass=monad-db-rw.postgres:5432:monad:monad:somereallylongandcomplexpassword port=5432 uri=postgresql://monad:somereallylongandcomplexpassword@monad-db-rw.postgres:5432/monad user=monad username=monad ``` Create the Secret `monad-db-app` from the file: ```bash kubectl create secret generic monad-db-app \ --from-env-file=db.env \ -n monad ``` Disable the CloudNativePG operator in your `values-override.yaml`: ```yaml postgresql: cnpg: enabled: false ``` ### Local Authentication Local authentication creates an admin user of `admin@monad.local` and a random password. To activate this, remove the Auth0 and Cognito configuration from the `api` and `ui` secrets, and set `MONAD_AUTH_TYPE` to `local` for both components in your `values-override.yaml`: ```yaml ui: env: MONAD_AUTH_TYPE: value: local api: env: MONAD_AUTH_TYPE: value: local http-input: env: MONAD_AUTH_TYPE: value: local ``` After completing the installation, retrieve the Secret with the username and password: ```bash kubectl get secret app-bootstrap-admin -n monad \ -o go-template='{{range $k,$v := .data}}{{printf "%s: %s\n" $k ($v | base64decode)}}{{end}}' ``` This secret is only used to deliver credentials after installation and can be deleted after retrieval. Local authentication does not allow the creation of more users than the local admin. If you wish to have multiple users, log in as the admin user and set up SSO from the Settings menu. --- ### Ingress Controller If your cluster uses a traditional Kubernetes `Ingress` resource rather than Gateway API, use the following `values-override.yaml` instead of the one in Phase 4. All other phases remain the same, except: - The TLS certificate Secret should be created in the `monad` namespace instead of the Gateway namespace - Use `kubectl get ingress -n monad` instead of `kubectl get httproute -n monad` to verify routing Remove the `routing` and `routes` keys from the earlier example of `values-override.yaml` and add the following: ```yaml ingress: className: "" # e.g., traefik, nginx annotations: cert-manager.io/cluster-issuer: "letsencrypt-prod" tls: - secretName: monad-example-com-crt hosts: - monad.example.com api: ingress: enabled: true docs: ingress: enabled: true http-input: ingress: enabled: true ui: ingress: enabled: true ``` :::note The `hostnames` value at the top automatically configures backend URLs and the UI origin for all components. Per-component `ingress.hosts` configuration is handled by the chart defaults and does not need to be set explicitly. :::