## Kong for Kubernetes [Kong for Kubernetes](https://github.com/Kong/kubernetes-ingress-controller) is an open-source Ingress Controller for Kubernetes that offers API management capabilities with a plugin architecture. This chart bootstraps all the components needed to run Kong on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. ## TL;DR; ```bash helm repo add kong https://charts.konghq.com helm repo update helm install kong/kong --generate-name ``` ## Table of contents - [Prerequisites](#prerequisites) - [Install](#install) - [Uninstall](#uninstall) - [FAQs](#faqs) - [Kong Enterprise](#kong-enterprise) - [Deployment Options](#deployment-options) - [Database](#database) - [DB-less deployment](#db-less-deployment) - [Using the Postgres sub-chart](#using-the-postgres-sub-chart) - [Postgres sub-chart considerations for OpenShift](#postgres-sub-chart-considerations-for-openshift) - [Runtime package](#runtime-package) - [Configuration method](#configuration-method) - [Separate admin and proxy nodes](#separate-admin-and-proxy-nodes) - [Standalone controller nodes](#standalone-controller-nodes) - [Hybrid mode](#hybrid-mode) - [Certificates](#certificates) - [Control plane node configuration](#control-plane-node-configuration) - [Data plane node configuration](#data-plane-node-configuration) - [Cert Manager Integration](#cert-manager-integration) - [CRD management](#crd-management) - [InitContainers](#initcontainers) - [HostAliases](#hostaliases) - [Sidecar Containers](#sidecar-containers) - [Migration Sidecar Containers](#migration-sidecar-containers) - [User Defined Volumes](#user-defined-volumes) - [User Defined Volume Mounts](#user-defined-volume-mounts) - [Removing cluster-scoped permissions](#removing-cluster-scoped-permissions) - [Using a DaemonSet](#using-a-daemonset) - [Using dnsPolicy and dnsConfig](#using-dnspolicy-and-dnsconfig) - [Example configurations](#example-configurations) - [Configuration](#configuration) - [Kong parameters](#kong-parameters) - [Kong Service Parameters](#kong-service-parameters) - [Admin Service mTLS](#admin-service-mtls) - [Stream listens](#stream-listens) - [Ingress Controller Parameters](#ingress-controller-parameters) - [The `env` section](#the-env-section) - [The `customEnv` section](#the-customenv-section) - [General Parameters](#general-parameters) - [The `env` section](#the-env-section-1) - [The `customEnv` section](#the-customenv-section-1) - [The `extraLabels` section](#the-extralabels-section) - [Kong Enterprise Parameters](#kong-enterprise-parameters) - [Overview](#overview) - [Prerequisites](#prerequisites-1) - [Kong Enterprise License](#kong-enterprise-license) - [Kong Enterprise Docker registry access](#kong-enterprise-docker-registry-access) - [Service location hints](#service-location-hints) - [RBAC](#rbac) - [Sessions](#sessions) - [Email/SMTP](#emailsmtp) - [Prometheus Operator integration](#prometheus-operator-integration) - [Argo CD considerations](#argo-cd-considerations) - [Changelog](https://github.com/Kong/charts/blob/main/charts/kong/CHANGELOG.md) - [Upgrading](https://github.com/Kong/charts/blob/main/charts/kong/UPGRADE.md) - [Seeking help](#seeking-help) ## Prerequisites - Kubernetes 1.17+. Older chart releases support older Kubernetes versions. Refer to the [supported version matrix](https://docs.konghq.com/kubernetes-ingress-controller/latest/references/version-compatibility/#kubernetes) and the [chart changelog](https://github.com/Kong/charts/blob/main/charts/kong/CHANGELOG.md) for information about the default chart controller versions and Kubernetes versions supported by controller releases. - PV provisioner support in the underlying infrastructure if persistence is needed for Kong datastore. ## Install To install Kong: ```bash helm repo add kong https://charts.konghq.com helm repo update helm install kong/kong --generate-name ``` ## Uninstall To uninstall/delete a Helm release `my-release`: ```bash helm delete my-release ``` The command removes all the Kubernetes components associated with the chart and deletes the release. > **Tip**: List all releases using `helm list` ## FAQs Please read the [FAQs](https://github.com/Kong/charts/blob/main/charts/kong/FAQs.md) document. ## Kong Enterprise If using Kong Enterprise, several additional steps are necessary before installing the chart: - Set `enterprise.enabled` to `true` in `values.yaml` file. - Update values.yaml to use a Kong Enterprise image. - Satisfy the two prerequisites below for [Enterprise License](#kong-enterprise-license) and [Enterprise Docker Registry](#kong-enterprise-docker-registry-access). - (Optional) [set a `password` environment variable](#rbac) to create the initial super-admin. Though not required, this is recommended for users that wish to use RBAC, as it cannot be done after initial setup. Once you have these set, it is possible to install Kong Enterprise. Please read through [Kong Enterprise considerations](#kong-enterprise-parameters) to understand all settings that are enterprise specific. ## Deployment Options Kong is a highly configurable piece of software that can be deployed in a number of different ways, depending on your use-case. All combinations of various runtimes, databases and configuration methods are supported by this Helm chart. The recommended approach is to use the Ingress Controller based configuration along-with DB-less mode. Following sections detail on various high-level architecture options available: ### Database Kong can run with or without a database (DB-less). By default, this chart installs Kong without a database. You can set the database the `env.database` parameter. For more details, please read the [env](#the-env-section) section. #### DB-less deployment When deploying Kong in DB-less mode(`env.database: "off"`) and without the Ingress Controller(`ingressController.enabled: false`), you have to provide a [declarative configuration](https://docs.konghq.com/gateway-oss/latest/db-less-and-declarative-config/#the-declarative-configuration-format) for Kong to run. You can provide an existing ConfigMap (`dblessConfig.configMap`) or Secret (`dblessConfig.secret`) or place the whole configuration into `values.yaml` (`dblessConfig.config`) parameter. See the example configuration in the default values.yaml for more details. You can use `--set-file dblessConfig.config=/path/to/declarative-config.yaml` in Helm commands to substitute in a complete declarative config file. Note that externally supplied ConfigMaps are not hashed or tracked in deployment annotations. Subsequent ConfigMap updates will require user-initiated new deployment rollouts to apply the new configuration. You should run `kubectl rollout restart deploy` after updating externally supplied ConfigMap content. #### Using the Postgres sub-chart The chart can optionally spawn a Postgres instance using [Bitnami's Postgres chart](https://github.com/bitnami/charts/blob/master/bitnami/postgresql/README.md) as a sub-chart. Set `postgresql.enabled=true` to enable the sub-chart. Enabling this will auto-populate Postgres connection settings in Kong's environment. The Postgres sub-chart is best used to quickly provision temporary environments without installing and configuring your database separately. For longer-lived environments, we recommend you manage your database outside the Kong Helm release. ##### Postgres sub-chart considerations for OpenShift Due to the default `securityContexts` in the postgres sub-chart, you will need to add the following values to the `postgresql` section to get postgres running on OpenShift: ```yaml volumePermissions: enabled: false securityContext: runAsUser: "auto" primary: containerSecurityContext: enabled: false podSecurityContext: enabled: false ``` ### Runtime package There are three different packages of Kong that are available: - **Kong Gateway**\ This is the [Open-Source](https://github.com/kong/kong) offering. It is a full-blown API Gateway and Ingress solution with a wide-array of functionality. When Kong Gateway is combined with the Ingress based configuration method, you get Kong for Kubernetes. This is the default deployment for this Helm Chart. - **Kong Enterprise K8S**\ This package builds up on top of the Open-Source Gateway and bundles in all the Enterprise-only plugins as well. When Kong Enterprise K8S is combined with the Ingress based configuration method, you get Kong for Kubernetes Enterprise. This package also comes with 24x7 support from Kong Inc. - **Kong Enterprise**\ This is the full-blown Enterprise package which packs with itself all the Enterprise functionality like Manager, Portal, Vitals, etc. This package can't be run in DB-less mode. The package to run can be changed via `image.repository` and `image.tag` parameters. If you would like to run the Enterprise package, please read the [Kong Enterprise Parameters](#kong-enterprise-parameters) section. ### Configuration method Kong can be configured via two methods: - **Ingress and CRDs**\ The configuration for Kong is done via `kubectl` and Kubernetes-native APIs. This is also known as Kong Ingress Controller or Kong for Kubernetes and is the default deployment pattern for this Helm Chart. The configuration for Kong is managed via Ingress and a few [Custom Resources](https://docs.konghq.com/kubernetes-ingress-controller/latest/concepts/custom-resources). For more details, please read the [documentation](https://docs.konghq.com/kubernetes-ingress-controller/) on Kong Ingress Controller. To configure and fine-tune the controller, please read the [Ingress Controller Parameters](#ingress-controller-parameters) section. - **Admin API**\ This is the traditional method of running and configuring Kong. By default, the Admin API of Kong is not exposed as a Service. This can be controlled via `admin.enabled` and `env.admin_listen` parameters. ### Separate admin and proxy nodes *Note: although this section is titled "Separate admin and proxy nodes", this split release technique is generally applicable to any deployment with different types of Kong nodes. Separating Admin API and proxy nodes is one of the more common use cases for splitting across multiple releases, but you can also split releases for split proxy and Developer Portal nodes, multiple groups of proxy nodes with separate listen configurations for network segmentation, etc. However, it does not apply to hybrid mode, as only the control plane release interacts with the database.* Users may wish to split their Kong deployment into multiple instances that only run some of Kong's services (i.e. you run `helm install` once for every instance type you wish to create). To disable Kong services on an instance, you should set `SVC.enabled`, `SVC.http.enabled`, `SVC.tls.enabled`, and `SVC.ingress.enabled` all to `false`, where `SVC` is `proxy`, `admin`, `manager`, `portal`, or `portalapi`. The standard chart upgrade automation process assumes that there is only a single Kong release in the Kong cluster, and runs both `migrations up` and `migrations finish` jobs. To handle clusters split across multiple releases, you should: 1. Upgrade one of the releases with `helm upgrade RELEASENAME -f values.yaml --set migrations.preUpgrade=true --set migrations.postUpgrade=false`. 2. Upgrade all but one of the remaining releases with `helm upgrade RELEASENAME -f values.yaml --set migrations.preUpgrade=false --set migrations.postUpgrade=false`. 3. Upgrade the final release with `helm upgrade RELEASENAME -f values.yaml --set migrations.preUpgrade=false --set migrations.postUpgrade=true`. This ensures that all instances are using the new Kong package before running `kong migrations finish`. Users should note that Helm supports supplying multiple values.yaml files, allowing you to separate shared configuration from instance-specific configuration. For example, you may have a shared values.yaml that contains environment variables and other common settings, and then several instance-specific values.yamls that contain service configuration only. You can then create releases with: ```bash helm install proxy-only -f shared-values.yaml -f only-proxy.yaml kong/kong helm install admin-only -f shared-values.yaml -f only-admin.yaml kong/kong ``` ### Standalone controller nodes The chart can deploy releases that contain the controller only, with no Kong container, by setting `deployment.kong.enabled: false` in values.yaml. There are several controller settings that must be populated manually in this scenario and several settings that are useful when using multiple controllers: * `ingressController.env.kong_admin_url` must be set to the Kong Admin API URL. If the Admin API is exposed by a service in the cluster, this should look something like `https://my-release-kong-admin.kong-namespace.svc:8444` * `ingressController.env.publish_service` must be set to the Kong proxy service, e.g. `namespace/my-release-kong-proxy`. * `ingressController.ingressClass` should be set to a different value for each instance of the controller. * `ingressController.env.kong_admin_filter_tag` should be set to a different value for each instance of the controller. * If using Kong Enterprise, `ingressController.env.kong_workspace` can optionally create configuration in a workspace other than `default`. Standalone controllers require a database-backed Kong instance, as DB-less mode requires that a single controller generate a complete Kong configuration. ### Hybrid mode Kong supports [hybrid mode deployments](https://docs.konghq.com/2.0.x/hybrid-mode/) as of Kong 2.0.0 and [Kong Enterprise 2.1.0](https://docs.konghq.com/enterprise/2.1.x/deployment/hybrid-mode/). These deployments split Kong nodes into control plane (CP) nodes, which provide the admin API and interact with the database, and data plane (DP) nodes, which provide the proxy and receive configuration from control plane nodes. You can deploy hybrid mode Kong clusters by [creating separate releases for each node type](#separate-admin-and-proxy-nodes), i.e. use separate control and data plane values.yamls that are then installed separately. The [control plane](#control-plane-node-configuration) and [data plane](#data-plane-node-configuration) configuration sections below cover the values.yaml specifics for each. Cluster certificates are not generated automatically. You must [create a certificate and key pair](#certificates) for intra-cluster communication. When upgrading the Kong version, you must [upgrade the control plane release first and then upgrade the data plane release](https://docs.konghq.com/gateway/latest/plan-and-deploy/hybrid-mode/#version-compatibility). #### Certificates > This example shows how to use Kong Hybrid mode with `cluster_mtls: shared`. > For an example of `cluster_mtls: pki` see the [hybrid-cert-manager example](https://github.com/Kong/charts/blob/main/charts/kong/example-values/hybrid-cert-manager/) Hybrid mode uses TLS to secure the CP/DP node communication channel, and requires certificates for it. You can generate these either using `kong hybrid gen_cert` on a local Kong installation or using OpenSSL: ```bash openssl req -new -x509 -nodes -newkey ec:<(openssl ecparam -name secp384r1) \ -keyout /tmp/cluster.key -out /tmp/cluster.crt \ -days 1095 -subj "/CN=kong_clustering" ``` You must then place these certificates in a Secret: ```bash kubectl create secret tls kong-cluster-cert --cert=/tmp/cluster.crt --key=/tmp/cluster.key ``` #### Control plane node configuration You must configure the control plane nodes to mount the certificate secret on the container filesystem is serve it from the cluster listen. In values.yaml: ```yaml secretVolumes: - kong-cluster-cert ``` ```yaml env: role: control_plane cluster_cert: /etc/secrets/kong-cluster-cert/tls.crt cluster_cert_key: /etc/secrets/kong-cluster-cert/tls.key ``` Furthermore, you must enable the cluster listen and Kubernetes Service, and should typically disable the proxy: ```yaml cluster: enabled: true tls: enabled: true servicePort: 8005 containerPort: 8005 proxy: enabled: false ``` Enterprise users with Vitals enabled must also enable the cluster telemetry service: ```yaml clustertelemetry: enabled: true tls: enabled: true servicePort: 8006 containerPort: 8006 ``` If using the ingress controller, you must also specify the DP proxy service as its publish target to keep Ingress status information up to date: ``` ingressController: env: publish_service: hybrid/example-release-data-kong-proxy ``` Replace `hybrid` with your DP nodes' namespace and `example-release-data` with the name of the DP release. #### Data plane node configuration Data plane configuration also requires the certificate and `role` configuration, and the database should always be set to `off`. You must also trust the cluster certificate and indicate what hostname/port Kong should use to find control plane nodes. Though not strictly required, you should disable the admin service (it will not work on DP nodes anyway, but should be disabled to avoid creating an invalid Service resource). ```yaml secretVolumes: - kong-cluster-cert ``` ```yaml admin: enabled: false ``` ```yaml env: role: data_plane database: "off" cluster_cert: /etc/secrets/kong-cluster-cert/tls.crt cluster_cert_key: /etc/secrets/kong-cluster-cert/tls.key lua_ssl_trusted_certificate: /etc/secrets/kong-cluster-cert/tls.crt cluster_control_plane: control-plane-release-name-kong-cluster.hybrid.svc.cluster.local:8005 cluster_telemetry_endpoint: control-plane-release-name-kong-clustertelemetry.hybrid.svc.cluster.local:8006 # Enterprise-only ``` Note that the `cluster_control_plane` value will differ depending on your environment. `control-plane-release-name` will change to your CP release name, `hybrid` will change to whatever namespace it resides in. See [Kubernetes' documentation on Service DNS](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/) for more detail. If you use multiple Helm releases to manage different data plane configurations attached to the same control plane, setting the `deployment.hostname` field will help you keep track of which is which in the `/clustering/data-plane` endpoint. ### Cert Manager Integration By default, Kong will create self-signed certificates on start for its TLS listens if you do not provide your own. The chart can create [cert-manager](https://cert-manager.io/docs/) Certificates for its Services and configure them for you. To use this integration, install cert-manager, create an issuer, set `certificates.enabled: true` in values.yaml, and set your issuer name in `certificates.issuer` or `certificates.clusterIssuer` depending on the issuer type. If you do not have an issuer available, you can install the example [self-signed ClusterIssuer](https://cert-manager.io/docs/configuration/selfsigned/#bootstrapping-ca-issuers) and set `certificates.clusterIssuer: selfsigned-issuer` for testing. You should, however, migrate to an issuer using a CA your clients trust for actual usage. The `proxy`, `admin`, `portal`, and `cluster` subsections under `certificates` let you choose hostnames, override issuers, set `subject` or set `privateKey` on a per-certificate basis for the proxy, admin API and Manager, Portal and Portal API, and hybrid mode mTLS services, respectively. To use hybrid mode, the control and data plane releases must use the same issuer for their cluster certificates. ### CRD management Earlier versions of this chart (<2.0) created CRDs associated with the ingress controller as part of the release. This raised two challenges: - Multiple release of the chart would conflict with one another, as each would attempt to create its own set of CRDs. - Because deleting a CRD also deletes any custom resources associated with it, deleting a release of the chart could destroy user configuration without providing any means to restore it. Helm 3 introduced a simplified CRD management method that was safer, but requires some manual work when a chart added or modified CRDs: CRDs are created on install if they are not already present, but are not modified during release upgrades or deletes. Our chart release upgrade instructions call out when manual action is necessary to update CRDs. This CRD handling strategy is recommended for most users. Some users may wish to manage their CRDs automatically. If you manage your CRDs this way, we _strongly_ recommend that you back up all associated custom resources in the event you need to recover from unintended CRD deletion. While Helm 3's CRD management system is recommended, there is no simple means of migrating away from release-managed CRDs if you previously installed your release with the old system (you would need to back up your existing custom resources, delete your release, reinstall, and restore your custom resources after). As such, the chart detects if you currently use release-managed CRDs and continues to use the old CRD templates when using chart version 2.0+. If you do (your resources will have a `meta.helm.sh/release-name` annotation), we _strongly_ recommend that you back up all associated custom resources in the event you need to recover from unintended CRD deletion. ### InitContainers The chart is able to deploy initContainers along with Kong. This can be very useful when there's a requirement for custom initialization. The `deployment.initContainers` field in values.yaml takes an array of objects that get appended as-is to the existing `spec.template.initContainers` array in the kong deployment resource. ### HostAliases The chart is able to inject host aliases into containers. This can be very useful when it's required to resolve additional domain name which can't be looked-up directly from dns server. The `deployment.hostAliases` field in values.yaml takes an array of objects that set to `spec.template.hostAliases` field in the kong deployment resource. ### Sidecar Containers The chart can deploy additional containers along with the Kong and Ingress Controller containers, sometimes referred to as "sidecar containers". This can be useful to include network proxies or logging services along with Kong. The `deployment.sidecarContainers` field in values.yaml takes an array of objects that get appended as-is to the existing `spec.template.spec.containers` array in the Kong deployment resource. ### Migration Sidecar Containers In the same way sidecar containers are attached to the Kong and Ingress Controller containers the chart can add sidecars to the containers that runs the migrations. The `migrations.sidecarContainers` field in values.yaml takes an array of objects that get appended as-is to the existing `spec.template.spec.containers` array in the pre-upgrade-migrations, post-upgrade-migrations and migration resrouces. Keep in mind the containers should be finite and they should be terminated with the migration containers, otherwise the migration could get the status as finished and the deployment of the chart will reach the timeout. ### User Defined Volumes The chart can deploy additional volumes along with Kong. This can be useful to include additional volumes which required during iniatilization phase (InitContainer). The `deployment.userDefinedVolumes` field in values.yaml takes an array of objects that get appended as-is to the existing `spec.template.spec.volumes` array in the kong deployment resource. ### User Defined Volume Mounts The chart can mount user-defined volumes. The `deployment.userDefinedVolumeMounts` and `ingressController.userDefinedVolumeMounts` fields in values.yaml take an array of object that get appended as-is to the existing `spec.template.spec.containers[].volumeMounts` and `spec.template.spec.initContainers[].volumeMounts` array in the kong deployment resource. ### Removing cluster-scoped permissions You can limit the controller's access to allow it to only watch specific namespaces for namespaced resources. By default, the controller watches all namespaces. Limiting access requires several changes to configuration: - Set `ingressController.watchNamespaces` to a list of namespaces you want to watch. The chart will automatically generate roles for each namespace and assign them to the controller's service account. - Optionally set `ingressController.installCRDs=false` if your user role (the role you use when running `helm install`, not the controller service account's role) does not have access to get CRDs. By default, the chart attempts to look up the controller CRDs for [a legacy behavior check](#crd-management). ### Using a DaemonSet Setting `deployment.daemonset: true` deploys Kong using a [DaemonSet controller](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) instead of a Deployment controller. This runs a Kong Pod on every kubelet in the Kubernetes cluster. For such configuration it may be desirable to configure Pods to use the network of the host they run on instead of a dedicated network namespace. The benefit of this approach is that the Kong can bind ports directly to Kubernetes nodes' network interfaces, without the extra network translation imposed by NodePort Services. It can be achieved by setting `deployment.hostNetwork: true`. ### Using dnsPolicy and dnsConfig The chart able to inject custom DNS configuration into containers. This can be useful when you have EKS cluster with [NodeLocal DNSCache](https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/) configured and attach AWS security groups directly to pod using [security groups for pods feature](https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html). ### Example configurations Several example values.yaml are available in the [example-values](https://github.com/Kong/charts/blob/main/charts/kong/example-values/) directory. ## Configuration ### Kong parameters | Parameter | Description | Default | | ---------------------------------- | ------------------------------------------------------------------------------------- | ------------------- | | image.repository | Kong image | `kong` | | image.tag | Kong image version | `3.5` | | image.effectiveSemver | Semantic version to use for version-dependent features (if `tag` is not a semver) | | | image.pullPolicy | Image pull policy | `IfNotPresent` | | image.pullSecrets | Image pull secrets | `null` | | replicaCount | Kong instance count. It has no effect when `autoscaling.enabled` is set to true | `1` | | plugins | Install custom plugins into Kong via ConfigMaps or Secrets | `{}` | | env | Additional [Kong configurations](https://getkong.org/docs/latest/configuration/) | | | customEnv | Custom Environment variables without `KONG_` prefix | | | envFrom | Populate environment variables from ConfigMap or Secret keys | | | migrations.preUpgrade | Run "kong migrations up" jobs | `true` | | migrations.postUpgrade | Run "kong migrations finish" jobs | `true` | | migrations.annotations | Annotations for migration job pods | `{"sidecar.istio.io/inject": "false" | | migrations.jobAnnotations | Additional annotations for migration jobs | `{}` | | migrations.backoffLimit | Override the system backoffLimit | `{}` | | waitImage.enabled | Spawn init containers that wait for the database before starting Kong | `true` | | waitImage.repository | Image used to wait for database to become ready. Uses the Kong image if none set | | | waitImage.tag | Tag for image used to wait for database to become ready | | | waitImage.pullPolicy | Wait image pull policy | `IfNotPresent` | | postgresql.enabled | Spin up a new postgres instance for Kong | `false` | | dblessConfig.configMap | Name of an existing ConfigMap containing the `kong.yml` file. This must have the key `kong.yml`.| `` | | dblessConfig.config | Yaml configuration file for the dbless (declarative) configuration of Kong | see in `values.yaml` | #### Kong Service Parameters The various `SVC.*` parameters below are common to the various Kong services (the admin API, proxy, Kong Manager, the Developer Portal, and the Developer Portal API) and define their listener configuration, K8S Service properties, and K8S Ingress properties. Defaults are listed only if consistent across the individual services: see values.yaml for their individual default values. `SVC` below can be substituted with each of: * `proxy` * `udpProxy` * `admin` * `manager` * `portal` * `portalapi` * `cluster` * `clustertelemetry` * `status` `status` is intended for internal use within the cluster. Unlike other services it cannot be exposed externally, and cannot create a Kubernetes service or ingress. It supports the settings under `SVC.http` and `SVC.tls` only. `cluster` is used on hybrid mode control plane nodes. It does not support the `SVC.http.*` settings (cluster communications must be TLS-only) or the `SVC.ingress.*` settings (cluster communication requires TLS client authentication, which cannot pass through an ingress proxy). `clustertelemetry` is similar, and used when Vitals is enabled on Kong Enterprise control plane nodes. `udpProxy` is used for UDP stream listens (Kubernetes does not yet support mixed TCP/UDP LoadBalancer Services). It _does not_ support the `http`, `tls`, or `ingress` sections, as it is used only for stream listens. | Parameter | Description | Default | |-----------------------------------|-------------------------------------------------------------------------------------------|--------------------------| | SVC.enabled | Create Service resource for SVC (admin, proxy, manager, etc.) | | | SVC.http.enabled | Enables http on the service | | | SVC.http.servicePort | Service port to use for http | | | SVC.http.containerPort | Container port to use for http | | | SVC.http.nodePort | Node port to use for http | | | SVC.http.hostPort | Host port to use for http | | | SVC.http.parameters | Array of additional listen parameters | `[]` | | SVC.http.appProtocol | `appProtocol` to be set in a Service's port. If left empty, no `appProtocol` will be set. | | | SVC.tls.enabled | Enables TLS on the service | | | SVC.tls.containerPort | Container port to use for TLS | | | SVC.tls.servicePort | Service port to use for TLS | | | SVC.tls.nodePort | Node port to use for TLS | | | SVC.tls.hostPort | Host port to use for TLS | | | SVC.tls.overrideServiceTargetPort | Override service port to use for TLS without touching Kong containerPort | | | SVC.tls.parameters | Array of additional listen parameters | `["http2"]` | | SVC.tls.appProtocol | `appProtocol` to be set in a Service's port. If left empty, no `appProtocol` will be set. | | | SVC.type | k8s service type. Options: NodePort, ClusterIP, LoadBalancer | | | SVC.clusterIP | k8s service clusterIP | | | SVC.loadBalancerClass | loadBalancerClass to use for LoadBalancer provisionning | | | SVC.loadBalancerSourceRanges | Limit service access to CIDRs if set and service type is `LoadBalancer` | `[]` | | SVC.loadBalancerIP | Reuse an existing ingress static IP for the service | | | SVC.externalIPs | IPs for which nodes in the cluster will also accept traffic for the servic | `[]` | | SVC.externalTrafficPolicy | k8s service's externalTrafficPolicy. Options: Cluster, Local | | | SVC.ingress.enabled | Enable ingress resource creation (works with SVC.type=ClusterIP) | `false` | | SVC.ingress.ingressClassName | Set the ingressClassName to associate this Ingress with an IngressClass | | | SVC.ingress.hostname | Ingress hostname | `""` | | SVC.ingress.path | Ingress path. | `/` | | SVC.ingress.pathType | Ingress pathType. One of `ImplementationSpecific`, `Exact` or `Prefix` | `ImplementationSpecific` | | SVC.ingress.hosts | Slice of hosts configurations, including `hostname`, `path` and `pathType` keys | `[]` | | SVC.ingress.tls | Name of secret resource or slice of `secretName` and `hosts` keys | | | SVC.ingress.annotations | Ingress annotations. See documentation for your ingress controller for details | `{}` | | SVC.ingress.labels | Ingress labels. Additional custom labels to add to the ingress. | `{}` | | SVC.annotations | Service annotations | `{}` | | SVC.labels | Service labels | `{}` | #### Admin Service mTLS On top of the common parameters listed above, the `admin` service supports parameters for mTLS client verification. If any of `admin.tls.client.caBundle` or `admin.tls.client.secretName` are set, the admin service will be configured to require mTLS client verification. If both are set, `admin.tls.client.caBundle` will take precedence. | Parameter | Description | Default | |-----------------------------|---------------------------------------------------------------------------------------------|---------| | admin.tls.client.caBundle | CA certificate to use for TLS verification of the Admin API client (PEM-encoded). | `""` | | admin.tls.client.secretName | CA certificate secret name - must contain a `tls.crt` key with the PEM-encoded certificate. | `""` | #### Stream listens The proxy configuration additionally supports creating stream listens. These are configured using an array of objects under `proxy.stream` and `udpProxy.stream`: | Parameter | Description | Default | | ---------------------------------- | ------------------------------------------------------------------------------------- | ------------------- | | protocol | The listen protocol, either "TCP" or "UDP" | | | containerPort | Container port to use for a stream listen | | | servicePort | Service port to use for a stream listen | | | nodePort | Node port to use for a stream listen | | | hostPort | Host port to use for a stream listen | | | parameters | Array of additional listen parameters | `[]` | ### Ingress Controller Parameters All of the following properties are nested under the `ingressController` section of `values.yaml` file: | Parameter | Description | Default | |--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------| | enabled | Deploy the ingress controller, rbac and crd | true | | image.repository | Docker image with the ingress controller | kong/kubernetes-ingress-controller | | image.tag | Version of the ingress controller | `3.0` | | image.effectiveSemver | Version of the ingress controller used for version-specific features when image.tag is not a valid semantic version | | | readinessProbe | Kong ingress controllers readiness probe | | | livenessProbe | Kong ingress controllers liveness probe | | | installCRDs | Legacy toggle for Helm 2-style CRD management. Should not be set [unless necessary due to cluster permissions](#removing-cluster-scoped-permissions). | false | | env | Specify Kong Ingress Controller configuration via environment variables | | | customEnv | Specify custom environment variables (without the CONTROLLER_ prefix) | | | envFrom | Populate environment variables from ConfigMap or Secret keys | | | ingressClass | The name of this controller's ingressClass | kong | | ingressClassAnnotations | The ingress-class value for controller | kong | | args | List of ingress-controller cli arguments | [] | | watchNamespaces | List of namespaces to watch. Watches all namespaces if empty | [] | | admissionWebhook.enabled | Whether to enable the validating admission webhook | true | | admissionWebhook.failurePolicy | How unrecognized errors from the admission endpoint are handled (Ignore or Fail) | Ignore | | admissionWebhook.port | The port the ingress controller will listen on for admission webhooks | 8080 | | admissionWebhook.address | The address the ingress controller will listen on for admission webhooks, if not 0.0.0.0 | | | admissionWebhook.annotations | Annotations for the Validation Webhook Configuration | | | admissionWebhook.certificate.provided | Use a provided certificate. When set to false, the chart will automatically generate a certificate. | false | | admissionWebhook.certificate.secretName | Name of the TLS secret for the provided webhook certificate | | | admissionWebhook.certificate.caBundle | PEM encoded CA bundle which will be used to validate the provided webhook certificate | | | admissionWebhook.namespaceSelector | Add namespaceSelector to the webhook. Please go to [Kubernetes doc for the specs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector) | | | admissionWebhook.timeoutSeconds | Kubernetes `apiserver`'s timeout when running this webhook. Default: 10 seconds. | | | userDefinedVolumes | Create volumes. Please go to Kubernetes doc for the spec of the volumes | | | userDefinedVolumeMounts | Create volumeMounts. Please go to Kubernetes doc for the spec of the volumeMounts | | | terminationGracePeriodSeconds | Sets the [termination grace period](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#hook-handler-execution) for Deployment pod | 30 | | gatewayDiscovery.enabled | Enables Kong instance service discovery (for more details see [gatewayDiscovery section][gd_section]) | false | | gatewayDiscovery.generateAdminApiService | Generate the admin API service name based on the release name (for more details see [gatewayDiscovery section][gd_section]) | false | | gatewayDiscovery.adminApiService.namespace | The namespace of the Kong admin API service (for more details see [gatewayDiscovery section][gd_section]) | `.Release.Namespace` | | gatewayDiscovery.adminApiService.name | The name of the Kong admin API service (for more details see [gatewayDiscovery section][gd_section]) | "" | | konnect.enabled | Enable synchronisation of data plane configuration with Konnect Runtime Group | false | | konnect.runtimeGroupID | Konnect Runtime Group's unique identifier. | | | konnect.apiHostname | Konnect API hostname. Defaults to a production US-region. | us.kic.api.konghq.com | | konnect.tlsClientCertSecretName | Name of the secret that contains Konnect Runtime Group's client TLS certificate. | konnect-client-tls | | konnect.license.enabled | Enable automatic license provisioning for Gateways managed by Ingress Controller in Konnect mode. | false | | adminApi.tls.client.enabled | Enable TLS client verification for the Admin API. By default, Helm will generate certificates automatically. | false | | adminApi.tls.client.certProvided | Use user-provided certificates. If set to false, Helm will generate certificates. | false | | adminApi.tls.client.secretName | Client TLS certificate/key pair secret name. Can be also set when `certProvided` is false to enforce a generated secret's name. | "" | | adminApi.tls.client.caSecretName | CA TLS certificate/key pair secret name. Can be also set when `certProvided` is false to enforce a generated secret's name. | "" | [gd_section]: #the-gatewayDiscovery-section #### The `env` section For a complete list of all configuration values you can set in the `env` section, please read the Kong Ingress Controller's [configuration document](https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/cli-arguments/). #### The `customEnv` section The `customEnv` section can be used to configure all environment variables other than Ingress Controller configuration. Any key value put under this section translates to environment variables. Every key is upper-cased before setting the environment variable. An example: ```yaml kong: ingressController: customEnv: TZ: "Europe/Berlin" ``` #### The `gatewayDiscovery` section Kong Ingress Controller v2.9 has introduced gateway discovery which allows the controller to discover Gateway instances that it should configure using an Admin API Kubernetes service. Using this feature requires a split release installation of Gateways and Ingress Controller. For exemplar `values.yaml` files which use this feature please see: [examples README.md](./example-values/README.md). or use the [`ingress` chart](../ingress/README.md) which can handle this for you. ##### Configuration You'll be able to configure this feature through configuration section under `ingressController.gatewayDiscovery`: - If `ingressController.gatewayDiscovery.enabled` is set to `false`: the ingress controller will control a pre-determined set of Gateway instances based on Admin API URLs (provided under the hood via `CONTROLLER_KONG_ADMIN_URL` environment variable). - If `ingressController.gatewayDiscovery.enabled` is set to `true`: the ingress controller will dynamically locate Gateway instances by watching the specified Kubernetes service. (provided under the hood via `CONTROLLER_KONG_ADMIN_SVC` environment variable). The following admin API Service flags have to be present in order for gateway discovery to work: - `ingressController.gatewayDiscovery.adminApiService.name` - `ingressController.gatewayDiscovery.adminApiService.namespace` If you set `ingressController.gatewayDiscovery.generateAdminApiService` to `true`, the chart will generate values for `name` and `namespace` based on the current release name and namespace. This is useful when consuming the `kong` chart as a subchart. Additionally, you can control the addresses that are generated for your Gateways via the `--gateway-discovery-dns-strategy` CLI flag that can be set on the Ingress Controller (or an equivalent environment variable: `CONTROLLER_GATEWAY_DISCOVERY_DNS_STRATEGY`). It accepts 3 values which change the way that Gateway addresses are generated: - `service` - for service scoped pod DNS names: `pod-ip-address.service-name.my-namespace.svc.cluster-domain.example` - `pod` - for namespace scope pod DNS names: `pod-ip-address.my-namespace.pod.cluster-domain.example` - `ip` (default, retains behavior introduced in v2.9) - for regular IP addresses When using `gatewayDiscovery`, you should consider configuring the Admin service to use mTLS client verification to make this interface secure. Without that, anyone who can access the Admin API from inside the cluster can configure the Gateway instances. On the controller release side, that can be achieved by setting `ingressController.adminApi.tls.client.enabled` to `true`. By default, Helm will generate a certificate Secret named `-admin-api-keypair` and a CA Secret named `-admin-api-ca-keypair` for you. To provide your own cert, set `ingressController.adminApi.tls.client.certProvided` to `true`, `ingressController.adminApi.tls.client.secretName` to the name of the Secret containing your client cert, and `ingressController.adminApi.tls.client.caSecretName` to the name of the Secret containing your CA cert. On the Gateway release side, set either `admin.tls.client.secretName` to the name of your CA Secret or set `admin.tls.client.caBundle` to the CA certificate string. ### General Parameters | Parameter | Description | Default | | ---------------------------------- | ------------------------------------------------------------------------------------- | ------------------- | | namespace | Namespace to deploy chart resources | | | deployment.kong.enabled | Enable or disable deploying Kong | `true` | | deployment.minReadySeconds | Minimum number of seconds for which newly created pods should be ready without any of its container crashing, for it to be considered available. | | | deployment.initContainers | Create initContainers. Please go to Kubernetes doc for the spec of the initContainers | | | deployment.daemonset | Use a DaemonSet instead of a Deployment | `false` | | deployment.hostname | Set the Deployment's `.spec.template.hostname`. Kong reports this as its hostname. | | | deployment.hostNetwork | Enable hostNetwork, which binds to the ports to the host | `false` | | deployment.userDefinedVolumes | Create volumes. Please go to Kubernetes doc for the spec of the volumes | | | deployment.userDefinedVolumeMounts | Create volumeMounts. Please go to Kubernetes doc for the spec of the volumeMounts | | | deployment.serviceAccount.create | Create Service Account for the Deployment / Daemonset and the migrations | `true` | | deployment.serviceAccount.automountServiceAccountToken | Enable ServiceAccount token automount in Kong deployment | `false` | | deployment.serviceAccount.name | Name of the Service Account, a default one will be generated if left blank. | "" | | deployment.serviceAccount.annotations | Annotations for the Service Account | {} | | deployment.test.enabled | Enable creation of test resources for use with "helm test" | `false` | | autoscaling.enabled | Set this to `true` to enable autoscaling | `false` | | autoscaling.minReplicas | Set minimum number of replicas | `2` | | autoscaling.maxReplicas | Set maximum number of replicas | `5` | | autoscaling.behavior | Sets the [behavior for scaling up and down](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#configurable-scaling-behavior) | `{}` | | autoscaling.targetCPUUtilizationPercentage | Target Percentage for when autoscaling takes affect. Only used if cluster does not support `autoscaling/v2` or `autoscaling/v2beta2` | `80` | | autoscaling.metrics | metrics used for autoscaling for clusters that supports `autoscaling/v2` or `autoscaling/v2beta2` | See [values.yaml](values.yaml) | | updateStrategy | update strategy for deployment | `{}` | | readinessProbe | Kong readiness probe | | | livenessProbe | Kong liveness probe | | | startupProbe | Kong startup probe | | | lifecycle | Proxy container lifecycle hooks | see `values.yaml` | | terminationGracePeriodSeconds | Sets the [termination grace period](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#hook-handler-execution) for Deployment pods | 30 | | affinity | Node/pod affinities | | | topologySpreadConstraints | Control how Pods are spread across cluster among failure-domains | | | nodeSelector | Node labels for pod assignment | `{}` | | deploymentAnnotations | Annotations to add to deployment | see `values.yaml` | | podAnnotations | Annotations to add to each pod | see `values.yaml` | | podLabels | Labels to add to each pod | `{}` | | resources | Pod resource requests & limits | `{}` | | tolerations | List of node taints to tolerate | `[]` | | dnsPolicy | Pod dnsPolicy | | | dnsConfig | Pod dnsConfig | | | podDisruptionBudget.enabled | Enable PodDisruptionBudget for Kong | `false` | | podDisruptionBudget.maxUnavailable | Represents the minimum number of Pods that can be unavailable (integer or percentage) | `50%` | | podDisruptionBudget.minAvailable | Represents the number of Pods that must be available (integer or percentage) | | | podSecurityPolicy.enabled | Enable podSecurityPolicy for Kong | `false` | | podSecurityPolicy.labels | Labels to add to podSecurityPolicy for Kong | `{}` | | podSecurityPolicy.annotations | Annotations to add to podSecurityPolicy for Kong | `{}` | | podSecurityPolicy.spec | Collection of [PodSecurityPolicy settings](https://kubernetes.io/docs/concepts/policy/pod-security-policy/#what-is-a-pod-security-policy) | | | priorityClassName | Set pod scheduling priority class for Kong pods | `""` | | secretVolumes | Mount given secrets as a volume in Kong container to override default certs and keys. | `[]` | | securityContext | Set the securityContext for Kong Pods | `{}` | | containerSecurityContext | Set the securityContext for Containers | See values.yaml | | serviceMonitor.enabled | Create ServiceMonitor for Prometheus Operator | `false` | | serviceMonitor.interval | Scraping interval | `30s` | | serviceMonitor.namespace | Where to create ServiceMonitor | | | serviceMonitor.labels | ServiceMonitor labels | `{}` | | serviceMonitor.targetLabels | ServiceMonitor targetLabels | `{}` | | serviceMonitor.honorLabels | ServiceMonitor honorLabels | `{}` | | serviceMonitor.metricRelabelings | ServiceMonitor metricRelabelings | `{}` | | extraConfigMaps | ConfigMaps to add to mounted volumes | `[]` | | extraSecrets | Secrets to add to mounted volumes | `[]` | | nameOverride | Replaces "kong" in resource names, like "RELEASENAME-nameOverride" instead of "RELEASENAME-kong" | `""` | | fullnameOverride | Overrides the entire resource name string | `""` | | extraObjects | Create additional k8s resources | `[]` | **Note:** If you are using `deployment.hostNetwork` to bind to lower ports ( < 1024), which may be the desired option (ports 80 and 433), you also need to tweak the `containerSecurityContext` configuration as in the example: ```yaml containerSecurityContext: # run as root to bind to lower ports capabilities: add: [NET_BIND_SERVICE] runAsGroup: 0 runAsNonRoot: false runAsUser: 0 ``` **Note:** The default `podAnnotations` values disable inbound proxying for Kuma and Istio. This is appropriate when using Kong as a gateway for external traffic inbound into the cluster. If you want to use Kong as an internal proxy within the cluster network, you should enable inbound the inbound mesh proxies: ```yaml # Enable inbound mesh proxying for Kuma and Istio podAnnotations: kuma.io/gateway: disabled traffic.sidecar.istio.io/includeInboundPorts: "*" ``` #### The `env` section The `env` section can be used to configured all properties of Kong. Any key value put under this section translates to environment variables used to control Kong's configuration. Every key is prefixed with `KONG_` and upper-cased before setting the environment variable. Furthermore, all `kong.env` parameters can also accept a mapping instead of a value to ensure the parameters can be set through configmaps and secrets. An example: ```yaml kong: env: # load PG password from a secret dynamically pg_user: kong pg_password: valueFrom: secretKeyRef: key: kong name: postgres nginx_worker_processes: "2" ``` For complete list of Kong configurations please check the [Kong configuration docs](https://docs.konghq.com/latest/configuration). > **Tip**: You can use the default [values.yaml](values.yaml) #### The `customEnv` section The `customEnv` section can be used to configure all custom properties of other than Kong. Any key value put under this section translates to environment variables that can be used in Kong's plugin configurations. Every key is upper-cased before setting the environment variable. An example: ```yaml kong: customEnv: api_token: valueFrom: secretKeyRef: key: token name: api_key client_name: testClient ``` #### The `extraLabels` section The `extraLabels` section can be used to configure some extra labels that will be added to each Kubernetes object generated. For example, you can add the `acme.com/some-key: some-value` label to each Kubernetes object by putting the following in your Helm values: ```yaml extraLabels: acme.com/some-key: some-value ``` ## Kong Enterprise Parameters ### Overview Kong Enterprise requires some additional configuration not needed when using Kong Open-Source. To use Kong Enterprise, at the minimum, you need to do the following: - Set `enterprise.enabled` to `true` in `values.yaml` file. - Update values.yaml to use a Kong Enterprise image. - Satisfy the two prerequisites below for Enterprise License and Enterprise Docker Registry. - (Optional) [set a `password` environment variable](#rbac) to create the initial super-admin. Though not required, this is recommended for users that wish to use RBAC, as it cannot be done after initial setup. Once you have these set, it is possible to install Kong Enterprise, but please make sure to review the below sections for other settings that you should consider configuring before installing Kong. Some of the more important configuration is grouped in sections under the `.enterprise` key in values.yaml, though most enterprise-specific configuration can be placed under the `.env` key. ### Prerequisites #### Kong Enterprise License Kong Enterprise 2.3+ can run with or without a license. If you wish to run 2.3+ without a license, you can skip this step and leave `enterprise.license_secret` unset. In this case only a limited subset of features will be available. Earlier versions require a license. If you have paid for a license, but you do not have a copy of yours, please contact Kong Support. Once you have it, you will need to store it in a Secret: ```bash kubectl create secret generic kong-enterprise-license --from-file=license=./license.json ``` Set the secret name in `values.yaml`, in the `.enterprise.license_secret` key. Please ensure the above secret is created in the same namespace in which Kong is going to be deployed. #### Kong Enterprise Docker registry access Kong Enterprise versions 2.2 and earlier use a private Docker registry and require a pull secret. **If you use 2.3 or newer, you can skip this step.** You should have received credentials to log into docker hub after purchasing Kong Enterprise. After logging in, you can retrieve your API key from \ \> Edit Profile \> API Key. Use this to create registry secrets: ```bash kubectl create secret docker-registry kong-enterprise-edition-docker \ --docker-server=hub.docker.io \ --docker-username= \ --docker-password= secret/kong-enterprise-edition-docker created ``` Set the secret names in `values.yaml` in the `image.pullSecrets` section. Again, please ensure the above secret is created in the same namespace in which Kong is going to be deployed. ### Service location hints Kong Enterprise add two GUIs, Kong Manager and the Kong Developer Portal, that must know where other Kong services (namely the admin and files APIs) can be accessed in order to function properly. Kong's default behavior for attempting to locate these absent configuration is unlikely to work in common Kubernetes environments. Because of this, you should set each of `admin_gui_url`, `admin_gui_api_url`, `proxy_url`, `portal_api_url`, `portal_gui_host`, and `portal_gui_protocol` under the `.env` key in values.yaml to locations where each of their respective services can be accessed to ensure that Kong services can locate one another and properly set CORS headers. See the [Property Reference documentation](https://docs.konghq.com/enterprise/latest/property-reference/) for more details on these settings. ### RBAC You can create a default RBAC superuser when initially running `helm install` by setting a `password` environment variable under `env` in values.yaml. It should be a reference to a secret key containing your desired password. This will create a `kong_admin` admin whose token and basic-auth password match the value in the secret. For example: ```yaml env: password: valueFrom: secretKeyRef: name: kong-enterprise-superuser-password key: password ``` If using the ingress controller, it needs access to the token as well, by specifying `kong_admin_token` in its environment variables: ```yaml ingressController: env: kong_admin_token: valueFrom: secretKeyRef: name: kong-enterprise-superuser-password key: password ``` Although the above examples both use the initial super-admin, we recommend [creating a less-privileged RBAC user](https://docs.konghq.com/enterprise/latest/kong-manager/administration/rbac/add-user/) for the controller after installing. It needs at least workspace admin privileges in its workspace (`default` by default, settable by adding a `workspace` variable under `ingressController.env`). Once you create the controller user, add its token to a secret and update your `kong_admin_token` variable to use it. Remove the `password` variable from Kong's environment variables and the secret containing the super-admin token after. ### Sessions Login sessions for Kong Manager and the Developer Portal make use of [the Kong Sessions plugin](https://docs.konghq.com/enterprise/latest/kong-manager/authentication/sessions). When configured via values.yaml, their configuration must be stored in Secrets, as it contains an HMAC key. Kong Manager's session configuration must be configured via values.yaml, whereas this is optional for the Developer Portal on versions 0.36+. Providing Portal session configuration in values.yaml provides the default session configuration, which can be overridden on a per-workspace basis. ```bash cat admin_gui_session_conf ``` ```json {"cookie_name":"admin_session","cookie_samesite":"off","secret":"admin-secret-CHANGEME","cookie_secure":true,"storage":"kong"} ``` ```bash cat portal_session_conf ``` ```json {"cookie_name":"portal_session","cookie_samesite":"off","secret":"portal-secret-CHANGEME","cookie_secure":true,"storage":"kong"} ``` ```bash kubectl create secret generic kong-session-config --from-file=admin_gui_session_conf --from-file=portal_session_conf ``` ```bash secret/kong-session-config created ``` The exact plugin settings may vary in your environment. The `secret` should always be changed for both configurations. After creating your secret, set its name in values.yaml in `.enterprise.rbac.session_conf_secret`. If you create a Portal configuration, add it at `env.portal_session_conf` using a secretKeyRef. ### Email/SMTP Email is used to send invitations for [Kong Admins](https://docs.konghq.com/enterprise/latest/kong-manager/networking/email) and [Developers](https://docs.konghq.com/enterprise/latest/developer-portal/configuration/smtp). Email invitations rely on setting a number of SMTP settings at once. For convenience, these are grouped under the `.enterprise.smtp` key in values.yaml. Setting `.enterprise.smtp.disabled: true` will set `KONG_SMTP_MOCK=on` and allow Admin/Developer invites to proceed without sending email. Note, however, that these have limited functionality without sending email. If your SMTP server requires authentication, you must provide the `username` and `smtp_password_secret` keys under `.enterprise.smtp.auth`. `smtp_password_secret` must be a Secret containing an `smtp_password` key whose value is your SMTP password. By default, SMTP uses `AUTH` `PLAIN` when you provide credentials. If your provider requires `AUTH LOGIN`, set `smtp_auth_type: login`. ## Prometheus Operator integration The chart can configure a ServiceMonitor resource to instruct the [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator) to collect metrics from Kong Pods. To enable this, set `serviceMonitor.enabled=true` in `values.yaml`. Kong exposes memory usage and connection counts by default. You can enable traffic metrics for routes and services by configuring the [Prometheus plugin](https://docs.konghq.com/hub/kong-inc/prometheus/). The ServiceMonitor requires an `enable-metrics: "true"` label on one of the chart's Services to collect data. By default, this label is set on the proxy Service. It should only be set on a single chart Service to avoid duplicate data. If you disable the proxy Service (e.g. on a hybrid control plane instance or Portal-only instance) and still wish to collect memory usage metrics, add this label to another Service, e.g. on the admin API Service: ``` admin: labels: enable-metrics: "true" ``` ## Argo CD Considerations The built-in database subchart (`postgresql.enabled` in values) is not supported when installing the chart via Argo CD. Argo CD does not support the full Helm lifecycle. There is no distinction between the initial install and upgrades. Both operations are a "sync" in Argo terms. This affects when migration Jobs execute in database-backed Kong installs. The chart sets the `Sync` and `BeforeHookCreation` deletion [hook policies](https://argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/) on the `init-migrations` and `pre-upgrade-migrations` Jobs. The `pre-upgrade-migrations` Job normally uses Helm's `pre-upgrade` policy. Argo translates this to its `PreSync` policy, which would create the Job before all sync phase resources. Doing this before various sync phase resources (such as the ServiceAccount) are in place would prevent the Job from running successfully. Overriding this with Argo's `Sync` policy starts the Job at the same time as the upgraded Deployment Pods. The new Pods may fail to start temporarily, but will eventually start normally once migrations complete. ## Seeking help If you run into an issue, bug or have a question, please reach out to the Kong community via [Kong Nation](https://discuss.konghq.com). Please do not open issues in [this](https://github.com/helm/charts) repository as the maintainers will not be notified and won't respond.