ric-aux/80-Auxiliary-Functions/helm/mc-stack/charts/logstash/README.md

   1 # Logstash
   2
   3 [Logstash](https://www.elastic.co/products/logstash) is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite “stash.”
   4
   5 ## TL;DR;
   6
   7 ```console
   8 $ helm install stable/logstash
   9 ```
  10
  11 ## Installing the Chart
  12
  13 To install the chart with the release name `my-release`:
  14
  15 ```console
  16 $ helm install --name my-release stable/logstash
  17 ```
  18
  19 ## Uninstalling the Chart
  20
  21 To uninstall/delete the `my-release` deployment:
  22
  23 ```console
  24 $ helm delete my-release
  25 ```
  26
  27 The command removes nearly all the Kubernetes components associated with the
  28 chart and deletes the release.
  29
  30 ## Best Practices
  31
  32 ### Release and tune this chart once per Logstash pipeline
  33
  34 To achieve multiple pipelines with this chart, current best practice is to
  35 maintain one pipeline per chart release. In this way configuration is
  36 simplified and pipelines are more isolated from one another.
  37
  38 ### Default Pipeline: Beats Input -> Elasticsearch Output
  39
  40 Current best practice for ELK logging is to ship logs from hosts using Filebeat
  41 to logstash where persistent queues are enabled. Filebeat supports structured
  42 (e.g. JSON) and unstructured (e.g. log lines) log shipment.
  43
  44 ### Load Beats-generated index template into Elasticsearch
  45
  46 To best utilize the combination of Beats, Logstash and Elasticsearch,
  47 load Beats-generated index templates into Elasticsearch as described [here](
  48 https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html).
  49
  50 On a remote-to-Kubernetes Linux instance you might run the following command to
  51 load that instance's Beats-generated index template into Elasticsearch
  52 (Elasticsearch hostname will vary).
  53
  54 ```
  55 filebeat setup --template -E output.logstash.enabled=false \
  56   -E 'output.elasticsearch.hosts=["elasticsearch.cluster.local:9200"]'
  57 ```
  58
  59 ### Links
  60
  61 Please review the following links that expound on current best practices.
  62
  63 - https://www.elastic.co/blog/structured-logging-filebeat
  64 - https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html
  65 - https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html
  66 - https://www.elastic.co/guide/en/logstash/current/persistent-queues.html
  67
  68 ## Configuration
  69
  70 The following table lists the configurable parameters of the chart and its default values.
  71
  72 |              Parameter      |                    Description                     |                     Default                      |
  73 | --------------------------- | -------------------------------------------------- | ------------------------------------------------ |
  74 | `replicaCount`                  | Number of replicas                                 | `1`                                              |
  75 | `podDisruptionBudget`           | Pod disruption budget                              | `maxUnavailable: 1`                              |
  76 | `updateStrategy`                | Update strategy                                    | `type: RollingUpdate`                            |
  77 | `image.repository`              | Container image name                               | `docker.elastic.co/logstash/logstash-oss`        |
  78 | `image.tag`                     | Container image tag                                | `6.7.0`                                          |
  79 | `image.pullPolicy`              | Container image pull policy                        | `IfNotPresent`                                   |
  80 | `service.type`                  | Service type (ClusterIP, NodePort or LoadBalancer) | `ClusterIP`                                      |
  81 | `service.annotations`           | Service annotations                                | `{}`                                             |
  82 | `service.ports`                 | Ports exposed by service                           | beats                                            |
  83 | `service.loadBalancerIP`        | The load balancer IP for the service               | unset                                            |
  84 | `service.loadBalancerSourceRanges` | CIDR ranges to allow access to load balancer       | unset                                            |
  85 | `service.clusterIP`             | The cluster IP for the service                     | unset                                            |
  86 | `service.nodePort`              | The nodePort for the service                       | unset                                            |
  87 | `service.externalTrafficPolicy` | Set externalTrafficPolicy                          | unset                                            |
  88 | `ports`                         | Ports exposed by logstash container                | beats                                            |
  89 | `ingress.enabled`               | Enables Ingress                                    | `false`                                          |
  90 | `ingress.annotations`           | Ingress annotations                                | `{}`                                             |
  91 | `ingress.path`                  | Ingress path                                       | `/`                                              |
  92 | `ingress.hosts`                 | Ingress accepted hostnames                         | `["logstash.cluster.local"]`                     |
  93 | `ingress.tls`                   | Ingress TLS configuration                          | `[]`                                             |
  94 | `logstashJavaOpts`              | Java options for logstash like heap size           | `"-Xmx1g -Xms1g"`                                |
  95 | `resources`                     | Pod resource requests & limits                     | `{}`                                             |
  96 | `priorityClassName`             | priorityClassName                                  | `nil`                                            |
  97 | `nodeSelector`                  | Node selector                                      | `{}`                                             |
  98 | `tolerations`                   | Tolerations                                        | `[]`                                             |
  99 | `affinity`                      | Affinity or Anti-Affinity                          | `{}`                                             |
 100 | `podAnnotations`                | Pod annotations                                    | `{}`                                             |
 101 | `podLabels`                     | Pod labels                                         | `{}`                                             |
 102 | `extraEnv`                      | Extra pod environment variables                    | `[]`                                             |
 103 | `extraInitContainers`           | Add additional initContainers                      | `[]`                                             |
 104 | `podManagementPolicy`          | podManagementPolicy of the StatefulSet              | `OrderedReady`                                   |
 105 | `livenessProbe`                 | Liveness probe settings for logstash container     | (see `values.yaml`)                              |
 106 | `readinessProbe`                | Readiness probe settings for logstash container    | (see `values.yaml`)                              |
 107 | `persistence.enabled`           | Enable persistence                                 | `true`                                           |
 108 | `persistence.storageClass`      | Storage class for PVCs                             | unset                                            |
 109 | `persistence.accessMode`        | Access mode for PVCs                               | `ReadWriteOnce`                                  |
 110 | `persistence.size`              | Size for PVCs                                      | `2Gi`                                            |
 111 | `volumeMounts`                  | Volume mounts to configure for logstash container  | (see `values.yaml`)                              |
 112 | `volumes`                       | Volumes to configure for logstash container        | []                              |
 113 | `terminationGracePeriodSeconds` | Duration the pod needs to terminate gracefully     | `30`
 114 | `exporter.logstash`             | Prometheus logstash-exporter settings              | (see `values.yaml`)                              |
 115 | `exporter.logstash.enabled`     | Enables Prometheus logstash-exporter               | `false`                                          |
 116 | `elasticsearch.host`            | ElasticSearch hostname                             | `elasticsearch-client.default.svc.cluster.local` |
 117 | `elasticsearch.port`            | ElasticSearch port                                 | `9200`                                           |
 118 | `config`                        | Logstash configuration key-values                  | (see `values.yaml`)                              |
 119 | `patterns`                      | Logstash patterns configuration                    | `nil`                                            |
 120 | `files`                         | Logstash custom files configuration                | `nil`                                            |
 121 | `binaryFiles`                   | Logstash custom binary files                       | `nil`                                            |
 122 | `inputs`                        | Logstash inputs configuration                      | beats                                            |
 123 | `filters`                       | Logstash filters configuration                     | `nil`                                            |
 124 | `outputs`                       | Logstash outputs configuration                     | elasticsearch                                    |
 125 | `securityContext.fsGroup`                          | Group ID for the container                                                                   | `1000`                                                  |
 126 | `securityContext.runAsUser`                        | User ID for the container                                                                    | `1000`                                                  |