Log Forwarding allows you to send all logs of your containers running on GKE to an existing centralized logging solution.
We deploy a Fluent Bit instance that forwards all your pod logs to a predefined address. On this address you need to run a receiver that supports the Fluentd Forward Protocol. Typically this would be Fluent Bit for simple use-cases or Fluentd for more flexibility. Usually this receiver should run within the GKE cluster. If you have the need to forward the logs to an external host, contact us to discuss the options.
Log Forwarding is available as an additional service with nine Managed GKE.
Let us know to which address we should forward the logs to. After we complete the installation of the service, the details will be shown on runway.
Example receiver with Fluentd
Here’s a basic example for a possible receiver that uses Fluentd and sends the logs to stdout. This configuration is just for debugging purposes to show that logs are coming in properly. In a real world scenario the logs should be forwarded to something like a Elasticsearch instance.
This example creates a ConfigMap, a Service and a Deployment in the namespace
logging. All the logs that we send will start with the tag
kube. so you have to create a
match kube.** to select all the logs. With this configuration the receiver address results in
fluentd.logging.svc:24224, which is what you would need to tell us so we can forward the logs to that address.
apiVersion: v1 kind: ConfigMap metadata: name: fluentd-config namespace: logging data: fluent.conf: |- <source> @type forward port 24224 bind 0.0.0.0 </source> <match kube.**> # here you would usually configure something like elasticsearch @type stdout </match> --- apiVersion: v1 kind: Service metadata: name: fluentd namespace: logging spec: selector: app: fluentd ports: - protocol: TCP port: 24224 targetPort: 24224 --- apiVersion: apps/v1 kind: Deployment metadata: name: fluentd namespace: logging spec: replicas: 1 selector: matchLabels: app: fluentd template: metadata: labels: app: fluentd annotations: # we exclude this pod so we don't create a logging loop fluentbit.io/exclude: "true" spec: volumes: - name: fluentd-config configMap: name: fluentd-config containers: - name: fluentd image: fluent/fluentd:v1.9 ports: - containerPort: 24224 volumeMounts: - name: fluentd-config mountPath: /fluentd/etc