Observability

What is Observability?

Observability lets us understand a system from the outside, by letting us ask questions about that system without knowing its inner workings. Furthermore, allows us to easily troubleshoot and handle novel problems and helps us answer the question, “Why is this happening?”

In order to be able to ask those questions of a system, the application must be properly instrumented. That is, the application code must emit signals such as traces, metrics, and logs.

We have used OpenTelemetry that is the mechanism by which application code is instrumented, to help make a system observable.

For all our components that support OpenTelemetry, tracing can be enabled as below:

global:
  tracing:
    enabled: true
    otelExporter:
      endpoint: "https://endpoint_of_tracing_backend"
      headers:
        authorization: "Basic ..."

To set up Honeycomb to export the traces and visualize them, you can follow the below configuration:

global:
  tracing:
    enabled: true
    otelExporter:
      endpoint: https://api.honeycomb.io
      headers:
        x-honeycomb-team: teamUID
        x-honeycomb-dataset: dataset

Depending on the instrumentation backend, you may want to add custom headers. Please refer to thr desired link to get more information about required headers: