End-user observability

This guide shows you how to monitor the resources you deploy on your ControlPlane — Crossplane managed resources, Deployments, HelmReleases, and other Kubernetes objects. If you are new to Observability, the Observability Primer is a good read. This guide focuses on the end-user perspective of observability within a ControlPlane. Platform owners can refer to Platform-owner observability for monitoring the platform itself.

How observability fits together

Collect: Gather telemetry from Kubernetes resources, controllers, and workloads. Use open standards — OpenTelemetry for metrics, logs, and traces.

Store: Send telemetry to an OTLP-compatible backend or let Prometheus scrape it. You choose the backend.

Analyze & Alert: Build dashboards, set thresholds, and create alerts so failures surface before users notice.

What to monitor

Start with the signals that have the highest impact:

Signal	Why it matters
Managed resource health	Crossplane Ready and Synced conditions tell you whether provisioned resources are healthy. Without telemetry, errors hide inside cluster objects.
Controller health	Reconcile error rate, duration, and work queue depth detect throttling or scaling issues before they cascade.
Resource inventory	Counts by kind and version support upgrade planning and compliance.
Backup status	Silent backup failures are discovered only when it's too late.
Workload availability	Deployment and workload conditions show whether services are reachable.

Prerequisites

• You have a running ControlPlane.
• You have installed the Metrics Operator service in the ControlPlane.
• You have an OpenTelemetry-compatible backend (any OTLP endpoint) or a Prometheus-compatible scraper.

Export metrics

The operator supports two export modes: push via OpenTelemetry and pull via Prometheus.

OpenTelemetry push

Create a DataSink that points to your OTLP endpoint. Store credentials in a Kubernetes Secret in the same namespace.

Apply to ControlPlane Cluster

data-sink.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: DataSink
metadata:
  name: default
  namespace: metrics-operator-system
spec:
  connection:
    endpoint: "https://metrics.example.com/v1/metrics"
  authentication:
    apiKey:
      secretKeyRef:
        name: metrics-credentials
        key: api-token
---
kind: Secret
apiVersion: v1
metadata:
  name: metrics-credentials
  namespace: metrics-operator-system
type: Opaque
stringData:
  api-token: <api-token>

A metric without dataSinkRef uses the DataSink named default. Set dataSinkRef.name to target a different sink.

The DataSink also supports mTLS authentication:

authentication:
  certificate:
    clientCertSecretKeyRef:
      name: tls-creds
      key: client-cert
    clientKeySecretKeyRef:
      name: tls-creds
      key: client-key
    caCertSecretKeyRef:
      name: tls-creds
      key: ca-cert

Prometheus pull

Prerequisites

• Installed Prometheus Operator

Apply to ControlPlane Cluster

kubectl apply -f https://raw.githubusercontent.com/openmcp-project/metrics-operator/main/config/prometheus/monitor.yaml

The ServiceMonitor scrapes the operator's HTTPS /metrics endpoint. It exposes metrics_operator_resource_count alongside standard controller-runtime metrics.

See Metrics Export for multiple data sinks, troubleshooting, and migration from legacy configurations.

Create metrics

The Metrics Operator provides four resource types:

Kind	What it monitors
Metric	Kubernetes resources in the local cluster
ManagedMetric	Crossplane managed resources (categories crossplane + managed)
FederatedMetric	Kubernetes resources across multiple clusters
FederatedManagedMetric	Crossplane managed resources across clusters

Monitor managed resource health

Crossplane managed resources expose Ready and Synced conditions — without metrics, failures stay hidden inside cluster objects.

managed-resource-condition.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: Metric
metadata:
  name: managed-resource-condition
  namespace: default
spec:
  name: "managed-resource-condition"
  interval: "1m"
  target:
    group: helm.crossplane.io
    version: v1beta1
    kind: Release
  projections:
    - name: ready
      fieldPath: "status.conditions[?(@.type=='Ready')].status"
    - name: synced
      fieldPath: "status.conditions[?(@.type=='Synced')].status"

Monitor HelmRelease status

Track Flux HelmRelease readiness:

helmrelease-status.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: Metric
metadata:
  name: helmrelease-status
  namespace: default
spec:
  name: "helmrelease-status"
  target:
    group: helm.toolkit.fluxcd.io
    kind: HelmRelease
    version: v2
  interval: 1m
  projections:
    - name: hr-name
      fieldPath: metadata.name
      type: primitive
    - name: hr-namespace
      fieldPath: metadata.namespace
      type: primitive
    - name: ready-status
      fieldPath: "status.conditions[?(@.type=='Ready')].status"
      type: primitive

Monitor Deployments with labels

Extract custom labels alongside status conditions. This enables alerts like "fire when a critical Deployment becomes unavailable":

deployment-monitor.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: Metric
metadata:
  name: deployment-monitor
spec:
  name: "deployment-monitor"
  target:
    kind: Deployment
    group: "apps"
    version: v1
  dataSinkRef:
    name: default
  projections:
    - name: available
      fieldPath: "status.conditions[?(@.type=='Available')].status"
    - name: operations
      fieldPath: "metadata.labels.operations"

Resource inventory

Count resources by kind and version for upgrade planning:

resource-inventory.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: Metric
metadata:
  name: resource-inventory
  namespace: default
spec:
  name: "resource-inventory"
  target:
    group: apps
    kind: Deployment
    version: v1
  interval: 1m
  projections:
    - name: count
      fieldPath: "metadata.name"

Backup monitoring

Verify backup completion:

velero-backup-status.yaml

apiVersion: metrics.openmcp.cloud/v1alpha1
kind: Metric
metadata:
  name: velero-backup-status
  namespace: default
spec:
  name: "velero-backup-status"
  target:
    group: velero.io
    kind: Backup
    version: v1
  interval: 1m
  projections:
    - name: backup-completed
      fieldPath: "status.phase"

Collect logs with OpenTelemetry Collector

The OpenTelemetry Collector collects logs, metrics, and traces and exports them to any OTLP-compatible backend.

Deploy the collector as a Deployment or DaemonSet in your ControlPlane. Configure receivers for your signal types:

Receiver	Signal	Description
filelog	Logs	Reads container log files from /var/log/pods
k8sobjects	Logs	Watches Kubernetes Events
hostmetrics	Metrics	CPU, memory, disk, network from the node
kubeletstats	Metrics	Pod and container metrics from kubelet
k8s_cluster	Metrics	Cluster-level resource metrics
otlp	All	Receives OTLP from instrumented applications
prometheus	Metrics	Scrapes Prometheus endpoints

See the OpenTelemetry Collector documentation for the full configuration reference.

Set up alerting

With metrics and logs flowing to your backend, configure alerts based on the signals that matter:

• Managed resource unhealthy: fire when Ready=False or Synced=False persists
• Controller errors: fire when reconcile error rate exceeds threshold
• Backup failure: fire when backup phase is not Completed
• Deployment unavailable: fire when critical Deployments lose availability

Alert configuration depends on your backend. Consult your backend's documentation for threshold rules, notification channels, and escalation policies.

What's next

• Metrics Operator documentation
• Metrics Operator usage examples
• Dimensions configuration — projections, valueFrom, cardinality
• Remote cluster access — federated monitoring
• Platform-owner observability — monitoring the platform itself