Thanks, Karsten, for writing up what we discussed recently with OpenTelemetry! Operate First can provide valuable information to the OpenTelemetry and K8s communities. Experimental tracing has been recently added to the kubernetes APIServer and etcd. Also, instrumentation is in the works with CRI-O and the kubelet. It will be necessary to turn on the tracing feature in each component in order to export and collect traces (and in OpenShift, to figure out the best way to do this). This is new to kube and etcd only in the current release so we'd be one of the first if not the first to try out the feature and report back. We will be able to measure performance overhead from trace collection, and determine from which code paths are the most useful places to export and visualize trace information. 

One thing to note, in the apiserver and etcd, as well as upcoming in CRI-O and kubelet, they are instrumented to export Otel trace data rather than logs or metrics (logs and metrics are collected also, but not with OTel). The OpenTelemetry trace APIs are stable and have been for some time, whereas logging is in the works and metrics very recently declared stable. Once exported to the OpenTelemetry collector, traces can then be exported to a variety of backends, such as zipkin and jaeger. 

I've been experimenting with a kubeadm cluster in the last few weeks, and here is a view of some trace data collected from etcd, apiserver, kubelet, and CRI-O. I am far from an expert with any of this so I look forward to working with the community to get everything running correctly!