2022-06-13: KubeCon EU insights, OpenTelemetry Metrics GA, eBPF+Tetragon, ArticDB and Keptn CNCF graduated to incubating¶
Thanks for reading the web version, you can subscribe to the Ops In Dev newsletter to receive it in your mail inbox.
👋 Hey, lovely to see you again¶
May has been very busy, conference driven development like Chris Aniszczyk said, my talk slides (and DNS chaos demo, yay!) amongst many exciting ideas for KubeCon EU. Meeting everyone in-person for the first time was amazing! My recap got delayed by Covid19, so this newsletter issue comes a bit later than planned, feeling much better now. The good thing is that everyone already published their KubECon EU analysis, and the video recordings are up on YouTube, with me linking the best resources for you here :)
That said, I hope you enjoy reading the extended issue :-)
☕ Hot Topics¶
- What SRE Could Be: How do we get to SRE 2.0? by Niall Murphy
- How Much Observability Is Enough? by Dotan Horovits and Jujhar Singh
🎯 Release speed-run: kube-state-metrics v2.50 brings more metrics for standard components, experimental feature for own CRD creation, etc., Prometheus v2.36.0 brings service discovery for Ionos Cloud and Vultr, OpenSearch 2.0 brings document-level alerting, and a new notification system, Kyverno v1.7.0 with Mutation/Generation, GitOps and Image Verification support, Dockerfile 1.4 supports multiple build contexts, GitLab 15.0 ships nested env variables for environments, Terraform registry CI/CD templates, agent server for Kubernetes enabled by default in the Helm chart, etc.
☸️ KubeCon EU¶
KubeCon EU brought many great insights and will take a while to unpack. In between the great talks and conversations we had, I am personally very excited about the new Environmental Conversation/Sustainability WG. Below are a few more summaries worthwhile to read:
- CloudNative Nordics summary video
- My Cloud Native Developer Diary: KubeCon EU by Edidiong Asikpo
- Daniel Bryant: My top five takeaways from #KubeCon Twitter thread
- LitmusChaos at KubeCon EU 2022
- How what we learned at KubeCon EU 2022 will impact our product roadmaps
Recommend checking out the talk playlists, and bookmark interesting session to watch later: KubeCon EU, eBPF day, SecurityCon. WASM day, GitOpsCon. Here are few selected talks with focus on Ops and Observability:
- Effective Disaster Recovery: The Day We Deleted Production
- From Kubernetes to PaaS to … Err, What’s Next?
- OpenTelemetry: The Road Ahead + Meet the Community
- Jaeger: Present and Future
- Prometheus Sparse High-Resolution Histograms in Action, related Twitter thread from Julius Volz
- Distributing PromQL for Fast and Efficient Kubernetes Fleet Monitoring
- Metrics as a First-Class Citizen in the E2E Testing Landscape
- Warp-Speed Debugging with Prometheus Exemplars
- How to Be 10x SRE? A Deep Dive to Prometheus Operator
I've shared more insights into my talk "From Monitoring to Observability: Left Shift your SLOs with Chaos" and KubeCon EU experience in this blog post.
🛡️ The Sec in Ops in Dev¶
Ever wondered about JWT (JSON Web Tokens), authorization workflows, and identity providers? Read no further in this blog post. Great diagram and workflow!
MongoDB announced "Queryable Encryption", an interesting new concept without the need to re-architect existing data sets. The MongoDB drivers analyze the query, and whenever an encrypted key is hit, it requests the encryption keys from AWS/GCP KMS, Azure Key Vault, or HashiCorp Vault for example. The encrypted query data is run by Queryable Encryption on encrypted data, and only on success, the results are returned to the driver which decrypts the data for the client. Sounds really nifty, I'm very curious about the performance and challenges with high availability/replication here.
Terraform as part of the software supply chain, Part 1 - Modules and Providers dives deep into security and potential problems with IaC tools and frameworks.
Merge+Diff: Building DAGs more efficiently: MergeOp and DiffOp are two new features released in BuildKit v0.10. These operations let you assemble container images by composing filesystems (MergeOp) and splitting them apart (DiffOp), all while minimizing the creation of duplicated data both locally on disk and in remote registries.
"Using the MaterializedPostgreSQL database engine, you can replicate an entire PostgreSQL database (or a subset of schemas and tables) into a ClickHouse database" - great blog post by Clickhouse.
⛅ Cloud Native¶
Isovalent open-sourced Tetragon as a new Cilium component that enables real-time, eBPF security observability and runtime enforcement. Recommend watching the eBPF day keynote at KubeCon EU, where Thomas Graf also explains the basics and future of eBPF in Cloud Native.
Congrats to Keptn, becoming a CNCF incubating project. It's been a great journey, seeding Keptn evolve as a quality gate for CI/CD to an Observability platform for Continuous Delivery. We had a first look in 2020 in the 9. #EveryoneCanContribute cafe meetup.
Kubernetes 1.24 brings a breaking change: ServiceAccounts to longer auto-generate a secret (Twitter thread)
Should I Run Postgres on Kubernetes? brings interesting takeaways, like "The biggest reason is to simplify your development and deployment picture by putting all application components on Kubernetes. It supplies a whole set of scaffolding to make deploying and integrating applications and databases easier, including shared secrets, universal discovery, load balancing, service monitoring, and scaling."
OpenTelemetry announced GA for metrics at KubeCon EU, which means that the APIs are stable, and we can look into the collector, auto-instrumentation, and much more. There are more suggestions coming this way, such as adding a new JSON log exporter component. A deep dive into OpenTelemetry metrics touches on the getting started questions, provides the architecture, tools/frameworks to use, and much more. Fantastic article!
The KubeCon EU community vote in TAG Observability is very interesting: Add profiling as OpenTelemetry supported event type. After tracing, metrics and log support, this adds the next event types allowing to correlate more Observability data. Great times ahead - building more use cases for Parca, Pyroscope, etc. Or like Frederic Branczyk said: "Do you even Observability if you don't know how many and where page faults are happening?"
PolarSignals announced arcticDB, an embeddable columnar database for Observability. The blog post explains the challenges with lots of profiling data collected with the Parca agent, leveraging eBPF, and dives into the first implementation iteration.
Getting started with eBPF? Cilium, Falco and Pixie are matured projects to run in production, and learn how to use them for your use cases. Bumblebee also brings in a new perspective, helping to build, run and distribute eBPF programs using OCI images. Another great example is Parca for Profiling: at eBPF day at KubeCon EU, the change from C to Rust for more programming safety was a super interesting talk.
Brendan Gregg, author of the Systems Performance book (highly recommended read!) shared helpful resource in the Learning eBPF Tracing: Tutorials and Examples (2019) post.
Jaeger Tracing can now accept the OpenTelemetry protocol directly, allowing trace data sent directly: "With this new capability, it is no longer necessary ... to run the OpenTelemetry Collector in front of the Jaeger backend."
🔍 The inner Dev¶
Scripting with Go dives into new development paradigms, system commands, pipes and error handling. "Everything is a pipeline" is an interesting approach.
Remember the times you were asked to architect or extend a database schema, or just pull data from several SQL tables? "Use a JOIN." Ok, what is the difference between left, right, inner, and outer join? Andreas Martinson shared a mind-blowing new way to visualize SQL joins instead of Venn diagrams. As a reminder, a Venn diagram is using overlapping circles to figure out what common data are shared between tables.
📈 Your next project could be ...¶
- Tracetest - trace-based testing with OpenTelemetry
- Learning Containers From The Bottom Up is a fantastic read on getting started, misunderstandings, limitations and a successful learning path, helped with many images.
- Demystifying the Kubernetes Iceberg
📚 Tools and tips for your daily use¶
- jc - JSON Convert dig command output parser,
jcprovides many more parsers out-of-the-box and works great with
- xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files.
- gron: Make JSON greppable!
- promcheck: A tool to identify faulty Prometheus rules
- sccache is ccache with cloud storage, supporting Rust, C/C++
- Kustomizer is an experimental package manager for distributing Kubernetes configuration as OCI artifacts.
- grpcurl, is a command-line tool that lets you interact with gRPC servers. It's basically curl for gRPC servers.
- kube-monkey is an implementation of Netflix's Chaos Monkey for Kubernetes clusters. It randomly deletes Kubernetes (k8s) pods in the cluster encouraging and validating the development of failure-resilient services.
- How to Troubleshoot Applications on Kubernetes
- 💬 New GitLab community newsletter, recommending signing up.
- 👁️ o11y.eu is a new company by friends and observability experts, they are hiring too!
- 🎤 Enlightning, learn cloud concepts with Whitney and her lightboard.
🎥 Events and CfPs¶
- Jun 29-30: Kubernetes Community days Berlin in Berlin, Germany. Join me there!
- Sep 5-7: Container Days EU in Hamburg, Germany. Join me there!
- Sep 13-16: OS Summit EU in Dublin, Ireland
- Oct 24-28: KubeCon NA in Detroit, Michigan
- Nov 10-11: All Day DevOps, virtual
- Nov 16-17: Continuous Lifecycle / Container Conf in Mannheim, Germany
👋 CfPs due soon
- Oct 25-27: SREcon22 EMEA, CfP due Jun 30
- Oct 24-25: Cloud Native SecurityCon NA 2022, CfP due Jul 25
Looking for more CfPs? Try CFP Land.
Everyone cheered and said hello at KubeCon EU, be it in person or remote. Community spirit at its best :-)
Thanks for reading! If you are viewing the website, make sure to subscribe to stay in the loop!
PS: If you want to share items for the next newsletter, please check out the contributing guide - tag me in tweet replies or send me a DM. Thanks!