2022-05-02: Perses, OpenTelemetry, eBPF, Chainguard, Podman play kube, CloudSeed, KubeCon, SLOConf and more¶
Thanks for reading the web version, you can subscribe to the Ops In Dev newsletter to receive it in your mail inbox.
👋 Hey, lovely to see you again¶
"With great instrumentation comes great observability" -- Michael Hausenblas pretty much sums up what's going on with Observability this month, directly tieing into Security, Cloud Native and DevSecOps team workflows (who owns Observability?). There is so much to learn and educate, and KubeCon EU is just two weeks away - hoping to meet you in person!
In this issue, I'll also share ideas on different learning practices, and how I try to stay up-to-date in between Dev, Sec and Ops. More on CI/CD Observability with GitLab soon at cdCon! Now, let's dive in ...
☕ Hot Topics¶
- Dropbox tested disaster readiness by shutting down a data center
- Google launched Prodcast, an SRE podcast
- A first look into Dagger in the 50. #EveryoneCanContribute cafe meetup, and its features and vision to create a portable dev kit for CI/CD.
🛡️ The Sec in Ops in Dev¶
Chainguard announced the beta release of their first product: Enforce, Software Supply Chain Security for Kubernetes. It comes with a policy agent, CI/CD integration, continuous verification and a so-called evidence lake, powered with CLI and UI access. We're looking forward to diving into Enforce in future #EveryoneCanContribute cafe meetups.
What the NSA and CISA Left Out of Their Kubernetes Hardening Guide dives deep into authentication and authorization, with great practical examples. Limiting access to Kubernetes resources with RBAC is an additional learning resource to bookmark, similar to How to mitigate Kubernetes runtime security threats. The Top 5 Kubernetes Configuration Mistakes - and how to avoid them prompts common mistakes, and I can already say that two of them are mine. The blog post also makes good suggestions to keep your Kubernetes safe and secure. Speaking of security, I've written a German review of the "Hacking Kubernetes" book for heise.de. If you are reading this newsletter in English, the TL;DR is - one of the best books for Kubernetes Security. Get yourself the ebook, and benefit from also learning container runtime security and eBPF.
Two sizes fit most: PostgreSQL and ClickHouse is an interesting read about relational database and column-based models. I see ClickHouse coming up more often, also in discussion with storing Observability (big) data. Worthwhile to follow the topic and project.
⛅ Cloud Native¶
Google Cloud and GitLab are collaborating on a new project CloudSeed. Jason Smith wrote a great blog post, citing the real problems:
Deploying web applications should be easy, like ridiculously easy. Why do I need to become an NGINX/Apache expert and a Linux Admin just to deploy an application? This has been the pain point for many developers and has left people looking for solutions. If you ever struggled with SSH keys in CI/CD variables to deploy a web app to a cloud VM, give CloudSeed a try.
Mike Julian started a great thread on the big differences between GCP, AWS, and Azure. Spoiler: Customer relations.
Project speedrun: Google has donated Istio to CNCF, Open Policy Agent (OPA) released v3.8.0, Prometheus brings 2.35.0 with more security, cosign pushed v1.8.0, Pixie announced that their OSS profiler now supports Java, Amazon EKS now supports Kubernetes 1.22.
The Perses team shared a slide update on the scope and roadmap. As a reminder, Perses is part of the Coredash community, aiming to build a GitOps style dashboard framework, Apache-licensed, after Grafana relicensing to AGPLv3. I highly recommend following the project and staying in the loop with progress with the API, configuration backends, and dashboards.
Dive into eBPF with Cilium in 30 minutes and level up your knowledge on Observability. Bilgin shared what Cilium is to eBPF and the ServiceMesh evolution towards Sidecarless with eBPF with great threads to dive in and learn.
Cisco and AppDynamics shared an Apache Module for OpenTelemetry that enables tracing of incoming requests to the server by injecting instrumentation into the Apache server at runtime. Nginx support is on the to-do list. There also is work underway to add auto-instrumentation for OpenTelemetry and Go using eBPF. Will be interesting to see if it can replace manual code instrumentation to a certain point.
Quick o11y.love 💜
- Elastisys shared their evaluation story for a long-term metrics backend, selecting Thanos amongst InfluxDB, TimescaleDB, Cortex, etc.
- If you are using the Prometheus Operator for Kubernetes Observability, this PR aims to bring support for the Prometheus agent which is currently guarded behind a feature flag for testing.
- Julius Volz shared a great run-through of the Prometheus Alert Manager.
- OpenTelemetry in Kubernetes: Deploying your Collector and Metrics Backend
- Unpacking Observability: How to Choose an Observability Vendor
- Continuous profiling in Kubernetes using Pyroscope
🔍 The inner Dev¶
PyCon keynotes announced a new innovation: py-script, dive into the announcement Twitter thread. It is Python embedded in HTML running in the browser, using WebAssembly compiled with CPython. See more practical examples in this demo video with Markdown.
Looking for different ways to learn a new language? Peeking into projects using Rust has been a great learning experiment for me, with quick try-outs locally in containers, or Gitpod. Ana shared insights into PL/Rust in PostgreSQL for example, and I have seen Terustry, a Terraform provider registry written in Rust, and a deep-drive into Decentralized cluster membership in Rust.
There are also some controversial discussions on language X vs. Y going on; they are a great way to explore language features and actually learn something useful. Lies we tell ourselves to keep using Golang is a long great read, showing the differences between Golang and Rust (when you ignore the ranting parts).
Closing the loop with WebAssembly to learn - maybe by writing a kubectl plugin?
📈 Your next project could be ...¶
Podman has many great features,
podman play kube as an alternative to Docker compose shared in this article is one of them. It allows to feed a Kubernetes YAML manifest to Podman and execute the pod locally. One benefit is that both Kubernetes and Podman can use the YAML configuration, and you do not need to port it into another YAML format with docker-compose.
OpenTelemetry has greatly enhanced its documentation and getting started guides. I've started with the Go SDK and could follow with the first implementation experiments in GitLab Runner already. It is very convenient to know that traces can be printed to stdout thus not requiring any collector daemon or file storage. I also came across this great article about How to use Contexts in Go. The OpenTelemetry Go library heavily uses context propagation, which is helpful to understand on the learning journey.
📚 Tools and tips for your daily use¶
- ValidIaC, online validator for Terraform code, including security scans and cost estimations. It is a fork of ValidKube for Kubernetes YAML manifests validation.
- zq, an easier and faster alternative to jq
- diagrams as code, for example drawing cloud architecture in Python code.
- 3 open source tools for people with learning difficulties
- How to choose the right static site generator
- S3 Manager, a web UI written in Go to manage S3 buckets from any provider.
- Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster.
- Smocker (server mock) is a simple and efficient HTTP mock server.
- ripsecrets is a command-line tool to prevent committing secret keys into your source code.
- Hacking APIs by Corey J. Ball
- Observability Engineering: Achieving Production Excellence by Charity Majors, Liz Fong-Jones and George Miranda
🎥 Events and CfPs¶
- May 4-5: PulumiUp, virtual
- May 9-12: SLOConf, virtual
- May 9-12: o11yfest, virtual
- May 10: dockerCon 2022, virtual
- May 16-20: KubeCon EU, Valencia, Spain - if it is your first KubeCon, David offered help
👋 CfPs due soon
- Oct 4-6: Open Source Automation Days, Munich, Germany, CfP due May 31
- Oct 24-28: KubeCon NA, Detroit, Michigan, CfP due May 27
- Nov 10-11: All Day DevOps, virtual, CfP due May 31
- Nov 16-17: Continuous Lifecycle / ContainerConf, Mannheim, Germany, CfP due May 8
Looking for more CfPs? Try CFP Land.
💡 Join my talks :-)
- "Left shift your SLOs with Chaos" at SLOConf on May 9-12
- "From Monitoring to Observability: Left Shift your SLOs with Chaos" at KubeCon EU on Friday, May 20 - in-person in Valencia, Spain.
- "How we build CI/CD Observability with OpenTelemetry" at cdCon on June 7-8
Shoutout to Zhang Heqing for the Cars vs. Giant Buldge video. A great distraction from fast-moving technology ;-)
2nd shoutout to Patrick Debois for creating an incredibly engaging tweet asking about engineer levels, wrong answers only.
Thanks for reading! If you are viewing the website, make sure to subscribe to stay in the loop!
PS: If you want to share items for the next newsletter, please check out the contributing guide - tag me in tweet replies or send me a DM. Thanks!