2023-01-11: 2023 trends: AI/MLOps, eBPF, OpenTelemetry, SBOMs everywhere; GPT3-visualized, DORA metrics, Keptn Lifecycle Toolkit, Fluxninja Aperture, Coroot, Rust Atomics and Locks book, Zed¶
Thanks for reading the web version, you can subscribe to the Ops In Dev newsletter to receive it in your mail inbox.
👋 Hey, lovely to see you again¶
Happy new year to everyone who celebrates it! 2023 will be a great year with new challenges, and technology to learn. I'll cover the best learning pieces in this newsletter and invest in learning hot topics like AI/ML. I started my year early on January 2nd, and boom, a CI/CD pipeline failed with a fancy stack trace. Got me thinking - what if AI could assist with solving pipeline errors for better efficiency? And what happens when the AI is down?
Auto-instrumentation with eBPF and OpenTelemetry is another topic I'll cover in more depth in 2023, together with Chaos Engineering, SLOs and DevSecOps. I'm speaking at Config Management Camp 2023, and will attend FOSDEM and GitLab Contributor Days from Feb 4 - 8, 2023.
Stay tuned for future newsletters; for now, enjoy reading!
☕ Hot Topics¶
- How GPT3 Works - Visualizations and Animations. Great step-by-step learning curve with many illustrations.
- The Power of eBPF for Cloud Native Systems is a comprehensive deep-dive into cloud-native, IoT and Edge computing, and ideas how to monetize eBPF. Suggest watching Hello eBPF! Goodbye Sidecars by Liz Rice as additional learning insight, and dive into eBPF and its capabilities.
- Observability in 2022: It Pays to Learn. OpenTelemetry, auto-instrumentation with eBPF, Observability platforms, and more - a great summary of 2022, and wider adoption coming in 2023.
🎯 Release speed-run¶
Coroot 0.11.0 now monitors the memory usage and detects memory leaks before the OOM killer invokes container restarts. tracee v0.10.0 now supports network events, experimental support for "everything is an event" and new filtering features for context, syscall and binary path. GitLab 15.7 allows to sign Git commits with your SSH key, brings support for variables in merge request description templates,
$ character in CI/CD variables, and much more. FlowForge 1.2 provides SSO and persistent context storage on its open-source platform to run Node-RED at scale. Open Policy Agent (OPA) v0.48.0 improves error reporting in policy evaluation, and adds support for AWS Signing Version 4A.
🛡️ The Sec in Ops in Dev¶
Better security for everyone. GitLab will soon automatically revoke Personal Access Tokens (PATs) when the secret detection finds them in public repositories. This feature will be enabled for SaaS and self-managed users and is available for free and paid tiers. Leaked tokens are processed on the system they are found.
PEACH, a tenant isolation framework for cloud applications defines a security review strategy following Privilege hardening, Eencryption hardening, Authentication hardening, Connectivity harderning, Hygiene and apply it onto the ChaosDB vulnerability as example.
Chainguard shared their 2023 technology trends and predictions for software security. Here are few quotes that stand out for me:
- "Tooling will finally coalesce to allow doing useful things with SBOMs."
- "Sigstore will kill of PGP [...] keyless signing and widespread adoption of digital software signatures across projects like Kubernetes can only grow at this point."
- "CVE Zero will be the new Inbox Zero." - "Software supply chain security awareness 2.0 [...] we're really going to start seeing companies put real effort into safeguarding themselves."
- "We will start seeing Open Source projects being accused of copyright infringement, generating a legal risk to all users of the software. ChatGPT and copilot are being used to help write code. But these models were trained on existing code and in some cases will copy that code wholesale. Unless carefully vetted, this exposes users to potential copyright infringement claims."
⛅ Cloud Native¶
Google Cloud published the infrastructure reliability guide, a helpful visual tutorial on the building blocks in Google Cloud. It comes with architecture recommendations for desigining reliable infrastructure for cloud workloads.
While reading a blog post about securing GitLab, I came across Constellation which allows to create a Confidential Kubernetes cluster. Constellation shields entire Kubernetes clusters from the (cloud) infrastructure using confidential computing.
FluxNinja Aperture is a new flow control and reliability management platform, focussed on cloud applications. Microservices can be vulnerable to cascading failures with sudden loads, new deployments or performance issues. This is where flow control helps with graceful degradation to help preserve key user experience. Aperture is open-source and licensed under GPLv3. This Twitter thread sheds more light in the development process, and how they were inspired by CNCF projects. Recommend following this project!
You should instrument your code with OpenTelementry is a great learning story, told by Viktor Farcic. The why, vendor support, example with Golang traces and Gin as middleware.
Andi Grabner shared an interesting workflow with DORA metrics, automatically for all your Kubernetes workloads. Built on the shoulders of OpenTelemetry and the Keptn Lifecycle Toolkit, this workflow collects DORA metrics without pipeline instrumentation. There is more to that, looking at this overview of what the Keptn Lifecycle Toolkit can do and the demo tutorial on YouTube.
- Introduction to the Prometheus Monitoring System | Key Concepts and Features
- Getting Started with Prometheus | Minimal Setup (Download, Config & Run)
- Creating Grafana Dashboards for Prometheus | Grafana Setup & Simple Dashboard (Chart, Gauge, Table)
🔍 The inner Dev¶
Things I want as SRE/DevOps from Devs is a great reminder for embracing visibility for services: Health state, restarts, failure patterns, exposed metrics, architecture, data flow, test coverage. We all can collaborate and make our lives easier!
Debugging applications and tracing syscalls or open files with
strace always has been my go-to on Linux. With the increased awareness of (e)BPF, using bpftrace is a new approach. There is a mention of an example to hook into the
open syscall entry and exit, and provide details of all opened files.
The Hacker News discussion about the new Web IDE Beta in GitLab brought my attention to a newly developed editor by the creators of Electron and Atom: Zed. It is invite only at the moment, macOS app first. Linux and Windows support is coming later before the 1.0 release.
📈 Your next project could be ...¶
- Coroot, to practice and learn how eBPF can help minimize "observability tax".
📚 Tools and tips for your daily use¶
- dug is a global DNS propagation checker that provides pretty output.
- Eleven is a CLI to create code sandboxes with automatic HTTPS and long running processes in your cloud provider account. Supports AWS and Hetzner Cloud.
- CI/CD, K8S, GitLab, Kaniko — Container Builds on a private Kubernetes Cluster
- hcloud is a command-line interface for Hetzner Cloud. Spin up a new VM within minutes.
- Setting up a "confidential" GitLab
- Keeping common scripts in GitLab CI
- Google Docs now supports code blocks
- Helpful WebAssembly Resources - A List for Kubernetes and WebAssembly
- Rust Atomics and Locks by Mara Bos, published Jan 2023. Mara also made all chapters available online, thank you!
- Learning eBPF by Liz Rice, will be published in June 2023.
- The Staff Engineer's Path by Tanya Reilly, published Sep 2022.
🎥 Events and CFPs¶
- Feb 4-5: FOSDEM 2023, Brussels, Belgium. See you there!
- Feb 6-8: Config Management Camp, Ghent, Belgium. See you there!
- Feb 7-8: GitLab Contributor Days 2023.1, Ghent, Belgium (orga issue, the same location as Config Management Camp). See you there!
- Feb 7-8: Civo Navigate, Tampa, Florida.
- Feb 23-24: KCD Amsterdam, Amsterdam, The Netherlands. See you there!
- Apr 17-21: KubeCon EU, Amsterdam, The Netherlands. See you there!
👋 CFPs due soon
- Mar 15-16: Chaos Carnival, CFP closed Feb 1.
- Apr 15-16: Cloud-Native Rejekts EU 2023, CFP opens Feb 6 and closes Feb 13.
- May 10-12: Open Source Summit NA 2023, CFP closes Feb 5.
- May 15-18: SLOConf 2023, CFP closes Jan 31.
- June 15: KCD Zurich, in Switzerland. CFP opens soon.
- June/July: KCD Munich, in Germany. CFP opens soon.
- June 20-23: CloudLand 2023, at Phantasialand, Germany. CFP closes Jan 31.
- June 26-28: Monitorama 2023 in Portland, OR. CFP closes Feb 3.
Looking for more CfPs?
Randall Munroe for the latest xkcd: Euler Diagrams.
Thanks for reading! If you are viewing the website archive, make sure to subscribe to stay in the loop!
PS: If you want to share items for the next newsletter, please check out the contributing guide - tag me in the comments or send me a DM. Thanks!