Skip to content

2024-01-11: 2024 predictions, Ollama Mixtral8x7b, k8sgpt CNCF incubation, ContainerCVE, KCD Munich, QCon London

Thanks for reading the web version, you can subscribe to the Ops In Dev newsletter to receive it in your mail inbox.

👋 Hey, lovely to see you again

Hope you had a relaxing time, and come refreshed and prepared for an exciting 2024. Next month, this newsletter turns 2 years old. Thanks for reading and engaging with the content and my many thoughts. Learning AI/ML in 2023 was the hardest challenge in my career, and my newsletter immensely helped me research, read, and reflect. I hope you learned new insights, too. :) Tip: You can use the web archive search for all things AI.

🌱 The Inner Dev 2024 predictions

I thought about my predictions for 2024, reflecting on the learnings in 2023:

eBPF matures with many tools and platforms on the market. Making a business case with just eBPF will still be hard; I predict more mergers and startup acquisitions will come in 2024.

Innovation with AI will continue to move rapidly, requiring practical learning tutorials and newsletter summaries to stay in the loop. Cloud-native open-source projects like k8sgpt or Ollama will drive innovation and adoption more rapidly, together with hyper-cloud AI services and SDKs.

AI in the DevSecOps lifecycle will move beyond code suggestions. Issue and MR/PR summaries, efficient code reviews, assisted Git commands, and integrated chat prompts avoid context switches and allow more focus time. AI agents that provide functions to query live information, enriching and refining the prompt responses will play a big role in 2024. The ecosystem started its expansion, and AI-powered tools will invest in agent integrations, always one innovation ahead of the competition. WebAssembly helps AI development and becomes the default for high-performance activities, for example, parsing code syntax for code generation (example: GitLab issue for tree-sitter integration).

After "shifting left" everything (security, observability, etc.) towards Dev, it is time to turn things around and optimize workflows with AI and platforms to consume. Developers take advantage of AI-assisted vulnerability explanation, resolution, and root cause analysis, empowering all teams. Naturally, internal developer platforms go hand in hand with platform engineering and developer experience. Reusable workflows and components and an OpenTelemetry-driven standard for CI/CD Observability will play a crucial part here -- moving from "Day 2 CI/CD" to general platform requirements for more efficient DevSecOps pipelines.

Observability for MLOps and LLMs will remain challenging, ensuring high-quality and low-latency services to customers. It requires new tools, experience, and knowledge sharing. I look forward to hearing these (incident) stories in future talks. LLM cost monitoring is the next big thing - not the AWS bill, but the one from *AI provider ...

🤖 The Inner Dev learning AI/ML

Ollama can now run Mixtral 8x7b locally with LlamaIndex. Mistral AI releases open permissive models such as Mixtral 8x7b, a "mixture of experts" model — eight of them, each trained with 7 billion parameters. hence the name. I was fascinated by the example tutorial on loading my tweets into the LLM context and asking specific questions. While learning, I also found a bug in the example code. Everything is documented in a new blog post on dnsmichi.at: Local Ollama running Mixtral LLM with LLamaIndex, loaded with personal tweet context, with a screenshot teaser below.

Ollama Mixtral/Mistral running locally with LLamaIndex and loaded tweets from 2020. Asking how the author feels about community

k8sgpt got accepted as CNCF incubating project, and brings more exciting updates to troubleshoot cloud-native environments, for example a wide range of supported LLM AI backends. Alex Jones also teased interactive debugging in this LinkedIn post video.

Google announced developer access for Gemini Pro, including SDKs and Google AI Studio support. You can start learning and exploring in the GoogleCloudPlatform/generative-ai repository. If you are more into Amazon Q, check out this Twitter/X thread for demos.

I will share my learning curve with AI at QCon London in April 2024, talking about "Efficient DevSecOps Workflows with a Little Help from AI". 2024 also brings a new learning series for GitLab Duo, and learn how to use AI workflows for DevSecOps efficiency. It is challenging and also rewarding when discovering new ways to write less code and ask more refined questions in the chat prompt: Deep-dive into AI-powered workflows in the DevSecOps lifecycle, C++ development with Duo Chat, Code Suggestions, Vulnerability Resolution. You can follow the GitLab Duo Coffee YouTube playlist and GitLab group for more insights. Please reach out if you want to join a coffee chat or share challenges we can take on :-)

Quick bites:

🐝 The Inner Dev learning eBPF

The blog post "Application traffic with eBPF" provides a great learning curve on how to inspect HTTP traffic and parse the data for further processing. Recommend reading "eBPF adventures in networking" before.

Quite bites:

👁️ Observability

Polar Signals announced new languages for memory profiling: Rust. Alongside the existing support for CPU profiling, memory profiling helps to detect memory leaks early before they cause problems. Polar Signals also collaborated with Ubuntu to enable frame pointers by default, starting with Ubuntu 24.04 LTS, helping developers with better tracing and profiling insights by default. The performance loss on 64-bit is claimed to be 1-2% and much higher on 32-bit, where the frame pointers are disabled by default.

Cisco acquires Isovalent, the creators of Cilium and Cilium Tetragon, and eBPF community builders. (announcement, Thomas Graf on LinkedIn). Interesting move by Cisco to strengthen their multi-cloud security portfolio, together with the cloud observability investments acquiring Splunk earlier this year. Cisco AppDynamics provides cloud observability capabilities too. The acquisition also brings thoughts on the future of Cilium, as well as potential changes and resource investments. Cisco is no stranger in the cloud-native community, highlighting eBPF and WebAssembly together, for example, and the announcement reassures the investment into the CNCF projects and landscape. This article by Aeydo (German) highlights different container network interfaces (CNIs) such as Cilium, Calico, Flannel and Weave. One of Cilium's advantages is using eBPF for load-balancing, observability, and network policies.

Incidents should have a post-mortem, including detailed analysis and corrective actions, to avoid future problems. But how do you approach a good playbook and strategy? This article shares insights with an example and template for your environments.

Quick bites:

🛡️ DevSecOps

ContainerCVE provides an online platform to search for vulnerabilities in public container images on Docker Hub. It is powered by Trivy. Can be useful for deciding on container images in CI/CD pipelines, Kubernetes deployment considerations, or just come in handy to not setup the scanning infrastructure and error reporting yourself.

Torii is a new internal developer portal, in active development. Its core concept are catalogs for services, workflows and scorecards, which define the building blocks that developers can use, and platform engineering teams need to provide.

If you are using Golang libraries maintained by Mitchell Hashimoto, review this list to take action to replace, or fork a local copy. Mitchell announced on Twitter/X to archive 15 Go libraries in early 2024.

Dagger shared that they are deprecating CUE SDK support (Twitter/X, blog post), in favor of existing SDKs for Go, Node.JS, Python and Elixir to programmatically build CI/CD pipelines.

Quick wins:

🌤️ Cloud Native

A new Kubernetes pocket guide is available in this LinkedIn post, 42 pages of illustrated architecture and learning insights. Tip: Open the LinkedIn preview in full-screen, and select the download icon in the upper right corner. This will generate a PDF download link.

Building ML Infrastructure with Terraform unveils the building blocks required to run machine-learning workflows, and follows a guided tutorial in Google Cloud.

Docker acquired AtomicJar, the creators of the Testcontainers open-source framework.

Quick wins:

📚 Tools and tips for your daily use

  • bat is a cat(1) clone with syntax highlighting and Git integration. It is written in Rust.
  • tldr pages are a community effort to simplify the beloved man pages with practical examples, on the terminal.
  • Terraform Live Graph Extension for VSCode is a plugin that allows you to generate a live Terraform graph as you code.
  • Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
  • Oncall is a calendar tool designed for scheduling and managing on-call shifts. It is used and developed by LinkedIn.
  • Otterize network mapper maps Kubernetes in-cluster traffic and export as text, intents, or an image.

🔖 Book'mark

🎯 Release speed-run

Kubernetes 1.29 Mandala provides Resource Metrics Endpoint (Stable) for better Observability, Pod Lifecycle Sleep Action (Alpha) to allow delaying the pod termination for debugging, Sidecar Containers (Beta) with better lifecycle management, and more.

Quickwit 0.7 brings Elasticsearch API compatibility, performance boosts, more ways to ingest OpenTelemetry data, and UX improvements for Grafana and Jaeger Tracing.

GitLab 16.7 releases GitLab Duo Code Suggestions in GA, the CI/CD components catalog in Beta, project renaming with container registry enabled, Beta support for OpenTofu, SAST results in MR changes view and more.

🎥 Events and CFPs

👋 CFPs due soon

Looking for more CfPs?

🎤 Shoutouts

A fun way to waste time of scammers: Use ChatGPT to predict questions and answers.

Read the answers to "Why is this program erroneously rejected by three C++ compilers?" :-)

🌐

Thanks for reading! If you are viewing the website archive, make sure to subscribe to stay in the loop! See you next month 🤗

Cheers, Michael

PS: If you want to share items for the next newsletter, just reply to this newsletter, send a merge request, or let me know through LinkedIn, Twitter/X, Mastodon, Blue Sky. Thanks!