Back to Blogs
July 19, 2026

Why I'm Open Sourcing Bouc.io

Open Source Kubernetes AI Assistant GitOps Bouc.io

After several years of building, breaking, and rebuilding on my own time, I'm making the core of Bouc.io open source. Bouc.io started as a personal sandbox to keep my infrastructure skills sharp and grew into something much larger: a full cloud-native stack with GitOps delivery, enterprise-grade authentication, observability, and, over the past year, a working AI assistant platform with long-term memory, tool use, and a terminal client. This post explains what the project is, what I'm publishing, and why I'm doing it now.

Everything lives under the bouc.io GitHub organization. If you've been following my blog posts on FluxCD, chatbots, agents, and Raspberry Pi clusters, you've already seen pieces of this story. The open-source release is the consolidated picture.

What Bouc.io Is

At its heart, Bouc.io is an opinionated reference implementation of how to run modern web and AI workloads on Kubernetes, from TLS certificates and identity through to a multi-service AI assistant that remembers things across conversations. It is by no means perfect, and it doesn't flawlessly follow every single best practice out there, but it is a working sandbox that I used to learn, experiment, and connect the dots between infrastructure and application layers, the kind of "all the pieces together" setup I wished existed when I started.

The stack combines technologies I've written about individually over the past two years:

  • GitOps delivery with FluxCD — declarative cluster state reconciled from Git
  • Service mesh and ingress with Istio, cert-manager, and external-dns
  • Authentication and authorization with Keycloak and OAuth2-Proxy
  • Observability with metrics-server, Prometheus, Grafana, OpenTelemetry, Kiali, and Datadog
  • LLM inference via Ollama deployed on-cluster
  • An AI assistant platform — agent orchestration, semantic memory, memory distillation, admin and portal services, web UIs, and a CLI

It is not a product. It is not production-ready out of the box. It can, however, be used as a starting point for a more production-ready setup — and as a way to learn about both the infrastructure and application layers.

What's in the Release

The open-source release covers the Kubernetes platform layer and its related components, the AI assistant application layer including the inference component using Ollama, and supporting documentation. Below is the full inventory of Git repositories, grouped by area. Each entry links to its GitHub project, extracted from the original GitLab repositories.

Note: this release deliberately excludes GCP Terraform modules (those remain private or separate for now for security reasons) and the Raspberry Pi / Jetson cluster setup (already covered in previous blog posts — here and here).

GitOps

  • fluxcdboucio — FluxCD GitOps manifests that reconcile Kubernetes deployments across local and sandbox environments, wiring together infrastructure components and application Helm releases.

Infrastructure Components

Each repository contains Helm/Kustomize overlays and FluxCD wiring for a specific cluster add-on:

  • cert-manager — Automated TLS certificate provisioning via Let's Encrypt.
  • istio — Service mesh installation, gateways, and traffic management policies.
  • keycloak — Identity provider deployment and realm configuration for the platform.
  • oauth2-proxy — OAuth2 authentication proxy that sits in front of protected applications.
  • external-dns — Automatic DNS record creation for Kubernetes ingresses and services.
  • external-secrets — External Secrets Operator integration for pulling secrets into the cluster.
  • metrics-server — Cluster resource metrics for horizontal pod autoscaling and kubectl top.
  • prometheus — Prometheus monitoring stack for metrics collection and alerting.
  • grafana — Grafana dashboards wired to Prometheus and other data sources.
  • opentelemetry — OpenTelemetry collector and tracing pipeline configuration.
  • kiali — Istio service mesh observability — topology, health, and traffic flows.
  • datadog — Datadog agent and integration overlays for external monitoring.

LLM Inference

  • ollama-chart — Helm chart to deploy Ollama on Kubernetes, serving open-weight models to the assistant platform.

jtop on an NVIDIA Jetson Orin Nano — llama-server serving models with GPU at 88% utilization

AI Assistant Platform — Agent

  • agent-api-server — Node.js backend with a two-phase Planner→Executor agentic loop, tool registry, BullMQ job queues, SSE streaming, and memory integration. The current primary backend.
  • monochrome-agent-ui — React/Vite web UI for conversing with the agent and observing tool execution in real time.
  • agent-api-chart — Helm chart for deploying agent-api-server.
  • agent-chart — Helm chart for deploying monochrome-agent-ui.

Agent workspace — two-pane UI with chat, reasoning plan, and tool activity log

AI Assistant Platform — Chatbot

The chatbot stack is the predecessor to the agent platform. It still runs in the cluster although the agent is more interesting.

Chatbot UI — Qwen3.5 conversation with session history and token usage

AI Assistant Platform — Memory

  • memory-api-server — PostgreSQL + pgvector API for storing, searching, and tiered retrieval of long-term agent memories.
  • memory-distiller — Python/FastAPI pipeline that processes completed conversations, extracts memory candidates, scores them, and stores the worthy ones.
  • monochrome-memory-ui — React/Vite UI for browsing, searching, and managing stored memories.
  • memory-api-chart — Helm chart for memory-api-server.
  • memory-distiller-chart — Helm chart for memory-distiller.
  • memory-chart — Helm chart for monochrome-memory-ui.

Memory Manager — semantic search, categorized memories, and AI-inferred preferences

AI Assistant Platform — Admin & Portal

Admin Console — dashboard with LLM providers, tool assignments, system health, and organization management

Admin Console — Configuration of the LLM providers

CLI

  • agent-cli — Terminal client for the AI assistant — interactive REPL with real-time SSE streaming, tool observation, and Keycloak authentication.

CLI — CLI command line agent capability.

Web

  • monochrome-web-ui — React/Vite public website for the platform, served as www on the cluster.
  • web-chart — Helm chart for monochrome-web-ui.

Documentation

  • getting-started — Bootstrap scripts, prerequisites, and setup guides for cloning and running the platform locally. Still a lot more to add here!
  • postman-setup — Postman collections for exercising the platform APIs.

Why Open Source It?

I can state three reasons.

1. Paying Forward What I Learned

The entire project was, first and foremost, a source of learning for me. I started Bouc.io four years ago because I wanted to stay current with the technologies used by the best infrastructure teams — Kubernetes, GitOps, service meshes, identity systems — and because I kept hitting the same wall: plenty of tutorials for individual pieces, almost nothing that showed how to wire them all together into something that actually runs.

Over two years ago, that learning expanded into AI: building a chatbot, then an agent with memory, then understanding (or at least questioning) how frontier labs might architect these systems. Every layer taught me something. I'd like others who are on a similar path to have a concrete reference they can clone, break, and rebuild the way I did — and provide feedback to improve it.

2. A Harness for Sovereign AI in a Middle-Power World

Two recent scenario exercises have been on my mind: Europe 2031 and AI 2027.

Europe 2031 paints a near-future where Europe's lack of compute capacity, capital, and home-grown AI infrastructure leaves it strategically dependent on American and Chinese providers, with sovereignty eroding one procurement decision at a time. AI 2027 explores a different axis: the pace of AI capability growth itself, and what institutions look like when agentic systems become genuinely capable in a few short years (with some disastrous outcomes).

And very recently, the same team released AI 2040, a scenario where superintelligence is deliberately delayed, AI research is made public, and dozens of organizations around the world operate at the frontier. I'm not sure it changes much of what I'm doing here, but a future where capability is spread across many public, inspectable stacks rather than a few closed ones is exactly the kind of world where projects like this one have a small role to play.

All three scenarios argue, in different ways, that the next few years matter enormously. If middle powers — Canada included, my home country — want any say in how AI is deployed in government, healthcare, or critical infrastructure, they need more than policy documents. They need working stacks they can inspect, adapt, and run on infrastructure they control. Open source is a reasonable strategy for that: it lowers the cost of experimentation and creates shared reference points across jurisdictions.

I'm under no illusion that Bouc.io is the answer to sovereign AI. It's a hobby project built by one person on evenings and weekends. But it is a real, end-to-end harness with a Kubernetes platform, identity, observability, LLM serving, agent orchestration, memory, and a CLI that someone could fork and harden. Maybe that's useful. Maybe it isn't. Who knows?! I'd rather find out by publishing it than by keeping it on my private GitLab group.

3. The Bare Minimum, Wired Together

I also hope that it can provide a starting point for others to build on: Bouc.io shows one version of the bare minimum you'd need to run both a credible Kubernetes platform and a working AI assistant solution.

On the infrastructure side: TLS, DNS, a service mesh, authentication, secrets management, metrics, tracing, and GitOps delivery. On the AI side: an LLM runtime, an orchestration layer with tools, a memory system with vector search, a distillation pipeline, admin controls for model assignment, and multiple client surfaces (web UI, CLI).

Each piece is a standard or off-the-shelf component. What's unusual is having them all connected and deployable from a single GitOps root. If you're evaluating "what would it take to stand up something like this internally?", Bouc.io is my honest answer to that question, with all the warts visible.

And if I'm honest, there's a fourth: I still have plenty of ideas for what to build and learn next, but beyond my own learning I don't know what else to do with the project. So I'm releasing it to everyone and hoping it's useful to someone...

What Comes Next

I'm publishing the repositories as they exist today — working, but very rough around the edges.

A few items I'd like to improve:

  • Unit tests and integration tests are sparse, including within the CI/CD pipelines.
  • Documentation is uneven and I definitely need to add a single script to start the project locally.
  • Some READMEs are still limited.
  • Keycloak and OAuth2-proxy were built using the old Bitnami Helm charts, which are no longer maintained, and hacked together. They need a serious cleanup and refactoring!
  • Keycloak is not configured to be production-ready, including the database and other components.
  • External Secrets usage is limited and needs to be expanded to support more secrets sources (starting with GCP external secrets implementation and Keycloak).
  • References to the GitLab projects are everywhere, which is fine when working off of the main projects but create issues if you refer to the GitHub OSS version. I need to clean that up.
  • Naming convention is ugly, I need to refactor it.
  • Release the small evals script and Rmd report for it
  • Audit the entire codebase!!

I could go on and on...

My hope is that making it public creates pressure for me personally to clean things up, and that others find pieces worth reusing, whether the FluxCD structure, the memory distillation pipeline, or just the Keycloak + OAuth2-Proxy wiring pattern. If you explore the code and have questions, find something broken, or think a piece is genuinely useful, I'd love to hear from you — GitHub issues, LinkedIn, or Mastodon links are below.

Note: If you start digging into the code, you'll notice that the infrastructure layer was started well before the advent of AI assistants. The AI system came much later and was built with a great deal of help from AI tools — at first copy-pasting into ChatGPT, then moving to CLI as well as Lovable — and a lot of trial and error. So yes, it does leverage AI code generation and completion to a great extent. Don't complain about "AI slop"!

AI Usage Disclosure

This document was created with assistance from AI tools. The content has been reviewed and edited by a human. For more information on the extent and nature of AI usage, please contact the author.

× Expanded image