Carl Andersson

• tofu-operator: Joins easykubenix and terranix, so Kubernetes and OpenTofu configuration share one NixOS module system and can read each other's values. Runs in the cluster (kopf, kr8s), keeps OpenTofu state in Kubernetes, and configures both the cluster and the applications on it, such as the Keycloak clients that make OIDC work. An approval names the digest of the saved plan, so a stale approval cannot apply a different plan. Providers come from Nix derivations generated for every version of every OpenTofu registry provider on every platform, so it can configure anything the provider ecosystem covers.
• Argo CD at scale: One Application with 623 resources pushed the controller to 2.4 GB and caused etcd timeouts. Split delivery into per-component units: 928 objects across 32 Applications.
• Keycloak as code: Realms, OIDC clients, groups and users as OpenTofu written in Nix, reconciled continuously with state kept in the cluster. Proven end to end with kube-apiserver OIDC login and group claims.
• Cluster provisioning: Five IPv6-only Talos nodes as KubeVirt VMs from a Nix-generated OpenTofu unit. The migration from the old tree gave byte-identical configuration and an empty tofu plan against live state.
• Monitoring you can trust: 36 scrape jobs, 223 alert rules, 0 evaluation errors. A verification tool runs every alert expression and dashboard query against the live backend (35 dashboards), and build-time checks validate dashboard JSON.
• Apply cost: 209 CRDs were 42.6 of about 45 MB of rendered output and added 922 MiB to apiserver memory. A resourceVersion gate cut a steady-state apply of 1,250 objects from 72 s to 28 s.
• Components: Cilium (LB-IPAM, BGP), Traefik with Gateway API, CloudNativePG with barman backups to Ceph RGW, Rook-Ceph (block, file, object), Harbor, External Secrets, oauth2-proxy, Kyverno, Vertical Pod Autoscaler. Modules for AWS, Azure and OpenStack storage and cloud integration.

Kubernetes deployment

Deployed Kubernetes on-premise using RKE2 on Ubuntu with Ansible to consolidate with previously deployed AWS infrastructure. Kubernetes replaced a custom SaltStack based "scheduler" that installed systemd units. Collaborated with application developers to enable control-plane software that uses Kubernetes jobs for heavy media processing workloads

• VMware deployment: Deployed vSphere + vCloud + NSX-T + vSAN stack to replace an unreliable Hyper-V S2D cluster. Customers got self-service via vCloud portal
• Datacenter consolidation: Migrated Dialect conglomerate hosting from InterXion to Skövde datacenter. Live-migrated VMs over stretched L2, then physically relocated and remounted servers to reuse hardware
• Network modernization: Continued L2 removal for remaining edge cases, deployed EVPN on Cumulus Linux (Mellanox + Broadcom hardware) for appropriate L2 stretching, initiated IPv6 rollout
• Backup infrastructure: Deployed Cohesity for VMware and legacy Hyper-V workloads

• Network architecture redesign: Eliminated chronic STP looping issues by migrating customer connectivity from L2 tunneling to L3 routing, requiring subnet migrations across the customer base
• O365 crisis response: When parent company went bankrupt, wrote PowerShell + Puppeteer automation to create admin users and accept new CSP partnerships across 1000+ Microsoft 365 tenants
• Internal tooling: Built PBX queue monitoring with notifications, custom TeamViewer auto-registration installer (before native support existed), HTTP API for physical phone PBX group login/logout
• MikroTik and networking instructor for all Dialect branch offices

Projects in the collection:

• nanopynix - Drive Nix from Python, batteries included
• pynixd - The Nix daemon protocol in Python
• easykubenix - Kubernetes manifests generated via the NixOS module system, plus OpenTofu management: cluster provisioning and day2 (infra on top of what's deployed), with Nix derivations for all OpenTofu registry providers
• nixkube (nix-csi) - CSI driver that mounts the Nix store into pods
• flake-compatish - Reads a flake this project does not own
• user-mode-nixos - NixOS integration tests on User-Mode Linux and QEMU, driven by one Python runner. Both runners behave the same inside and outside the build sandbox (guests get internet only outside it); the UML path needs no KVM.

Each project keeps its own repository and history. The umbrella tool pins them to a known-good set of commits, so cross-project changes build and test together.

Projects in the collection:

• pymux - Terminal multiplexer, arranges several terminals
• ptterm - Draws one terminal with prompt-toolkit
• txterm - Draws one terminal with Textual
• ptyhost - Runs a program and carries its bytes
• pyte - Terminal screen parsing and state
• prompt-toolkit - The toolkit underneath it all, patched for render performance

Test suite: Differential and conformance-driven rather than hand-written expectations. The emulator is judged against six independent implementations (kitty, WezTerm, Alacritty, Ghostty, libvterm, xterm.js) and against borrowed suites—xterm's esctest, vttest, libvterm and Alacritty reference tests—each run both directly and through a pymux pane, so the diff is exactly what the multiplexer adds. Pixel-level checks screenshot a real terminal on its own Wayland display with and without pymux and subtract the images; Hypothesis property tests hold the resize/reflow invariants; every check is a hermetic Nix derivation that keeps the artifacts of a failing run.

Problem: Container images have inefficient layer sharing, fragile build caching, and no intrinsic SBOM. Nix solves these at the package level but needed Kubernetes integration.

How it works:

1. DaemonSet deployment—no node modifications required. Works on managed clusters (EKS, GKE, AKS).
2. Creates a node-local Nix store at /var/lib/nix-csi.
3. For each CSI volume, creates a hardlink view of requested store paths.
4. Mount modes: read-only (direct bind mount, shared inodes) or read-write (overlayfs layer).

Key benefit: Hardlinked files share page cache across pods. When one pod loads a shared library, other pods get cached pages. Container layers can't do this—each container gets separate inodes even with the same base image.

Comparison: Similar concept to nix-snapshotter, but doesn't require containerd modifications. Inspired Flox's "imageless Kubernetes" approach.

Written in Python. In production use in hetzkube.

Why: Helm templates are stringly-typed and hard to debug. Kustomize patches are limited—strategic merge works until it doesn't, JSON patches are verbose. The NixOS module system already solves configuration composition well.

Features:

• Deep merging from multiple sources
• Type checking with clear errors
• Override at any level with mkForce/mkOverride
• Conditional configuration with dependency tracking

Helm compatibility: Renders Helm charts and imports results into the module system. Override rendered resources like any other. Covers most use cases, though some hooks/lifecycle features don't translate.

nix-csi integration: Reference Nix packages directly in manifests—nix-csi automatically identifies and pulls required store paths.

In production use managing all hetzkube deployments: cert-manager, Cilium, kube-prometheus-stack, Keycloak, and applications.

Stack:

• Control plane: ClusterAPI managed CX22 node(s).
• Workers: NixOS nodes rebuilt from scratch on every initialization.
• Networking: Cilium for CNI, ingress, Gateway API and network policies. Full dual-stack (IPv4+IPv6) enforced via Kyverno.
• Custom IPAM: Reuses node IP addresses as LoadBalancer addresses to avoid Hetzner floating IP/LB costs. Each node's /64 IPv6 block is carved up for services. Kyverno enforces IP sharing annotations to maximize IPv4 utilization.
• Storage: hetzner-csi for traveling volumes, local-path-provisioner for node-bound volumes.
• Observability: kube-prometheus-stack (Prometheus, Grafana, Alertmanager).
• Auth: Keycloak OIDC for all! (Kube, Grafana, Headlamp, pgAdmin, ...)

Deployed components:

• cert-manager
• Cilium
• CloudnativePG
• CoreDNS
• external-dns
• external-secrets
• Gateway API
• Headlamp
• Keycloak
• kube-prometheus-stack
• Kyverno
• metrics-server
• pgAdmin
• Vertical Pod Autoscaler

  All manifests generated via easykubenix. Application delivery via nix-csi.
</details>

Architecture: PostgREST in front of Postgres and nothing else serving data; a shared core (client, path naming, three-way merge, render pipeline); the mount runs on trio via pyfuse3; the CLI covers status, diff, push, revert, merge, search, render and preview.

• Saving is not publishing: writes through the mount become private drafts until pushed; push three-way merges against the revision you actually read, so two people editing one page get a conflict instead of a silent lost update (SVN-style commit mode).
• Forbidden is indistinguishable from missing: a page the ACL denies renders, lists and collides exactly like a page that does not exist—the difference between the two is itself a disclosure, and the ACL never makes it.
• Check an ACL by being someone else: read-only impersonation mounts—read-only transaction, ro in the mount options, 0444 files—logged server-side before a byte is served; borrow a membership to preview what a new engineer would see.
• Reads are audited by the request that serves them: document and audit row commit in one transaction, so there is no window where one happens without the other; cache-served opens queue locally and ship later.

Testing: 1,039 tests against real Postgres and PostgREST (nothing mocked), 91% line coverage; the sandbox-safe half runs as a Nix check.

Model: staged state lives in memory; one function writes to disk. A dry-run is not a second code path—it is a run that never called apply. The staged state prints as a unified diff for review; --apply is the only thing that touches disk, and an applied run prints an undo id that reverses the whole thing.

• Scopes, no drift: with pyedit.VFS() opens an independent overlay over disk truth; on exit git merges it (disk is ancestor, parent is ours, scope is theirs). Line numbers never enter it: a conflict is a Collision, never a guess, and rename detection follows an edit to the file's new name.
• git does the paperwork: dry-run and undo patches are git-canonical, written and applied by libgit2 on a mempack—no Python parses a diff on the replay path, so line numbers are exact by construction.
• Syntax gate: staged text is parsed before it is shown—compile() for Python, tree-sitter for seventeen languages (Go, Rust, C, C++, Nix, Bash, ...); applying known-broken syntax refuses without --force. tree-sitter also serves position queries and file outlines; rope does Python renames, a real language server the rest.
• Speaks foreign patches: OpenAI apply_patch envelopes, libgit2-exact git diffs, and unified diffs anchored by search—because an agent's @@ numbers are a suggestion, not a promise. One hunk that will not anchor fails the whole input.

Rails: fail closed, loud errors, a watchdog that kills hung runs and dumps every thread's stack, and stored diffs at 0600. Round-trip tested byte for byte: text, create, delete, binary, symlink, no-EOL.

• pyjj-bindings - Rust/PyO3 crate around jj_lib, arriving as a normal crates.io dependency—no jj monorepo source in the repo at all
• pyjj - pure Python, wraps the bindings in a pythonic API for consumers
• pyjj-cli - CLI on top of pyjj
• pyjjui - Textual TUI on top of pyjj

Split out of the jj monorepo so each project's Nix build is genuinely self-contained—its own directory is its src, full stop.

Problem: WoW factions (Alliance/Horde) had imbalanced PvP populations. On private servers, 80/20 splits meant 30+ minute queues for one faction, instant for the other.

Implementation (C++):

• Reused arena faction field to temporarily assign players to opposite faction during battlegrounds
• Invalidated Player info cache to all BG players
• Faked faction responses in player info cache packets
• Modified scoreboard packets so client UI displayed teams correctly
• Queue modes: simple (first-come-first-served) and item-level balanced (with gear-swap prevention)

Impact: Published publicly—first implementation any server could use. Spread rapidly across the private server community during the 3.3.5a era when faction imbalance was severe. Still in use today.

Blizzard later added crossfaction instances to retail WoW (dungeons/raids first, then battlegrounds). The concept was proven on private servers years earlier.

One of my first real programming projects—modifying a large C++ codebase to solve a problem affecting thousands of players.