Skip to content
qjolyPublic

About

Homelab setup based on Omni and Talos.

Topics

Resources

Stars

114 stars

Watchers

2 watching

Forks

Repository files navigation

Helm LOGO
Typing SVG

Blog Kubernetes Talos Corium

HomeLab

Homelab setup based on Talos, Omni and Corium.

Overview

This repository contains the configuration files for my homelab. The homelab is a collection of servers and services that I run at home or in the cloud. The homelab is used for learning, testing, and hosting projects.

Stack

The home cluster runs Talos (don't hesitate to check a little article I wrote about it), managed by a self-hosted Omni instance. The two OVH servers run Corium (bootc + k0s) and are driven with cctl.

Core Components

  • Omni (Self-hosted) : Manage the Talos nodes (turing).
  • Corium : bootc + k0s image for the OVH servers (mocha, doppio).
  • Cilium as CNI and LB (ARP mode)
  • ArgoCD to manage the GitOps workflow
  • Traefik as Ingress controller (and for middleware management)
  • Cert Manager for TLS certificates
  • External DNS to publish records from Ingresses
  • Storage:
    • Rook (Ceph) on turing, plus csi-driver-nfs for the NAS.
    • OpenEBS on mocha: LVM (openebs-lvmpv by default, openebs-lvmpv-thin) and openebs-hostpath.
    • OpenEBS ZFS on doppio (doppio-zfs).
  • External Secrets to fetch secrets from a remote store
  • OpenBao (via bank-vaults) as secret store, one per cluster
  • Authentik for SSO / forward-auth, and Dex (mocha)
  • SigNoz + OpenTelemetry Operator for metrics, logs and traces
  • CrowdSec to filter traffic hitting the Ingress (mocha)
  • Cloudflare Tunnels to expose services to the internet (turing)
  • restic CronJobs (database dumps, OpenBao snapshots) to S3 (mocha)
  • Spegel to share images between nodes (turing)
  • Reloader to restart workloads when a ConfigMap/Secret changes

Clusters

  • Mocha : single bare-metal node hosted by OVH (128GB RAM, 8 CPU, 2x512GB NVMe), running Corium. Production cluster, also hosts SigNoz. Its node document is mocha/corium/mocha.yaml.
  • Turing : cluster made of small devices (ARM and x86) at home. Local hosting, storage (Rook + NAS) and testing.
  • Doppio : single bare-metal node at OVH, kept for storage and backup. The odd one out — it runs Corium (bootc + k0s) rather than Talos, so it is driven with cctl and has no Omni template. See doppio/README.md.
  • Ristretto : single-node Talos VM on Proxmox, not managed by ArgoCD and reachable through ProxyAuthK8S. See ristretto/README.md.

cortado and the standalone kubevirt cluster have been retired — the old configs live in the git history.

Not Kubernetes

  • Americano : a bare host running podman quadlets, reconciled by Materia instead of ArgoCD. See americano/README.md for the branch trick that makes it work from a subdirectory.
  • external/ : OTel collector and exporter units for machines outside the clusters (Proxmox hosts).

Usage

turing is driven by Omni. Install omnictl, merge the omniconfig downloaded from the Omni instance, then:

omnictl config merge ./omniconfig.yaml
omnictl cluster template sync -f turing/template.yaml
omnictl kubeconfig --cluster turing

mocha and doppio are driven by cctl with their node document, see doppio/README.md for the full procedure:

cctl apply <node> --file mocha/corium/mocha.yaml

On mocha CI does it: a push to mocha/corium/ runs corium-mocha, which upgrades the node to the image in image.yaml (a reboot) and then applies mocha.yaml. Renovate bumps both, so merging its PR rolls the node.

mocha/kubeconfig logs into mocha through Authentik (OIDC).

Recurring operations (Vault unseal and KV access) are mise tasks:

mise tasks          # list them
mise run vault:status

CI/CD

Workflow What it does
omni-template-sync.yaml Syncs the Omni cluster template of turing on every push to main (see below)
materia-americano.yaml Republishes americano/ to the root of the americano branch for Materia
tangled-mirror.yaml Mirrors the repo to tangled
page.yaml Builds and publishes the mkdocs site (docs)
spindle-image.yaml, abcdesktop-image.yaml, atcr-hold-image.yaml Build the container images for a few self-hosted apps

Automatic cluster template sync

The workflow .github/workflows/omni-template-sync.yaml runs omnictl cluster template sync and helmfile apply on every push to main that modifies turing/template.yaml, turing/patches/, turing/helmfile.yaml or turing/helm/. On a pull request it posts the dry-run and the helmfile diff as a comment.

It authenticates against Omni using a service account (not a Kubernetes ServiceAccount — it is an Omni-native token-based credential).

Create the Omni service account (one-time setup)

Run this once from a machine where omnictl is already configured:

omnictl serviceaccount create github-actions --role Operator --ttl 8760h

The command prints an OMNI_ENDPOINT and an OMNI_SERVICE_ACCOUNT_KEY. Store both as GitHub Actions secrets:

Secret name Value
OMNI_ENDPOINT Endpoint printed by the command
OMNI_SERVICE_ACCOUNT_KEY Key printed by the command

The OMNI_SERVICE_ACCOUNT_KEY is shown only once — save it immediately.

Add them at: Repository → Settings → Secrets and variables → Actions → New repository secret


Example of kubeconfig file
apiVersion: v1
kind: Config
clusters:
  - cluster:
      server: https://omni.home.une-tasse-de.cafe:8100/
    name: omni-turing
contexts:
  - context:
      cluster: omni-turing
      namespace: default
      user: omni-turing-quentinj@une-pause-cafe.fr
    name: omni-turing
current-context: omni-turing
users:
- name: omni-turing-quentinj@une-pause-cafe.fr
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
        - oidc-login
        - get-token
        - --oidc-issuer-url=https://omni.home.une-tasse-de.cafe/oidc
        - --oidc-client-id=native
        - --oidc-extra-scope=cluster:turing
      command: kubectl
      env: null
      provideClusterInfo: false

About

Homelab setup based on Omni and Talos.

Topics

Resources

Stars

114 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages