Skip to main content

HomaAi — Technical documentation

Evidence date: 18 November 2026. The original host was reported as deployed and healthy during the observed checks. The portable installer has not passed an end-to-end test on a fresh VM. This site is an engineering case study, not a deployable production runbook.

HomaAi is a self-hosted inference node running google/gemma-4-12B-it in BF16 with vLLM and local monitoring via Prometheus, Grafana, Node Exporter and an NVIDIA SMI textfile collector.

Contents​

  1. Architecture and deployment
  2. Monitoring and troubleshooting
  3. Benchmarking and operations
  4. Security and next steps

Deployment snapshot — 2026-11-18​

ComponentObserved / configured
OSDebian 12 (bookworm)
GPUNVIDIA GPU; host evidence labels it RTX 4090 with 49,140 MiB VRAM and driver 620.70.06 (model/memory discrepancy: verify before publication)
Modelgoogle/gemma-4-12B-it (BF16)
vLLM0.32.0; 32,768-token max model length, --max-num-seqs 8
Model API127.0.0.1:14041, protected with API key
Grafana12.2.0, private-interface port 13000
Prometheus3.7.3, 127.0.0.1:19090
Node Exporter1.9.1, 127.0.0.1:9100
GPU metricsnvidia-smi textfile collector, 15-second systemd timer
Dashboardsv2, v3, v4; final v4 render for every panel still requires UI acceptance
PhoenixSeparate RAG-side observability; not part of this deployment

Publication rule: Never include real server IPs, keys, Grafana admin passwords, internal hostnames, tenant identifiers, raw prompts or traces in public content. Values and URLs here are generic or localhost-only.

Evidence levels​

  • Observed on original host: service health, authenticated inference, scrape target health, short-prompt load tests and one GPU query through Grafana's query API.
  • Generated but not fully UI-accepted: Grafana v4 (88 panels / 78 query targets); verify each visible panel in a browser.
  • Candidate only: portable bootstrap script; never describe it as reproducible across clean VMs until independently tested.
  • Not deployed in this run: v5 dashboard, alerting, Phoenix-to-vLLM traces, quality evaluation, cross-host RAG access and backend cutover.

Needs verification: The hardware report combines an RTX 4090 label with 49,140 MiB of memory. The hardware SKU and reported memory must be reconciled before claiming a specific GPU model.