HomaAi — Technical documentation
Evidence date: 18 November 2026. The original host was reported as deployed and healthy during the observed checks. The portable installer has not passed an end-to-end test on a fresh VM. This site is an engineering case study, not a deployable production runbook.
HomaAi is a self-hosted inference node running google/gemma-4-12B-it in BF16 with vLLM and local monitoring via Prometheus, Grafana, Node Exporter and an NVIDIA SMI textfile collector.
Contents
- Architecture and deployment
- Monitoring and troubleshooting
- Benchmarking and operations
- Security and next steps
Deployment snapshot — 2026-11-18
| Component | Observed / configured |
|---|---|
| OS | Debian 12 (bookworm) |
| GPU | NVIDIA GPU; host evidence labels it RTX 4090 with 49,140 MiB VRAM and driver 620.70.06 (model/memory discrepancy: verify before publication) |
| Model | google/gemma-4-12B-it (BF16) |
| vLLM | 0.32.0; 32,768-token max model length, --max-num-seqs 8 |
| Model API | 127.0.0.1:14041, protected with API key |
| Grafana | 12.2.0, private-interface port 13000 |
| Prometheus | 3.7.3, 127.0.0.1:19090 |
| Node Exporter | 1.9.1, 127.0.0.1:9100 |
| GPU metrics | nvidia-smi textfile collector, 15-second systemd timer |
| Dashboards | v2, v3, v4; final v4 render for every panel still requires UI acceptance |
| Phoenix | Separate RAG-side observability; not part of this deployment |
Publication rule: Never include real server IPs, keys, Grafana admin passwords, internal hostnames, tenant identifiers, raw prompts or traces in public content. Values and URLs here are generic or localhost-only.
Evidence levels
- Observed on original host: service health, authenticated inference, scrape target health, short-prompt load tests and one GPU query through Grafana's query API.
- Generated but not fully UI-accepted: Grafana v4 (88 panels / 78 query targets); verify each visible panel in a browser.
- Candidate only: portable bootstrap script; never describe it as reproducible across clean VMs until independently tested.
- Not deployed in this run: v5 dashboard, alerting, Phoenix-to-vLLM traces, quality evaluation, cross-host RAG access and backend cutover.
Needs verification: The hardware report combines an RTX 4090 label with 49,140 MiB of memory. The hardware SKU and reported memory must be reconciled before claiming a specific GPU model.