Security, trace placement and roadmap
Security boundary
- The vLLM REST API is bound to
127.0.0.1:14041; it is not currently reachable from a separate RAG server. Network access must be explicitly engineered and authorized before cutover. - Grafana listens on a private interface and is intended for use through an approved VPN/private network route. Private bind alone is not a substitute for firewall/network ACL validation, and HTTP alone does not encrypt credentials.
- Prometheus and Node Exporter listen on localhost only.
- Authentication keys and Grafana passwords must not be committed, pasted into tickets, or printed in installer logs.
- Phoenix tracing can collect prompt/response content and retrieved chunks: apply tenant boundaries, retention limits and organization approval before enabling payload capture.
Why Phoenix remains on the RAG side
Phoenix tracks the application pipeline (input, retrieval, context assembly, model call, output and evaluation), rather than only GPU resource health. The earlier DBestRag/Phoenix environment already exists on the application side; a second copy on HomaAi would duplicate storage/operations and could increase sensitivity of captured data. Keep the current Phoenix deployment unless cross-application observability or load justifies a dedicated observability host.
Future trace propagation should link API/RAG requests to their model invocation using trace identifiers; the documented deployed vLLM alone does not provide per-tenant attribution or groundedness evaluation.
Future work (NOT deployed in this run)
- LLMOps dashboard v5 with better visual design and full verification of all live panels.
- Grafana Alerting for queues, latency, missing metrics, VRAM pressure, thermal risk and service downtime.
- API gateway instrumentation for HTTP status groups, request timeouts and tenant-safe usage accounting.
- Quality/groundedness evaluations, longitudinal regression suites and retrieval metrics via RAG + Phoenix.
- Workload-aware capacity forecast, context-length testing, model comparisons and controlled backend cutover.
- Hardened deployment with secrets management, explicit network ACL review, TLS at an authorized internal reverse proxy and backup/recovery procedures.
- Clean-VM end-to-end acceptance of the portable bootstrap.
Documentation validation checklist
- Confirm content and version numbers against live services.
- Ensure no secrets, public/private host addresses or customer/tenant data are present.
- Validate Docusaurus build against the actual repository configuration.
- Confirm mobile rendering, code blocks and Mermaid support (or replace Mermaid with native content).
- Publish through the project's real deployment workflow; this bundle does not publish by itself.
Publication checklist
Public documentation deliberately omits original host addresses, internal hostnames, model API keys, tenant identifiers, password values, collected prompt/response bodies and private traces. Examples use localhost or placeholders. Do not publish any of the five implementation-reference shell scripts without independent review and sanitization: they are not part of this public-ready content directory.
The following paths describe secret storage locations only; do not expose their contents: /etc/gemma12b-auth.yaml and /opt/services/monitoring/grafana.env.
Architecture decisions not yet verified
- The intended blog's Docs plugin state, route base paths, Mermaid availability, tags and author registrations, repository and deploy mechanism must be inspected before inserting links or publishing.
- Grafana on a private IP is not end-to-end network authorization or TLS. Do not expose any administrative or inference interface to the public Internet based on the snippets here.
- The original reported GPU model and VRAM capacity should be reconciled against hardware inventory.
- Do not promote the portable bootstrap to a supported installer before a clean-VM test, reboot/recovery checks, version/revision pin review and rollback exercise.