Skip to main content

Security, trace placement and roadmap

Security boundary​

  • The vLLM REST API is bound to 127.0.0.1:14041; it is not currently reachable from a separate RAG server. Network access must be explicitly engineered and authorized before cutover.
  • Grafana listens on a private interface and is intended for use through an approved VPN/private network route. Private bind alone is not a substitute for firewall/network ACL validation, and HTTP alone does not encrypt credentials.
  • Prometheus and Node Exporter listen on localhost only.
  • Authentication keys and Grafana passwords must not be committed, pasted into tickets, or printed in installer logs.
  • Phoenix tracing can collect prompt/response content and retrieved chunks: apply tenant boundaries, retention limits and organization approval before enabling payload capture.

Why Phoenix remains on the RAG side​

Phoenix tracks the application pipeline (input, retrieval, context assembly, model call, output and evaluation), rather than only GPU resource health. The earlier DBestRag/Phoenix environment already exists on the application side; a second copy on HomaAi would duplicate storage/operations and could increase sensitivity of captured data. Keep the current Phoenix deployment unless cross-application observability or load justifies a dedicated observability host.

Future trace propagation should link API/RAG requests to their model invocation using trace identifiers; the documented deployed vLLM alone does not provide per-tenant attribution or groundedness evaluation.

Future work (NOT deployed in this run)​

  • LLMOps dashboard v5 with better visual design and full verification of all live panels.
  • Grafana Alerting for queues, latency, missing metrics, VRAM pressure, thermal risk and service downtime.
  • API gateway instrumentation for HTTP status groups, request timeouts and tenant-safe usage accounting.
  • Quality/groundedness evaluations, longitudinal regression suites and retrieval metrics via RAG + Phoenix.
  • Workload-aware capacity forecast, context-length testing, model comparisons and controlled backend cutover.
  • Hardened deployment with secrets management, explicit network ACL review, TLS at an authorized internal reverse proxy and backup/recovery procedures.
  • Clean-VM end-to-end acceptance of the portable bootstrap.

Documentation validation checklist​

  • Confirm content and version numbers against live services.
  • Ensure no secrets, public/private host addresses or customer/tenant data are present.
  • Validate Docusaurus build against the actual repository configuration.
  • Confirm mobile rendering, code blocks and Mermaid support (or replace Mermaid with native content).
  • Publish through the project's real deployment workflow; this bundle does not publish by itself.

Publication checklist​

Public documentation deliberately omits original host addresses, internal hostnames, model API keys, tenant identifiers, password values, collected prompt/response bodies and private traces. Examples use localhost or placeholders. Do not publish any of the five implementation-reference shell scripts without independent review and sanitization: they are not part of this public-ready content directory.

The following paths describe secret storage locations only; do not expose their contents: /etc/gemma12b-auth.yaml and /opt/services/monitoring/grafana.env.

Architecture decisions not yet verified​

  • The intended blog's Docs plugin state, route base paths, Mermaid availability, tags and author registrations, repository and deploy mechanism must be inspected before inserting links or publishing.
  • Grafana on a private IP is not end-to-end network authorization or TLS. Do not expose any administrative or inference interface to the public Internet based on the snippets here.
  • The original reported GPU model and VRAM capacity should be reconciled against hardware inventory.
  • Do not promote the portable bootstrap to a supported installer before a clean-VM test, reboot/recovery checks, version/revision pin review and rollback exercise.