Skip to content
Rafael Vera Marañón Senior Data Engineer & Data Architect

Project

LLM Observer

Open-source MVP for identity-aware LLM FinOps, usage visibility and operational hygiene with metadata-only ingestion by default.

LLM Observer
Context
Open-source and extensible reference implementation documented on Medium and GitHub.
Problem
Companies need to understand who consumes LLM tokens, where cost originates and which usage patterns should improve without exporting prompts or provider credentials.
Solution
Built a customer-controlled LiteLLM gateway, FastAPI ingestion service, identity attribution, pricing and budget controls, deterministic hygiene rules, audit exports and a Docker-based dashboard.
Impact
Validated the local product contract with 38 automated tests and an end-to-end UAT covering identity, virtual keys, metadata ingestion, pricing, budgets, reporting and readiness checks.

Stack

LiteLLMFastAPIPythonPostgreSQLDockerAzureTerraform

Links

Medium article GitHub repository

LLM Observer is an open product foundation rather than a hosted SaaS or an enterprise GA release. Real provider traffic, corporate identity integration and production cloud hardening remain environment-specific validation work.