Cut LLM API costs by up to 60% with an OpenTelemetry-native semantic caching proxy and strict budget guardrails.

What’s included:

• Redis-powered dynamic vector semantic caching with adjustable cosine similarity thresholds
• OpenTelemetry native OTLP span and metric export for Grafana, Jaeger, and Prometheus
• Real-time token budget guardrails with automated circuit breaking and per-tenant rate limits
• Zero-code OpenAI-compatible drop-in reverse proxy architecture built on asynchronous FastAPI