Tag: semantic-caching
-
Borrowed confidence is fragile in agentic systems
Production-like evals revealed the retrieval architecture I actually needed and reminded me that confidence in agentic systems has to be earned, not borrowed.
-
Semantic Caching in Production
Repeated user intents can quietly inflate LLM cost and latency. Semantic caching helps, but production use comes with trade-offs.