#Latency

Prompt caching can cut latency and cost on repeated context by an order of magnitude. Here's how it works and why most teams leave it on the table.