Best approach to maintain rules between sessions

Hi Cornete, use Prompt Caching. Put your ~150-line policy as a static system message at the very start of every request. On cache hits, this reduces latency and provides an input token discount on the cached portion. Do not put any variable data before the rules (for example timestamps or user IDs), since any changing prefix can break cache hits. Keeping a long session and telling the model to “ignore” earlier turns will not reduce token or context usage.

More on docs: https://console.groq.com/docs/prompt-caching

1 Like