What are the best practices for preserving conversation context ? I am developing a telegram based application, where users might return to a conversation to for eg, refine it / correct previous response, after an indefinite amount of time - time based context cleaning is an obvious straightforward approach but should I also have a separate intent classification llm call before actual query processing to decide whether or not to continue with previous conversation or initiate new conversation ? I have been using kimi-k2 on Groq but for the intent classification maybe something like gpt-oss-20b could suffice - sorry if this is not the right place to ask this question
Hi!
I’ve built a system like this for myself and friends, but it runs on WhatsApp.
I built a simple conversation context system through using Cloudflare Workers and Durable Objects, which can be thought of as a server function with persistent storage; and it’s designed to support use cases like persistent chat: Overview · Cloudflare Durable Objects docs
Basically, I had the Cloudflare Worker do the API calls to Groq as a “conversation router”, and it stores conversation threads within the Durable Object; the WhatsApp client is basically a thin API wrapper around the Worker.
Hope that helps!
Hi - Thanks for sharing your implementation solution - for me the issue wasnt really how to persist the conversation, which for now I am doing on a postgres database, it was more about deciding whether a new message is continuing an earlier conversation, especially when the conversation is happening in a messenger app like whatsapp / telegram - what is working for me right now is making an additional call to a smaller but faster model, before actual query processing, which does an intent classfication - it increases latency and cost slightly but has worked well so far
Oh, I got it! Yeah I usually have a small 3.1b model doing a sort of “decisioning / routing” system too, for every incoming message - but mostly to figure out tool calls and conversation threads too.
I actually don’t know what the answer to this is; I think it’s mostly a stylistic/design choice for classifying previous or new conversations — that choice would probably really affect the entire “vibe” of the assistant itself, whether it chooses to start new convos often (it’ll feel like a support assistant) vs. continue conversations more often (it’ll feel like a confidante)
1 Like