Hello. Most providers accept add_generation_prompt and continue_final_message from VLLM/SGLang. Is it possible to support these parameters with Groq? I would be able to migrate my workload to Groq once these are available.
If chat_template and chat_template_kwargs were available, I could accomplish the same functionality of add_generation_prompt and continue_final_message by being able to control the chat template
Thanks, I’ll list this request for our consideration!
I’m curious what your use case is with these args?
EDIT: Currently, we do support prefilling: Prefilling - GroqDocs
The prefilling kind of works, but that means switching between groq and other providers I have to put in custom code for groq execution. If I can control the chat template I can control the world
Ah got it, we’re considering adding continue_final_message but probably not add_generation_prompt