I was experimenting with different tool use models today and I noticed that of the production models that support tool use, llama-3.3-70b-versatile seems to be the best model currently supported. I did some testing with some of the preview models (specifically the llama 4 models) and noticed that they performed better. Is there a timeline on when I can expect these models to be upgraded into the production tier? I know preview models are meant for R&D only, not for production environments.
I wanted to ask the same thing: since I’m having problems with JSON output in Llama4 models (but not in Llama3.3) I would like to invest some time in switching my code to structured outputs, but only Llama4 models support it, so if in the end they will not be adopted it won’t be worth the time.
Do you have any estimation is they will ever pass to production? They’ve been more than 3 months in preview mode…
Thanks in advance!
For JSON output in Llama 4 and Kimi K2, we just added structured outputs: https://console.groq.com/docs/structured-outputs, which should give you reliable JSON outputs.
I managed to make it work with Llama4 models (but not yet with openAI oss-gpt-120b and 20b), but I had to switch from langchain-groq (in Python) to the Groq SDK. I adapted the example in the page https://console.groq.com/docs/structured-outputs and it works. With langchain-groq I’ve tried a lot of things but I always kept getting 400 errors.
Thanks for reporting, I’ll take a look at the langchain-groq implementation again and report back.
Thanks so much!
Hi @kai, both Llama 4 Scout and Maverick models are stable, well-provisioned and more than ready for anything you throw at it. Please try it out!