Skip to main content

Forum

The Groq Community

25 Topics
yawnxyz
yawnxyzCommunity Manager
asked in Discussion Forum
👋 Introduce yourself!

Hi everyone,Welcome to the Groq community — we’re so glad you’re here!Say hi to the community, tell us a little about yourselves, like:🌎 Where are you from🧑‍🎨 What do you do🧑‍🍳 What are you building / cooking🥳 What do you like to do for fun?And anything else you can think of!

175035
J
yawnxyz
yawnxyzCommunity Manager
published in Feature Requests
Feature Request Guidelines

How to Submit a Feature RequestClick New Post Select Category → Feature Requests Write your post (template at the bottom if you need it!) Tips for Great Feature RequestsSearch first - Someone might have already asked for this! If you find the feature request you want in a previous post, please upvote it. We will be especially tracking feature requests that get a high amount of upvotes to help us with prioritization.  One feature per post - Keeps the conversation focused so it’s easier to track and upvote. Be specific - For example, instead of "make it faster", try "reduce wait time for audio transcriptions". Explain the why - Tell us what you're trying to do! Understanding your workflow helps us build better features. Add context - What product are you using? How often do you hit this issue? Good Titles Help EveryoneMake your title clear so others can find it:✅ "Add support for PDF uploads in chat" ✅ "Increase character limit for TTS requests" ❌ "Please fix this" ❌ "Suggestion" What Ha

510
yawnxyz
K
kelvincaiNew Participant
asked in Discussion Forum
tool use fail rate increases recently

I noticed the tool use fail more often than before recently while there is no related code changes on our side.  The model is llama-3.3-70b-versatileIt is generating a valid JSON response, but it's wrapping it in XML-like function tags (<function=json>...</function>), which is causing the tool use to fail because it expects pure JSON.A sample error response:url: 'https://api.groq.com/openai/v1/chat/completions',  requestBodyValues: [Object],  statusCode: 400,  responseHeaders: [Object],  responseBody: `{"error":{"message":"Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.","type":"invalid_request_error","code":"tool_use_failed","failed_generation":"\\u003cfunction=json\\u003e{ … }\\u003c/function\\u003e"}}\n`,  isRetryable: false,  data: [Object] 

11
yawnxyz
T
tmarkovNew Participant
asked in Discussion Forum
Combine text generation and tool use

Let’s consider the following example use case:We want to ask a model that supports tool use the following question: “What is the weather like today in the largest city in Japan that doesn’t contain a y? Make sure to explain your reasoning in detail before calling any tools.” We have two tools: get_weather, which takes two parameters: location and date, and returns the corresponding weather forecast. get_current_date, which returns today’s date. I’d like the model to first determine the largest city in Japan without a ‘y’ - hence the chain-of-thought instruction in the prompt. Without it, it often decides to go with Tokyo. Then, it should call the tool to get the current date. Finally, when it has determined the date, it would be able to call the weather tool.In general, I’d like to have a loop of text generation, then a tool call, then perhaps more text generation, then another tool call, etc. And I'd like to be able to follow through and inspect what the model was thinking throughout

333
yawnxyz
gardasio
gardasioNew Participant
asked in Discussion Forum
GROQ KIMI K2 TOOL CALL ISSUES

I was one of the first people to try K2 on Groq via OpenRouter last night, and everything was running smoothly. But this morning, we’ve been running into a lot of issues with tool calling.As you can see from the screenshot below, tons of API calls are being wasted due to failed tool calls. The ones that finish with "tool_calls" are the ones that work — all the others are failing.Just checking if you’re aware of this and if a fix is on the way. Cheers!  

2391
yawnxyz
N
NoahNew Participant
asked in Discussion Forum
Facing issue in use API key!

Hi! everyone, I am new here, and I am facing the issue, and unable to use API Keys, because, it saying only team owner and developer can access this or use the API key, Can anyone please help me please? 

337
yawnxyz
A
acraftNew Member
asked in Feature Requests
Chunked Audio Upload for Speech-to-Text Processing

Hello, I have a feature request:Chunked Audio Upload for Speech-to-Text ProcessingCurrent Limitation:The API currently requires the full audio file to be uploaded before processing, leading to increased latency.Proposed Feature:Add an API endpoint to support chunked audio uploads during recording, allowing processing to begin as audio is being sent.Benefit: Reduces upload latency, improving user experience. This feature would significantly improve real-time processing capabilities and user satisfaction. Please provide feedback on this feature request.Thanks! 

111
yawnxyz
P
petrNew Participant
asked in Discussion Forum
Playground vs API

Hello,I am using the whisper-large-v3 model in Playground and getting great results on one of my audio files. However, when I use the same model through the API, the results are significantly worse. The Turbo model works well, providing the same high-quality results as in Playground.My main question is why the results in Playground and the API don't match. Here is my code...import os from groq import Groq client = Groq() file_path = "/content/denoised (29).m4a"with open(file_path, "rb") as file:transcription = client.audio.transcriptions.create(file=file,model="whisper-large-v3",# prompt="Specify context or spelling",response_format="verbose_json",timestamp_granularities=["word", "segment"],language="uk",temperature=0.0)transcription.to_json()Could you help me understand the reasons for this discrepancy in quality and suggest ways to improve the results when using the API? We're here to help Reply directly to this email or through our Messenger  

251
yawnxyz
S
ShaliniNew Participant
asked in Discussion Forum
Restricted access Problem

Im receiving this message and unable to access anything - Your organization has been restricted due to violating our terms of service. If you believe this was in error, please contact support. You can click "Questions?" in the nav to start a conversation

342
yawnxyz
D
deanlongmireNew Participant
asked in Discussion Forum
Website Auth Broken

I tried creating an account with GH, when it routed me back I had a blank profile page, literally just a black screen. It says I need to be a team owner to generate a key, but the teams page is blank too, I can’t even type into the create organization input boxes. Clicking ‘chat with us’ takes me to /login when I am already logged in. Fair to say I am rather frustrated with the experience so far. Where’s my profile?What am I even searching for? 

513
yawnxyz
C
carygnoelNew Member
asked in Feature Requests
Hi dev community! The Groq team is looking for your input on console.groq.com

Hi Groq devs,My name is Cary and I work on the Groq developer experience team, mostly working on console.groq.com.  I’m always looking to connect with our developers and hear about what you’re building with Groq and how you use the API and Console.  Please reach out with any feedback you might have. What ideas do you have for how we can improve Console.  What is missing for you?  What frustrations do you have?  How can we better enable you to build fast and easy? Thank you!-Cary

272
C
D
DeepscribesNew Participant
asked in Discussion Forum
Whisper taking incredibly long (over 60s), retrying takes less than 10s

Hello,I have a transcription service using whisper to transcribe audios, and I’m really happy with the service so far, however, sometimes the API will transcribe for over 60s (over which the request is stopped), but retrying completes the job in about 5s, see attached logs. All times are in GMT+2.Affected request ids:req_01jynty635fhwrw808qwwcge08 req_01jynty5f9e1vsfryhbthev5bz req_01jynty2y4e1nrz7yyk67pwmde req_01jynty2x0fhra7njk896aztwtSome requests a few days ago:req_01jxmetn46fkgr342pnv2f9v9k req_01jxmbmg4weg0ant4cs9q3w56j (this one had a high TTFT)  As you can see, all audios are <20mins in length. They are all compressed in 16kbps opus format, so they are at most 33.5MBs in size. I haven’t been able to reproduce this error with my own audios so unfortunately I can’t share any, but do let me know if you need some metadata

704
D
E
ExcalibarNew Participant
asked in Discussion Forum
Inquiries about Qwen-32B to be available as a production model

Hi all,Our team is building an AI workflow in Japanese. We are prioritizing response time, but using LLaMA-3.1 8B, which is the smallest model in production env, will dramatically dampen the performance. As a result, we used LLaMA-3.3 70B to find a balence between response time and performance.From our experiments, qwen models turn out to excel among all models, but it’s not feasible to use it in production env (suffering from ~50% server error rate). If qwen models become available in production env, we would definitely use them instead of LLaMA models. So is there a clear roadmap about when will these models be available? Thanks in advance.

842
J
V
VishwaSriramNew Member
posted in Feature Requests
Can i get the models to be hosted inside of india regeion?

We are building agents in the bfsi space and would like to know if there are plans of groq launching on indian endpoints?

432
ozenhati
R
ramendikNew Participant
asked in Discussion Forum
How to use a 10M context window? Rate limit issue

Hello,I would really like to experiment with the 10M token context window of the Llama 4 Scout model. Groq is one of the very few places that offer this model over an API.However, the 300000 TPM limit seems to defeat the purpose. As far as I understand the API is stateless, so I’d have to have several megatokens in a request to use the context window, and the TPM limit will always relect this request no matter how long I wait?If there is a solution please tell me so (maybe there *is* a way to have incremental requests that would let me drip-freed the large context at 300000 TPM?)Alternatively, could it be possible to request a removal of this TPM limit in exchange for a VERY strict RPM limit? I can do with 1 RPM for my idea, which is basically “stuff the entire docset into the context window then ask questions or create code based on the docset”.. I will of course go on the paid tier if this request could be granted. My experiments are not expected to reach large scale anytime soon, th

1511
KTibow
DevRay
DevRayNew Participant
posted in Discussion Forum
👋 Hey Groq Community!

Excited to be here! I'm DevRay, 🔧 Has anyone tried dynamically switching LLM prompts based on user intent (e.g., symptom checker vs. career advice) within a single Groq session? Curious how you structure that without bloating token usage.Looking forward to learning, sharing, and building with everyone!— DevRay

3413
yawnxyz
S
sean_carmodyNew Participant
asked in Feature Requests
Chaining Models Support

Support for chaining multiple model requests into a single API call would be fantastic. Groq’s inference speed means the primary latency bottleneck is network I/O when making sequential model calls. Enabling request chaining would reduce network calls, making things faster and more efficient for workflows that need several model inferences in a row!

1144
S
yawnxyz
yawnxyzCommunity Manager
asked in Discussion Forum
What examples, tutorials, and starters would you like to see?

Hi all,I’ve been cooking away making a Groq Cookbook in the last few days, and I was wondering what examples, tutorials, templates and uses cases you’d like from us?What kind of code samples would make your life 100x easier to help you launch your startup, internal product, or feature?A few categories I’ve heard you might want:Voice agents Support chat agent Chat with documents (from PDFs to CSVs to images) OCR and data extraction from PDFs and imagesIs there anything else you’d like to see?From categories above, is there a very specific nice use case you’d like us to build out?We’re here to build out examples to get you started much, much faster! Just let us know your use case and your needs, and we can build out a template and a tutorial for you to take and run with!(All of our templates and examples will be open source!)

150
yawnxyz
A
aliepNew Participant
asked in Discussion Forum
realtime telephony

Hello everyone,I was checking out the pricing page and noticed ASR (Automatic Speech Recognition) and TTS (Text-to-Speech) listed, but I couldn’t find any information on the real-time service that '@phoney ai' is using through Groq. Has anyone here come across this service or knows more about what Groq offers in terms of real-time capabilities? Any help or insights would be greatly appreciated!

1501
yawnxyz
A
adityagaharawarNew Participant
posted in Discussion Forum
Hi Groq

Hello Community I am Aditya I led Research At WEBSPACEAI and Trying To Solve for technology 

711
yawnxyz
S
softcodaNew Participant
posted in Discussion Forum
introduction

Hi everyone, I’m Justin, a lead Senior Software Engineer at AT Ghana Ltd, a Telco company based in Accra, Ghana. I have 14+ years developing full stack web and mobile apps. 

751
yawnxyz
dev
devNew Participant
posted in Discussion Forum
AutoQuill: best (free forever and open source) voice-to-text app made with Groq

i made the best speech-to-text app on the planet, powered by Groq. completely free and open source.speak your mind.http://getautoquill.com (i’m divyansh (dev) lalwani btw! i study biomedical engineering and applied math at johns hopkins, would love to connect with everyone here because i 🧡 groq) 

521
yawnxyz
yawnxyz
yawnxyzCommunity Manager
posted in Feature Requests
Model Request: Parakeet v2 0.6b Model

(Posted by peacock on the Groq Discord: https://discord.com/channels/1207099205563457597/1370606104966594680) Parakeet v2 0.6b can transcribe 1hr of audio in just 1 second https://huggingface.co/spaces/nvidia/parakeet-tdt-0.6b-v2https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2Only English SupportedIs this on roadmap? 

320
yawnxyz
yawnxyz
yawnxyzCommunity Manager
posted in Feature Requests
Add speech-to-speech model support

(Posted by Aly on the Groq Discord: https://discord.com/channels/1207099205563457597/1376241264101425273)Hi Groq Team,I'd like to request the addition of support for speech-to-speech (or voice-to-voice) models like ultravox on the Groq inference engine.Thanks!

240
yawnxyz
yawnxyz
yawnxyzCommunity Manager
posted in Feature Requests
Add support for logprobs in model API response

(Posted by Verdi on the Groq Discord: https://discord.com/channels/1207099205563457597/1377607663004811294) Looking to use Groq for writing our eval set. We are intending to use a design which is LLM-as-judge in our CD/CI pipeline. However, one of the issues as highlighted in this research (https://arxiv.org/pdf/2303.16634) is when asked a model will bias its "score" answer to be almost-always 3 and a better technique to get more high-quality test is to use the probability distribution instead. This is available when using OpenAI's API directly, but not for models on Groq. I would prefer to use Groq if possible since it's much faster!

190
yawnxyz
Terms & ConditionsCookie settings