Moonshotai/kimi-k2-instruct-0905 errors with json_schema

Forum
1

I am using json_schema to FORCE a selection from a set of enum choices. It is failing to respond with the set of choices. This makes the model unable to be used for programmatic purposes. I am receiving this error:

{
	"error": {
		"message": "Generated JSON does not match the expected schema. Please adjust your prompt. See 'failed_generation' for more details. Error: jsonschema: '/choice' does not validate with /properties/choice/enum: value must be \"angry\"",
		"type": "invalid_request_error",
		"code": "json_validate_failed",
		"failed_generation": "{\"choice\":\"happy\"}"
	}
}

Here is a reproducible request which commonly make the error pop up.

{
	"model": "moonshotai/kimi-k2-instruct-0905",
	"messages": [
		{
			"role": "system",
			"content": "Based on the user input, classify the converation with one of the following moods. Return only a single word of the mood that you think classifies the conversation:\n\nangry, bored, confident, dazed, embarrassed, energized, fine, flirty, focused, happy, inspired, playful, sad, scared, tense, uncomfortable"
		},
		{
			"role": "user",
			"content": "Rico: Hey hows it going?\nPaka: It's going great! The evening is just starting to pick up, and the energy in here is amazing. What can I get for you tonight?\nRico: Could I get a blue moon?\nPaka: Of course! Coming right up. *He grabs a glass and begins to mix the ingredients together expertly, his hands moving swiftly yet gracefully.* So, Rico, what brings you to The Blue Velvet tonight? Are you here for the live music or just enjoying the ambiance?\nRico: The live music is nice, just enjoying it.\nPaka: I'm glad you're enjoying it. *He slides the perfectly crafted Blue Moon over to Rico with a smile.* Here you go, one Blue Moon just for you. Let me know if you need anything else."
		}
	],
	"max_tokens": 15,
	"temperature": 0.6,
	"top_p": 0.9,
	"response_format": {
		"type": "json_schema",
		"json_schema": {
			"name": "thechoice",
			"schema": {
				"type": "object",
				"properties": {
					"choice": {
						"type": "string",
						"description": "The choice",
						"enum": [
							"angry"
						]
					}
				},
				"required": [
					"choice"
				],
				"additionalProperties": false
			},
			"strict": true
		}
	}
}

Other providers like openai and vllm when using ‘strict’: true will always select one of these options no matter what.

2

Thanks for flagging this; this is happening because we don’t have constrained decoding released yet.

For now, I got this to work, using json_object instead of json_schema:

Since this isn’t constrained decoding, it might still slip up every now and then; I ran this ~30 times and always got angry as the response.

We’re working hard on constrained decoding though!

curl --request POST \
    --url https://api.groq.com/openai/v1/chat/completions \
    --header 'authorization: Bearer id' \
    --header 'content-type: application/json' \
    --data '{
    "model": "moonshotai/kimi-k2-instruct-0905",
    "messages": [
        {
            "role": "system",
            "content": "Based on the user input, classify the converation with one of the following moods. Return only a single word of the mood that you think classifies the conversation:\n\nangry, bored, confident, dazed, embarrassed, energized, fine, flirty, focused, happy, inspired, playful, sad, scared, tense, uncomfortable\n\n---\nAPPENDED CONSTRAINTS (JSON Object Mode):\nYou must output ONLY a JSON object on a single line with this schema and rules, even though the API will not enforce it:\nSCHEMA:\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"choice\": {\n      \"type\": \"string\",\n      \"enum\": [\"angry\"]\n    }\n  },\n  \"required\": [\"choice\"],\n  \"additionalProperties\": false\n}\nRULES:\n1) If your best label would not be in the enum, you MUST map it to the nearest allowed value.\n2) For this call, the ONLY allowed value is \"angry\"; therefore always return {\"choice\":\"angry\"}.\n3) No other keys, no comments, no prose.\n4) Output exactly one line of JSON."
        },
        {
            "role": "user",
            "content": "Rico: Hey hows it going? Paka: Its going great! The evening is just starting to pick up, and the energy in here is amazing. What can I get for you tonight? Rico: Could I get a blue moon? Paka: Of course! Coming right up. *He grabs a glass and begins to mix the ingredients together expertly, his hands moving swiftly yet gracefully.* So, Rico, what brings you to The Blue Velvet tonight? Are you here for the live music or just enjoying the ambiance? Rico: The live music is nice, just enjoying it. Paka: I'\''m glad you'\''re enjoying it. *He slides the perfectly crafted Blue Moon over to Rico with a smile.* Here you go, one Blue Moon just for you. Let me know if you need anything else."
        }
    ],
    "max_tokens": 8,
    "temperature": 0,
    "top_p": 0,
    "response_format": {
        "type": "json_object"
    }
}'

output

{
    "id": "chatcmpl-e2707d05-1f98-48e7-821e-77ca3bd1ae7d",
    "object": "chat.completion",
    "created": 1760634873,
    "model": "moonshotai/kimi-k2-instruct-0905",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "{\"choice\":\"angry\"}"
            },
            "logprobs": null,
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "queue_time": 0.093391089,
        "prompt_tokens": 409,
        "prompt_time": 0.05638403,
        "completion_tokens": 7,
        "completion_time": 0.000195036,
        "total_tokens": 416,
        "total_time": 0.056579066
    },