完整模型列表和定价请参阅 references/models.md。

常用模式

流式响应

stream = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

系统消息

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello"}
    ]
)

异步客户端 (Python)

import asyncio
from groq import AsyncGroq

async def main():
    client = AsyncGroq()
    response = await client.chat.completions.create(
        model="llama-3.3-70b-versatile",
        messages=[{"role": "user", "content": "Hello"}]
    )
    return response.choices[0].message.content

print(asyncio.run(main()))

JSON 模式

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "List 3 colors as JSON array"}],
    response_format={"type": "json_object"}
)

结构化输出 (JSON 模式)

强制输出匹配模式。提供两种模式：

严格模式（保证合规）：

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    }
)

使用 Pydantic：

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Extract: John is 30"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": Person.model_json_schema()
        }
    }
)
person = Person.model_validate(json.loads(response.choices[0].message.content))

模式要求、验证库和示例请参阅 references/structured-outputs.md。

音频

转录（语音转文本）

with open("audio.mp3", "rb") as f:
    transcription = client.audio.transcriptions.create(
        model="whisper-large-v3-turbo",
        file=f,
        language="en",  # 可选：ISO-639-1 代码
        response_format="verbose_json",  # json, text, verbose_json
        timestamp_granularities=["word", "segment"]
    )
print(transcription.text)

翻译（至英语）

with open("french_audio.mp3", "rb") as f:
    translation = client.audio.translations.create(
        model="whisper-large-v3",
        file=f
    )
print(translation.text)  # 英语文本

文本转语音

response = client.audio.speech.create(
    model="playai-tts",
    input="Hello, world!",
    voice="Fritz-PlayAI",
    response_format="wav",  # flac, mp3, mulaw, ogg, wav
    speed=1.0  # 0.5 到 5
)
response.write_to_file("output.wav")

视觉

使用 Llama 4 多模态模型处理图像。每个请求最多支持 5 张图像。

模型： meta-llama/llama-4-scout-17b-16e-instruct（更快），meta-llama/llama-4-maverick-17b-128e-instruct（更高质量）

来自 URL 的图像

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
        ]
    }]
)

本地图像 (Base64)

import base64

def encode_image(path: str) -> str:
    with open(path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encode_image('photo.jpg')}"}}
        ]
    }]
)

OCR / 以 JSON 格式提取数据

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Extract all text and data as JSON"},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{base64_image}"}}
        ]
    }],
    response_format={"type": "json_object"}
)

多图像处理、图像工具使用和多轮对话请参阅 references/vision.md。

工具使用

工具调用模式和示例请参阅 references/tool-use.md。

快速示例：

import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Weather in Paris?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    for tc in response.choices[0].message.tool_calls:
        args = json.loads(tc.function.arguments)
        # 执行函数并继续对话

内置工具（智能体）

使用 groq/compound 或 openai/gpt-oss-120b 进行内置网络搜索和代码执行：

response = client.chat.completions.create(
    model="groq/compound",
    messages=[{"role": "user", "content": "Search for latest Python news"}]
)
# 模型自动使用网络搜索

MCP（远程工具）

连接到第三方 MCP 服务器以使用 Stripe、GitHub、网络抓取等工具。使用 Responses API：

import openai

client = openai.OpenAI(
    api_key=os.environ.get("GROQ_API_KEY"),
    base_url="https://api.groq.com/openai/v1"
)

response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="What models are trending on Huggingface?",
    tools=[{
        "type": "mcp",
        "server_label": "Huggingface",
        "server_url": "https://huggingface.co/mcp"
    }]
)

MCP 配置和流行服务器请参阅 references/tool-use.md。

推理模型

控制模型如何思考复杂问题。

模型： openai/gpt-oss-20b, openai/gpt-oss-120b（低/中/高），qwen/qwen3-32b（无/默认）

带推理强度的 GPT-OSS

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "How many r's in strawberry?"}],
    reasoning_effort="high",  # low, medium, high
    temperature=0.6,
    max_completion_tokens=1024
)

print(response.choices[0].message.content)
print("Reasoning:", response.choices[0].message.reasoning)

带解析推理的 Qwen3

response = client.chat.completions.create(
    model="qwen/qwen3-32b",
    messages=[{"role": "user", "content": "Solve: x + 5 = 12"}],
    reasoning_format="parsed"  # raw, parsed, hidden
)

print("Answer:", response.choices[0].message.content)
print("Reasoning:", response.choices[0].message.reasoning)

隐藏推理 (GPT-OSS)

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "What is 15% of 80?"}],
    include_reasoning=False  # 在响应中隐藏推理
)

流式处理、带推理的工具使用和最佳实践请参阅 references/reasoning.md。

批处理

用于高容量异步处理（24小时-7天完成窗口）：

# 1. 创建包含请求的 JSONL 文件
# 2. 上传文件
# 3. 创建批次
batch = client.batches.create(
    input_file_id=file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

# 4. 检查状态
batch = client.batches.retrieve(batch.id)
if batch.status == "completed":
    results = client.files.content(batch.output_file_id)

完整的批处理 API 详情请参阅 references/api-reference.md。

提示缓存

通过自动缓存重复的提示前缀，可将延迟和成本降低 50%。无需更改代码。

支持的模型： moonshotai/kimi-k2-instruct-0905, openai/gpt-oss-20b, openai/gpt-oss-120b, openai/gpt-oss-safeguard-20b

工作原理：

• 将静态内容（系统提示、工具、示例）放在开头
• 将动态内容（用户查询）放在末尾
• 缓存自动匹配前缀并应用 50% 折扣
• 缓存 2 小时未使用后过期

跟踪缓存使用情况：

response = client.chat.completions.create(
    model="moonshotai/kimi-k2-instruct-0905",
    messages=[{"role": "system", "content": large_system_prompt}, ...]
)

cached = response.usage.prompt_tokens_details.cached_tokens
print(f"Cached tokens: {cached}")  # 对这些令牌应用了 50% 折扣

优化策略和示例请参阅 references/prompt-caching.md。

内容审核

使用安全防护模型检测和过滤有害内容。

Llama Guard 4

通用内容安全分类。返回 safe 或 unsafe\nSX（类别代码）。

response = client.chat.completions.create(
    model="meta-llama/Llama-Guard-4-12B",
    messages=[{"role": "user", "content": user_input}]
)

if response.choices[0].message.content.startswith("unsafe"):
    # 阻止或处理不安全内容
    pass

GPT-OSS Safeguard 20B

使用自定义策略进行提示注入检测。返回结构化 JSON。

response = client.chat.completions.create(
    model="openai/gpt-oss-safeguard-20b",
    messages=[
        {"role": "system", "content": injection_detection_policy},
        {"role": "user", "content": user_input}
    ]
)
# 返回：{"violation": 1, "category": "Direct Override", "rationale": "..."}

完整策略、危害分类和集成模式请参阅 references/moderation.md。

错误处理

from groq import Groq, RateLimitError, APIConnectionError, APIStatusError

client = Groq()

try:
    response = client.chat.completions.create(
        model="llama-3.3-70b-versatile",
        messages=[{"role": "user", "content": "Hello"}]
    )
except RateLimitError:
    # 等待并使用指数退避重试
    pass
except APIConnectionError:
    # 网络问题
    pass
except APIStatusError as e:
    # API 错误（检查 e.status_code）
    pass

完整的音频 API 参考，包括文件处理、元数据字段和提示指南，请参阅 references/audio.md。

资源

• 模型与定价 : references/models.md
• 工具使用指南 : references/tool-use.md
• 视觉指南 : references/vision.md
• 音频指南 : references/audio.md
• 推理指南 : references/reasoning.md
• 结构化输出 : references/structured-outputs.md
• 提示缓存 : references/prompt-caching.md
• 审核指南 : references/moderation.md
• SDK 参考 : references/sdk.md
• 完整 API 参考 : references/api-reference.md
• 官方文档 : https://console.groq.com/docs
• Python SDK : https://github.com/groq/groq-python
• TypeScript SDK : https://github.com/groq/groq-typescript

每周安装量

61

仓库

diskd-ai/groq-api

首次出现

2026 年 1 月 23 日

安全审计

Gen Agent Trust HubPassSocketPassSnykWarn

安装于

opencode53

gemini-cli52

codex52

github-copilot47

amp46

kimi-cli45

Model Selection

See references/models.md for full model list and pricing.

Common Patterns

Streaming Responses

stream = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

System Messages

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello"}
    ]
)

Async Client (Python)

import asyncio
from groq import AsyncGroq

async def main():
    client = AsyncGroq()
    response = await client.chat.completions.create(
        model="llama-3.3-70b-versatile",
        messages=[{"role": "user", "content": "Hello"}]
    )
    return response.choices[0].message.content

print(asyncio.run(main()))

JSON Mode

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "List 3 colors as JSON array"}],
    response_format={"type": "json_object"}
)

Structured Outputs (JSON Schema)

Force output to match a schema. Two modes available:

Strict Mode (guaranteed compliance):

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    }
)

With Pydantic:

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Extract: John is 30"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": Person.model_json_schema()
        }
    }
)
person = Person.model_validate(json.loads(response.choices[0].message.content))

See references/structured-outputs.md for schema requirements, validation libraries, and examples.

Audio

Transcription (Speech-to-Text)

with open("audio.mp3", "rb") as f:
    transcription = client.audio.transcriptions.create(
        model="whisper-large-v3-turbo",
        file=f,
        language="en",  # Optional: ISO-639-1 code
        response_format="verbose_json",  # json, text, verbose_json
        timestamp_granularities=["word", "segment"]
    )
print(transcription.text)

Translation (to English)

with open("french_audio.mp3", "rb") as f:
    translation = client.audio.translations.create(
        model="whisper-large-v3",
        file=f
    )
print(translation.text)  # English text

Text-to-Speech

response = client.audio.speech.create(
    model="playai-tts",
    input="Hello, world!",
    voice="Fritz-PlayAI",
    response_format="wav",  # flac, mp3, mulaw, ogg, wav
    speed=1.0  # 0.5 to 5
)
response.write_to_file("output.wav")

Vision

Process images with Llama 4 multimodal models. Supports up to 5 images per request.

Models: meta-llama/llama-4-scout-17b-16e-instruct (faster), meta-llama/llama-4-maverick-17b-128e-instruct (higher quality)

Image from URL

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
        ]
    }]
)

Local Image (Base64)

import base64

def encode_image(path: str) -> str:
    with open(path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encode_image('photo.jpg')}"}}
        ]
    }]
)

OCR / Extract Data as JSON

response = client.chat.completions.create(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Extract all text and data as JSON"},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{base64_image}"}}
        ]
    }],
    response_format={"type": "json_object"}
)

See references/vision.md for multi-image, tool use with images, and multi-turn conversations.

Tool Use

For tool calling patterns and examples, see references/tool-use.md.

Quick example:

import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Weather in Paris?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    for tc in response.choices[0].message.tool_calls:
        args = json.loads(tc.function.arguments)
        # Execute function and continue conversation

Built-In Tools (Agentic)

Use groq/compound or openai/gpt-oss-120b for built-in web search and code execution:

response = client.chat.completions.create(
    model="groq/compound",
    messages=[{"role": "user", "content": "Search for latest Python news"}]
)
# Model automatically uses web search

MCP (Remote Tools)

Connect to third-party MCP servers for tools like Stripe, GitHub, web scraping. Use the Responses API:

import openai

client = openai.OpenAI(
    api_key=os.environ.get("GROQ_API_KEY"),
    base_url="https://api.groq.com/openai/v1"
)

response = client.responses.create(
    model="openai/gpt-oss-120b",
    input="What models are trending on Huggingface?",
    tools=[{
        "type": "mcp",
        "server_label": "Huggingface",
        "server_url": "https://huggingface.co/mcp"
    }]
)

See references/tool-use.md for MCP configuration and popular servers.

Reasoning Models

Control how models think through complex problems.

Models: openai/gpt-oss-20b, openai/gpt-oss-120b (low/medium/high), qwen/qwen3-32b (none/default)

GPT-OSS with Reasoning Effort

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "How many r's in strawberry?"}],
    reasoning_effort="high",  # low, medium, high
    temperature=0.6,
    max_completion_tokens=1024
)

print(response.choices[0].message.content)
print("Reasoning:", response.choices[0].message.reasoning)

Qwen3 with Parsed Reasoning

response = client.chat.completions.create(
    model="qwen/qwen3-32b",
    messages=[{"role": "user", "content": "Solve: x + 5 = 12"}],
    reasoning_format="parsed"  # raw, parsed, hidden
)

print("Answer:", response.choices[0].message.content)
print("Reasoning:", response.choices[0].message.reasoning)

Hide Reasoning (GPT-OSS)

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "What is 15% of 80?"}],
    include_reasoning=False  # Hide reasoning in response
)

See references/reasoning.md for streaming, tool use with reasoning, and best practices.

Batch Processing

For high-volume async processing (24h-7d completion window):

# 1. Create JSONL file with requests
# 2. Upload file
# 3. Create batch
batch = client.batches.create(
    input_file_id=file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

# 4. Check status
batch = client.batches.retrieve(batch.id)
if batch.status == "completed":
    results = client.files.content(batch.output_file_id)

See references/api-reference.md for full batch API details.

Prompt Caching

Automatically reduce latency and costs by 50% for repeated prompt prefixes. No code changes required.

Supported models: moonshotai/kimi-k2-instruct-0905, openai/gpt-oss-20b, openai/gpt-oss-120b, openai/gpt-oss-safeguard-20b

How it works:

• Place static content (system prompts, tools, examples) at the beginning
• Place dynamic content (user queries) at the end
• Cache automatically matches prefixes and applies 50% discount
• Cache expires after 2 hours without use

Track cache usage:

response = client.chat.completions.create(
    model="moonshotai/kimi-k2-instruct-0905",
    messages=[{"role": "system", "content": large_system_prompt}, ...]
)

cached = response.usage.prompt_tokens_details.cached_tokens
print(f"Cached tokens: {cached}")  # 50% discount applied to these

See references/prompt-caching.md for optimization strategies and examples.

Content Moderation

Detect and filter harmful content using safeguard models.

Llama Guard 4

General content safety classification. Returns safe or unsafe\nSX (category code).

response = client.chat.completions.create(
    model="meta-llama/Llama-Guard-4-12B",
    messages=[{"role": "user", "content": user_input}]
)

if response.choices[0].message.content.startswith("unsafe"):
    # Block or handle unsafe content
    pass

GPT-OSS Safeguard 20B

Prompt injection detection with custom policies. Returns structured JSON.

response = client.chat.completions.create(
    model="openai/gpt-oss-safeguard-20b",
    messages=[
        {"role": "system", "content": injection_detection_policy},
        {"role": "user", "content": user_input}
    ]
)
# Returns: {"violation": 1, "category": "Direct Override", "rationale": "..."}

See references/moderation.md for complete policies, harm taxonomy, and integration patterns.

Error Handling

from groq import Groq, RateLimitError, APIConnectionError, APIStatusError

client = Groq()

try:
    response = client.chat.completions.create(
        model="llama-3.3-70b-versatile",
        messages=[{"role": "user", "content": "Hello"}]
    )
except RateLimitError:
    # Wait and retry with exponential backoff
    pass
except APIConnectionError:
    # Network issue
    pass
except APIStatusError as e:
    # API error (check e.status_code)
    pass

See references/audio.md for complete audio API reference including file handling, metadata fields, and prompting guidelines.

Resources

• Models & pricing: references/models.md
• Tool use guide : references/tool-use.md
• Vision guide : references/vision.md
• Audio guide : references/audio.md
• Reasoning guide : references/reasoning.md
• Structured outputs : references/structured-outputs.md
• Prompt caching : references/prompt-caching.md
• Moderation guide : references/moderation.md
• SDK reference : references/sdk.md
• Full API reference : references/api-reference.md
• Official docs : https://console.groq.com/docs
• Python SDK : https://github.com/groq/groq-python
• TypeScript SDK : https://github.com/groq/groq-typescript

Weekly Installs

61

Repository

diskd-ai/groq-api

First Seen

Jan 23, 2026

Security Audits

Gen Agent Trust HubPassSocketPassSnykWarn

Installed on

opencode53

gemini-cli52

codex52

github-copilot47

amp46

kimi-cli45