Providers | Min (tokens/s) | Max (tokens/s) | Avg (tokens/s) |
---|---|---|---|
Anthropic | 4.51 | 44 | 12.44 |
Vertex AI | 4.88 | 56.28 | 22.66 |
Amazon Bedrock | 36.25 | 48.06 | 44.80 |
Providers | Min (ms) | Max (ms) | Avg (ms) |
---|---|---|---|
Anthropic | 705 | 3593 | 1960.97 |
Vertex AI | 765 | 39980 | 9927.54 |
Amazon Bedrock | 1524 | 3501 | 2152.00 |
pythonfrom openai import OpenAI client = OpenAI( base_url="https://zenmux.ai/api/v1", api_key="<ZenMux_API_KEY>", ) completion = client.chat.completions.create( model="anthropic/claude-3.7-sonnet", messages=[ { "role": "user", "content": [ { "type": "text", "text": "What is in this image?" } ] } ] ) print(completion.choices[0].message.content)