---
title: Together AI
url: https://arcana.otnelhq.com/docs/providers/together
---

# Together AI



Together AI provides fast inference for open-source models with competitive pricing. They host a wide range of models including Llama, Mixtral, and support fine-tuned model deployment.




## Setup



```bash
# 1. Get your API key from https://api.together.xyz/settings/api-keys
export TOGETHER_API_KEY=...

# 2. Verify detection
arcana doctor

# 3. Run
arcana run "hello"
```




## Available Models



| Model | Parameters | Best For |
| --- | --- | --- |
| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | 70B | Best quality, fast inference |
| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | 8B | Fast, general purpose |
| `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | 405B | Highest quality open model |
| `mistralai/Mixtral-8x22B-Instruct-v0.1` | 8x22B | Mixture of experts, versatile |
| `mistralai/Mistral-7B-Instruct-v0.3` | 7B | Fast, instruction following |
| `deepseek-ai/DeepSeek-R1` | 671B MoE | Reasoning, math, code |
| `Qwen/Qwen2.5-72B-Instruct-Turbo` | 72B | Multilingual, coding |



See the full list at [api.together.xyz/models](https://api.together.xyz/models).




## Config File



```json
{
  "provider": "together",
  "model": "meta-llama/Llama-3.3-70B-Instruct-Turbo"
}
```




## Environment Variables



| Variable | Description |
| --- | --- |
| `TOGETHER_API_KEY` | Your Together AI API key (required) |




## OpenAI-Compatible API



Together AI is OpenAI-compatible. You can also use it via OpenAI's client by setting the base URL:



```bash
export OPENAI_API_KEY=$TOGETHER_API_KEY
export OPENAI_BASE_URL=https://api.together.xyz/v1
export ARCANA_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
```




## Pricing Notes



Together AI charges per-token. Approximate costs per 1M tokens:



| Model | Input | Output |
| --- | --- | --- |
| Llama 3.3 70B | $0.88 | $0.88 |
| Llama 3.1 405B | $3.50 | $3.50 |
| Llama 3.1 8B | $0.10 | $0.10 |
| Mixtral 8x22B | $1.20 | $1.20 |
| DeepSeek R1 | $3.00 | $7.00 |



Free tier: $1 credit for new accounts. Check [together.ai/pricing](https://www.together.ai/pricing) for current rates.




## Fine-Tuned Models



Together AI supports deploying your own fine-tuned models. After fine-tuning, you can reference your model directly:



```bash
# Use your fine-tuned model
export ARCANA_MODEL=your-org/your-fine-tuned-model

# Or in config
# { "provider": "together", "model": "your-org/your-fine-tuned-model" }
```




## Tips



- The `Turbo` variants of models are optimized for faster inference
- Together AI is OpenAI-compatible — most OpenAI tools work out of the box
- Great choice for open-source model inference at scale
- Their pricing is very competitive, especially for smaller models
- Supports both streaming and non-streaming responses




## Limitations



- Only hosts open-source models (no GPT, Claude, or Gemini)
- Fine-tuning has additional costs beyond inference
- Some models may have longer cold-start times
- Rate limits vary by plan (free tier: 200 RPM)