# TPU

## Supported Models

### Text-only Language Models

| Model                                               | Architecture                   | Supported |
|-----------------------------------------------------|--------------------------------|-----------|
| mistralai/Mixtral-8x7B-Instruct-v0.1                | MixtralForCausalLM             | 🟨 |
| mistralai/Mistral-Small-24B-Instruct-2501           | MistralForCausalLM             | ✅ |
| mistralai/Codestral-22B-v0.1                        | MistralForCausalLM             | ✅ |
| mistralai/Mixtral-8x22B-Instruct-v0.1               | MixtralForCausalLM             | ❌ |
| meta-llama/Llama-3.3-70B-Instruct                   | LlamaForCausalLM               | ✅ |
| meta-llama/Llama-3.1-8B-Instruct                    | LlamaForCausalLM               | ✅ |
| meta-llama/Llama-3.1-70B-Instruct                   | LlamaForCausalLM               | ✅ |
| meta-llama/Llama-4-*                                | Llama4ForConditionalGeneration | ❌ |
| microsoft/Phi-3-mini-128k-instruct                  | Phi3ForCausalLM                | 🟨 |
| microsoft/phi-4                                     | Phi3ForCausalLM                | ❌ |
| google/gemma-3-27b-it                               | Gemma3ForConditionalGeneration | 🟨 |
| google/gemma-3-4b-it                                | Gemma3ForConditionalGeneration | ❌ |
| deepseek-ai/DeepSeek-R1                             | DeepseekV3ForCausalLM          | ❌ |
| deepseek-ai/DeepSeek-V3                             | DeepseekV3ForCausalLM          | ❌ |
| RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8  | LlamaForCausalLM               | ✅ |
| RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 | LlamaForCausalLM               | ✅ |
| Qwen/Qwen3-8B                                       | Qwen3ForCausalLM               | ✅ |
| Qwen/Qwen3-32B                                      | Qwen3ForCausalLM               | ✅ |
| Qwen/Qwen2.5-7B-Instruct                            | Qwen2ForCausalLM               | ✅ |
| Qwen/Qwen2.5-32B                                    | Qwen2ForCausalLM               | ✅ |
| Qwen/Qwen2.5-14B-Instruct                           | Qwen2ForCausalLM               | ✅ |
| Qwen/Qwen2.5-1.5B-Instruct                          | Qwen2ForCausalLM               | 🟨 |

✅ Runs and optimized.  
🟨 Runs and correct but not optimized to green yet.  
❌ Does not pass accuracy test or does not run.