Validated Models

Note- All validated models support Continuous Batching functionality.

Model Name	Model Support
CodeGemma-2b	✔️
CodeGemma-7b	✔️
CodeLlama-7b-hf	✔️
CodeLlama-13b-hf	✔️
CodeLlama-34b-hf	✔️
Codestral-22B-v0.1	✔️
Falcon-40b	✔️
GPT-J-6B	✔️
GPT2	✔️
Gemma-2b	✔️
Gemma-7b	✔️
Gemma-2-2b	✔️
Gemma-2-9b	✔️
Gemma-2-27b	✔️
Jais-adapted-7b	✔️
Jais-adapted-13b-chat	✔️
Jais-adapted-70b	✔️
Llama-2-7b-chat-hf	✔️
Llama-2-13b-chat-hf	✔️
Llama-2-70b	✔️
Llama-3-8b	✔️
Llama-3-70b	✔️
Llama-3.1-8B	✔️
Llama-3.1-70B	✔️
Llama-3.2-1B	✔️
Llama-3.2-3B	✔️
MPT-7b	✔️
Mistral-7B-Instruct-v0.1	✔️
Mixtral-8x7B	✔️
Phi3-Mini-4K-Instruct	✔️
Qwen2-1.5B-Instruct	✔️
Starcoder1-15B	✔️
Starcoder2-15B	✔️
Vicuna-v0	✔️
Vicuna-v1.3	✔️
Vicuna-v1.5	✔️

Models Coming Soon

• Baichuan2-7B-Base

• CohereForAI/c4ai-command-r-v01

• Chatglm2-6b

• databricks/dbrx-base

• Llama-3.1-405B

• Llama-3.2-11B-Vision

• Llama-3.2-90B-Vision