alt text

Introduction Qualcomm efficient-transformers library

Train anywhere, Infer on Qualcomm Cloud AI with a Developer-centric Toolchain

This library provides reimplemented blocks of LLMs which are used to make the models functional and highly performant on Qualcomm Cloud AIxxx (AI100, AI200 and so on). We support wide range of models architectures, for easy efficient deployment on Cloud AIxxx (AI100, AI200 and so on) cards. Users only need to provide model card from HuggingFace or Path to the local model and the library will take care of transforming model to it’s efficient implementation for Cloud AIxxx (AI100, AI200 and so on).

For other models, there is comprehensive documentation to inspire upon the changes needed and How-To(s).

Typically for LLMs, the library provides:

  1. Reimplemented blocks from Transformers which enable efficient on-device retention of intermediate states.

  2. Graph transformations to enable execution of key operations in lower precision

  3. Graph transformations to replace some operations to other mathematically equivalent operations that are efficient/supported on HW backend

  4. Handling for underflow and overflows in lower precision

  5. Patcher modules to map weights of original model’s operations to updated model’s operations

  6. Exporter module to export the model source into a ONNX Graph.

  7. Sample example applications and demo notebooks

  8. Unit test templates.

Latest news :

  • [07/2026] Added dynamo flag to QEFFAutoModelForCausalLM.export() to support torch.onnx.export dynamo-based ONNX export for CausalLM models

  • [06/2026] Added support for Gemma4 models, google/gemma-4-E2B-it, google/gemma-4-26B-A4B-it

  • [06/2026] Added support for Qwen3.6 model Qwen/Qwen3.6-35B-A3B

  • [06/2026] Added support for Qwen3.5 modelQwen/Qwen3.5-0.8B

  • [06/2026] Added support for Qwen3 VL and Qwen3 MoE VL models, Qwen/Qwen3-VL-30B-A3B-Instruct, Qwen/Qwen3-VL-32B-Instruct

  • [06/2026] Added support for GLM-4.5 MoE model and enabled its disaggregated mode zai-org/GLM-4.5

  • [06/2026] Enabled disaggregated support for Qwen3 MoE model

  • [04/2026] Added WAN non-unified execution support in QEffWanPipeline with separate transformer_high and transformer_low modules

  • [04/2026] Added first-block-cache support for WAN non-unified mode and FLUX (QEffWanPipeline, QEffFluxPipeline)

More