efficient-transformers

Getting Started

  • Introduction Qualcomm efficient-transformers library
  • Validated Models
  • Models Coming Soon

Installation

  • Pre-requisites
  • Installation
  • Sanity Check

Upgrade Efficient-Transformers

  • Using GitHub Repository

Quick start

  • Transformed models and QPC storage
  • Command Line Interface
  • Python API

Command Line Interface Use (CLI)

  • QEfficient.cloud.infer
  • QEfficient.cloud.execute
  • QEfficient.cloud.compile
  • QEfficient.cloud.export
  • QEfficient.cloud.finetune

Python API

  • High Level API
  • Low Level API

QAIC Finetune

  • Finetune Infra

Blogs

  • Train anywhere, Infer on Qualcomm Cloud AI 100
  • How to Quadruple LLM Decoding Performance with Speculative Decoding (SpD) and Microscaling (MX) Formats on Qualcomm® Cloud AI 100
  • Power-efficient acceleration for large language models – Qualcomm Cloud AI SDK
  • Qualcomm Cloud AI 100 Accelerates Large Language Model Inference by ~2x Using Microscaling (Mx) Formats
  • Qualcomm Cloud AI Introduces Efficient Transformers: One API, Infinite Possibilities

Reference

  • Qualcomm Cloud AI home
  • Qualcomm Cloud AI SDK download
  • Qualcomm Cloud AI API reference
  • User Guide
  • OCP Microscaling Formats (MX) Specification
efficient-transformers
  • Overview: module code

All modules for which code is available

  • QEfficient.compile.compile_helper
  • QEfficient.exporter.export_hf_to_cloud_ai_100
  • QEfficient.generation.text_generation_inference
  • QEfficient.peft.auto
  • QEfficient.peft.lora.auto
  • QEfficient.transformers.models.modeling_auto
  • QEfficient.utils.device_utils
  • QEfficient.utils.generate_inputs
  • QEfficient.utils.run_utils

© Copyright 2024, Qualcomm.

Built with Sphinx using a theme provided by Read the Docs.
Version: Main
Versions
main
release/v1.18