Pre-requisites

System Requirements:

1. Supported Linux OS - Ubuntu, RHEL and AWS Linux

2. Cloud AI 100 Platform and Apps SDK installed

3. SDK Pre-requisites

4. Multi-device support enabled for model sharding

Linux Installation

There are two different way to install efficient-transformers.

Using SDK

• Download Apps SDK: Cloud AI 100 Platform and Apps SDK install

# Install using Apps SDK

bash install.sh --enable-qeff
source  /opt/qti-aic/dev/python/qeff/bin/activate

Using GitHub Repository

# Create Python virtual env and activate it. (Required Python 3.8)

python3.8 -m venv qeff_env
source qeff_env/bin/activate
pip install -U pip

# Clone and Install the QEfficient Repo.
pip install git+https://github.com/quic/efficient-transformers

Sanity Check

After any of the above installation methods, you can check if QEfficient is installed correctly by using

python -c "import QEfficient; print(QEfficient.__version__)"

If the above line executes successfully, you are good to go ahead and start deploying models on Cloud AI 100 cards using QEfficient library.