AI Model Efficiency Toolkit Logo
1.32.0
AIMET Variant: Universal
Documentation Versions
Universal
PyTorch
  • Quantization User Guide
    • Use Cases
    • AIMET Quantization Features
      • Quantization Simulation
        • Overview
        • QuantSim Workflow
        • Simulating Quantization Noise
        • Determining Quantization Parameters (Encodings)
        • Quantization Schemes
        • Configuring Quantization Simulation Ops
        • Frequently Asked Questions
      • Quantization-Aware Training (QAT)
        • Overview
        • QAT workflow
        • QAT modes
        • Recommendations for Quantization-Aware Training
      • Post-Training Quantization
        • AutoQuant
          • Overview
          • Workflow
        • Adaptive Rounding (AdaRound)
          • AdaRound Use Cases
          • Common terminology
          • Use Cases
        • Cross-Layer Equalization
          • Overview
          • User Flow
          • FAQs
          • References
        • BN Re-estimation
          • Overview
          • Workflow
        • Bias Correction [Depricated]
          • Overview
          • User Flow
          • FAQs
          • References
      • Debugging/Analysis Tools
        • QuantAnalyzer
          • Overview
          • Requirements
          • Detailed Analysis Descriptions
        • Visualizations
          • Overview
          • Quantization
            • PyTorch
            • TensorFlow
    • AIMET Quantization Workflow
      • PyTorch
        • PyTorch Model Guidelines
        • AIMET PyTorch Quantization APIs
          • Model Guidelines
          • Architecture Checker API
            • check_model_arch()
          • Model Preparer API
            • Top-level API
              • prepare_model()
            • Code Examples
            • Limitations of torch.fx symbolic trace API
          • Model Validator API
          • Quant Analyzer API
            • Top-level API
              • QuantAnalyzer
                • QuantAnalyzer.analyze()
                • QuantAnalyzer.enable_per_layer_mse_loss()
            • Code Examples
          • Quantization Simulation API
            • User Guide Link
            • Examples Notebook Link
            • Guidelines
            • Top-level API
              • QuantizationSimModel
                • QuantizationSimModel.compute_encodings()
                • QuantizationSimModel.export()
              • quantsim.save_checkpoint()
              • quantsim.load_checkpoint()
            • Enum Definition
              • QuantScheme
                • QuantScheme.post_training_percentile
                • QuantScheme.post_training_tf
                • QuantScheme.post_training_tf_enhanced
                • QuantScheme.training_range_learning_with_tf_enhanced_init
                • QuantScheme.training_range_learning_with_tf_init
            • Code Example - Quantization Aware Training (QAT)
          • Adaptive Rounding API
            • User Guide Link
            • Examples Notebook Link
            • Top-level API
              • apply_adaround()
            • Adaround Parameters
              • AdaroundParameters
            • Enum Definition
              • QuantScheme
                • QuantScheme.post_training_percentile
                • QuantScheme.post_training_tf
                • QuantScheme.post_training_tf_enhanced
                • QuantScheme.training_range_learning_with_tf_enhanced_init
                • QuantScheme.training_range_learning_with_tf_init
            • Code Example - Adaptive Rounding (AdaRound)
          • Cross-Layer Equalization API
            • User Guide Link
            • Examples Notebook Link
            • Introduction
            • Cross Layer Equalization API
              • equalize_model()
            • Code Example
            • Primitive APIs
              • Primitive APIs for Cross Layer Equalization
                • Introduction
                • ClsSetInfo Definition
                  • ClsSetInfo
                    • ClsSetInfo.ClsSetLayerPairInfo
                • Higher Level APIs for Cross Layer Equalization
                  • fold_all_batch_norms()
                  • scale_model()
                  • bias_fold()
                • Code Examples for Higher Level APIs
                • Lower Level APIs for Cross Layer Equalization
                  • fold_given_batch_norms()
                  • scale_cls_sets()
                  • bias_fold()
                • Code Examples for Lower Level APIs
          • Bias Correction API
            • User Guide Link
            • Bias Correction API
              • correct_bias()
            • ConvBnInfoType
              • ConvBnInfoType
            • ActivationType
              • ActivationType
                • ActivationType.no_activation
                • ActivationType.relu
                • ActivationType.relu6
            • Quantization Params
              • QuantParams
            • Code Example #1 Empirical Bias Correction
            • Code Example #2 Analytical + Empirical Bias correction
          • AutoQuant API
            • User Guide Link
            • Examples Notebook Link
            • Top-level API
              • AutoQuant
                • AutoQuant.run_inference()
                • AutoQuant.optimize()
                • AutoQuant.set_adaround_params()
                • AutoQuant.set_export_params()
                • AutoQuant.set_model_preparer_params()
                • AutoQuant.get_quant_scheme_candidates()
                • AutoQuant.set_quant_scheme_candidates()
              • AutoQuant
                • AutoQuant.apply()
                • AutoQuant.set_adaround_params()
                • AutoQuant.set_export_params()
            • Code Examples
          • BN Re-estimation APIs
            • Examples Notebook Link
            • Introduction
            • Top-level APIs
              • reestimate_bn_stats()
              • fold_all_batch_norms_to_scale()
            • Code Example - BN-Reestimation
          • Multi-GPU guidelines
      • Tensorflow
        • TensorFlow Model Guidelines
          • update_keras_bn_ops_trainable_flag()
    • Debugging Guidelines
      • Quantization Guidebook
  • Compression User Guide
    • Overview
      • Compression Guidebook
    • Use Case
    • Compression ratio selection
      • Greedy Compression Ratio Selection
        • Overview
        • How it works
        • Per-layer Exploration
        • Compression Ratio Selection
      • Visualization
        • Overview
        • Design
        • Compression
        • Starting a Bokeh Server Session:
        • How to use the tool
    • Model Compression
      • Weight SVD
      • Spatial SVD
      • Channel Pruning
        • Overall Procedure
        • Channel Selection
        • Winnowing
          • Winnowing
            • Overview
            • Winnowing Overview
            • How Winnowing Works
        • Weight Reconstruction
    • Optional techniques to get better compression results
      • Rank Rounding
      • Per-layer Fine-tuning
    • FAQs
    • References
  • API Documentation
    • AIMET APIs for PyTorch
      • PyTorch Model Quantization API
        • Model Guidelines
        • Architecture Checker API
          • check_model_arch()
        • Model Preparer API
          • Top-level API
            • prepare_model()
          • Code Examples
          • Limitations of torch.fx symbolic trace API
        • Model Validator API
        • Quant Analyzer API
          • Top-level API
            • QuantAnalyzer
              • QuantAnalyzer.analyze()
              • QuantAnalyzer.enable_per_layer_mse_loss()
          • Code Examples
        • Quantization Simulation API
          • User Guide Link
          • Examples Notebook Link
          • Guidelines
          • Top-level API
            • QuantizationSimModel
              • QuantizationSimModel.compute_encodings()
              • QuantizationSimModel.export()
            • quantsim.save_checkpoint()
            • quantsim.load_checkpoint()
          • Enum Definition
            • QuantScheme
              • QuantScheme.post_training_percentile
              • QuantScheme.post_training_tf
              • QuantScheme.post_training_tf_enhanced
              • QuantScheme.training_range_learning_with_tf_enhanced_init
              • QuantScheme.training_range_learning_with_tf_init
          • Code Example - Quantization Aware Training (QAT)
        • Adaptive Rounding API
          • User Guide Link
          • Examples Notebook Link
          • Top-level API
            • apply_adaround()
          • Adaround Parameters
            • AdaroundParameters
          • Enum Definition
            • QuantScheme
              • QuantScheme.post_training_percentile
              • QuantScheme.post_training_tf
              • QuantScheme.post_training_tf_enhanced
              • QuantScheme.training_range_learning_with_tf_enhanced_init
              • QuantScheme.training_range_learning_with_tf_init
          • Code Example - Adaptive Rounding (AdaRound)
        • Cross-Layer Equalization API
          • User Guide Link
          • Examples Notebook Link
          • Introduction
          • Cross Layer Equalization API
            • equalize_model()
          • Code Example
          • Primitive APIs
            • Primitive APIs for Cross Layer Equalization
              • Introduction
              • ClsSetInfo Definition
                • ClsSetInfo
                  • ClsSetInfo.ClsSetLayerPairInfo
              • Higher Level APIs for Cross Layer Equalization
                • fold_all_batch_norms()
                • scale_model()
                • bias_fold()
              • Code Examples for Higher Level APIs
              • Lower Level APIs for Cross Layer Equalization
                • fold_given_batch_norms()
                • scale_cls_sets()
                • bias_fold()
              • Code Examples for Lower Level APIs
        • Bias Correction API
          • User Guide Link
          • Bias Correction API
            • correct_bias()
          • ConvBnInfoType
            • ConvBnInfoType
          • ActivationType
            • ActivationType
              • ActivationType.no_activation
              • ActivationType.relu
              • ActivationType.relu6
          • Quantization Params
            • QuantParams
          • Code Example #1 Empirical Bias Correction
          • Code Example #2 Analytical + Empirical Bias correction
        • AutoQuant API
          • User Guide Link
          • Examples Notebook Link
          • Top-level API
            • AutoQuant
              • AutoQuant.run_inference()
              • AutoQuant.optimize()
              • AutoQuant.set_adaround_params()
              • AutoQuant.set_export_params()
              • AutoQuant.set_model_preparer_params()
              • AutoQuant.get_quant_scheme_candidates()
              • AutoQuant.set_quant_scheme_candidates()
            • AutoQuant
              • AutoQuant.apply()
              • AutoQuant.set_adaround_params()
              • AutoQuant.set_export_params()
          • Code Examples
        • BN Re-estimation APIs
          • Examples Notebook Link
          • Introduction
          • Top-level APIs
            • reestimate_bn_stats()
            • fold_all_batch_norms_to_scale()
          • Code Example - BN-Reestimation
        • Multi-GPU guidelines
      • PyTorch Model Compression API
        • Introduction
        • Top-level API for Compression
          • ModelCompressor
            • ModelCompressor.compress_model()
        • Greedy Selection Parameters
          • GreedySelectionParameters
        • TAR Selection Parameters
          • TarRankSelectionParameters
        • Spatial SVD Configuration
          • SpatialSvdParameters
            • SpatialSvdParameters.AutoModeParams
            • SpatialSvdParameters.ManualModeParams
            • SpatialSvdParameters.Mode
              • SpatialSvdParameters.Mode.auto
              • SpatialSvdParameters.Mode.manual
        • Weight SVD Configuration
          • WeightSvdParameters
            • WeightSvdParameters.AutoModeParams
            • WeightSvdParameters.ManualModeParams
            • WeightSvdParameters.Mode
              • WeightSvdParameters.Mode.auto
              • WeightSvdParameters.Mode.manual
        • Channel Pruning Configuration
          • ChannelPruningParameters
            • ChannelPruningParameters.AutoModeParams
            • ChannelPruningParameters.ManualModeParams
            • ChannelPruningParameters.Mode
              • ChannelPruningParameters.Mode.auto
              • ChannelPruningParameters.Mode.manual
        • Configuration Definitions
          • ModuleCompRatioPair
        • Code Examples
      • PyTorch Model Visualization API for Compression
        • Top-level API Compression
          • VisualizeCompression
            • VisualizeCompression.display_eval_scores()
            • VisualizeCompression.display_comp_ratio_plot()
        • Code Examples
      • PyTorch Model Visualization API for Quantization
        • Top-level API Quantization
          • visualize_relative_weight_ranges_to_identify_problematic_layers()
          • visualize_weight_ranges()
          • visualize_changes_after_optimization()
        • Code Examples
      • PyTorch Debug API
        • Top-level API
          • LayerOutputUtil
            • LayerOutputUtil.generate_layer_outputs()
        • Enum Definition
          • NamingScheme
            • NamingScheme.ONNX
            • NamingScheme.PYTORCH
            • NamingScheme.TORCHSCRIPT
        • Code Example
    • AIMET APIs for TensorFlow
      • TensorFlow Model Guidelines
        • update_keras_bn_ops_trainable_flag()
      • TensorFlow Model Quantization API
      • TensorFlow Model Compression API
        • Introduction
        • Top-level API for Compression
          • ModelCompressor
            • ModelCompressor.compress_model()
        • Greedy Selection Parameters
        • Spatial SVD Configuration
          • SpatialSvdParameters
            • SpatialSvdParameters.AutoModeParams
            • SpatialSvdParameters.ManualModeParams
            • SpatialSvdParameters.Mode
              • SpatialSvdParameters.Mode.auto
              • SpatialSvdParameters.Mode.manual
        • Channel Pruning Configuration
          • ChannelPruningParameters
            • ChannelPruningParameters.AutoModeParams
            • ChannelPruningParameters.ManualModeParams
            • ChannelPruningParameters.Mode
              • ChannelPruningParameters.Mode.auto
              • ChannelPruningParameters.Mode.manual
        • Configuration Definitions
          • CostMetric
            • CostMetric.mac
            • CostMetric.memory
          • CompressionScheme
            • CompressionScheme.channel_pruning
            • CompressionScheme.spatial_svd
            • CompressionScheme.weight_svd
          • ModuleCompRatioPair
        • Code Examples
        • Weight SVD Top-level API
          • Svd
            • Svd.compress_net()
        • Code Examples for Weight SVD
      • TensorFlow Model Visualization API for Quantization
        • Top-level API for Visualization of Weight tensors
          • visualize_weight_ranges_single_layer()
          • visualize_relative_weight_ranges_single_layer()
        • Code Examples for Visualization of Weight tensors
      • Using AIMET Tensorflow APIs with Keras Models
        • Introduction
        • APIs
          • save_tf_session_single_gpu()
          • load_tf_sess_variables_to_keras_single_gpu()
          • save_as_tf_module_multi_gpu()
          • load_keras_model_multi_gpu()
        • Code Example
        • Utility Functions
      • Tensorflow Debug API
        • Top-level API
          • LayerOutputUtil
            • LayerOutputUtil.generate_layer_outputs()
        • Code Example
    • AIMET APIs for Keras
      • Keras Model Quantization API
        • Model Guidelines
        • Model Preparer API
          • Top-level API
            • prepare_model()
          • Code Examples
          • Limitations
        • Quant Analyzer API
          • Top-level API
            • QuantAnalyzer
              • QuantAnalyzer.analyze()
          • Code Examples
        • Quantization Simulation API
          • User Guide Link
          • Top-level API
            • QuantizationSimModel
              • QuantizationSimModel.compute_encodings()
              • QuantizationSimModel.export()
          • Code Examples
        • Adaptive Rounding API
          • User Guide Link
          • Examples Notebook Link
          • Top-level API
          • Adaround Parameters
            • AdaroundParameters
          • Enum Definition
            • QuantScheme
              • QuantScheme.post_training_percentile
              • QuantScheme.post_training_tf
              • QuantScheme.post_training_tf_enhanced
              • QuantScheme.training_range_learning_with_tf_enhanced_init
              • QuantScheme.training_range_learning_with_tf_init
          • Code Examples
        • Cross-Layer Equalization API
          • User Guide Link
          • Examples Notebook Link
          • Introduction
          • Cross Layer Equalization API
            • equalize_model()
          • Code Example
          • Primitive APIs
            • Primitive APIs for Cross Layer Equalization
              • Introduction
              • Higher Level APIs for Cross Layer Equalization
                • fold_all_batch_norms()
                • scale_model()
                • bias_fold()
              • Code Examples for Higher Level APIs
              • Lower Level APIs for Cross Layer Equalization
                • fold_given_batch_norms()
                • scale_cls_sets()
                • bias_fold()
              • Custom Datatype used
                • ClsSetInfo
                  • ClsSetInfo.ClsSetLayerPairInfo
              • Code Example for Lower level APIs
              • Example helper methods to perform CLE in manual mode
        • BN Re-estimation APIs
          • Examples Notebook Link
          • Introduction
          • Top-level APIs
            • reestimate_bn_stats()
            • fold_all_batch_norms_to_scale()
          • Code Example
          • Limitations
      • Keras Debug API
        • Top-level API
          • LayerOutputUtil
            • LayerOutputUtil.generate_layer_outputs()
        • Code Example
      • Keras Model Compression API
        • Introduction
        • Top-level API for Compression
          • ModelCompressor
            • ModelCompressor.compress_model()
        • Greedy Selection Parameters
        • Spatial SVD Configuration
          • SpatialSvdParameters
            • SpatialSvdParameters.AutoModeParams
            • SpatialSvdParameters.ManualModeParams
            • SpatialSvdParameters.Mode
              • SpatialSvdParameters.Mode.auto
              • SpatialSvdParameters.Mode.manual
        • Configuration Definitions
          • CostMetric
            • CostMetric.mac
            • CostMetric.memory
          • CompressionScheme
            • CompressionScheme.channel_pruning
            • CompressionScheme.spatial_svd
            • CompressionScheme.weight_svd
          • ModuleCompRatioPair
        • Code Examples
    • AIMET APIs for ONNX
      • ONNX Model Quantization API
        • Quantization Simulation API
          • Top-level API
          • Code Examples
        • Cross-Layer Equalization API
          • User Guide Link
          • Introduction
          • Cross Layer Equalization API
          • Code Example
        • Adaptive Rounding API
          • User Guide Link
          • Top-level API
          • Adaround Parameters
          • Code Example - Adaptive Rounding (AdaRound)
        • AutoQuant API
          • User Guide Link
          • Top-level API
          • Code Examples
        • QuantAnalyzer API
          • Top-level API
          • Run specific utility
          • Code Examples
      • ONNX Debug API
        • Top-level API
        • Code Example
    • Indices and tables
  • Examples Documentation
    • Browse the notebooks
    • Running the notebooks
      • Install Jupyter
      • Download the Example notebooks and related code
      • Run the notebooks
  • Installation
    • Quick Install
    • Release Packages
    • System Requirements
    • Advanced Installation Instructions
      • Install in Host Machine
        • Install prerequisite packages
        • Install GPU packages
          • Install GPU packages for PyTorch 1.13 or ONNX
          • Install GPU packages for PyTorch 2.1 or TensorFlow
        • Install AIMET packages
          • From PyPI
          • From Release Package
        • Install common debian packages
        • Install tensorflow GPU debian packages
        • Install torch GPU debian packages
        • Install ONNX GPU debian packages
        • Replace Pillow with Pillow-SIMD
        • Replace onnxruntime with onnxruntime-gpu
        • Post installation steps
        • Environment setup
      • Install in Docker Container
        • Set variant
        • Use prebuilt docker image
        • Build docker image locally
        • Start docker container
        • Install AIMET packages
          • From PyPI
          • From Release Package
        • Environment setup
AI Model Efficiency Toolkit
  • Search

© Copyright 2020, Qualcomm Innovation Center, Inc..

Built with Sphinx using a theme provided by Read the Docs.