AIMET Tensorflow Layer Output Generation API
This API captures and saves intermediate layer-outputs of a model. The model can be original (FP32) or quantsim. The layer-outputs are named according to the exported Tensorflow model by the quantsim export API. This allows layer-output comparison amongst FP32 model, quantization simulated model and actually quantized model on target-device to debug accuracy miss-match issues.
Top-level API
- class aimet_tensorflow.layer_output_utils.LayerOutputUtil(session, starting_op_names, output_op_names, dir_path)[source]
Implementation to capture and save outputs of intermediate layers of a model (fp32/quantsim)
Constructor for LayerOutputUtil.
- Parameters:
session (
Session
) – Session containing the model whose layer-outputs are needed.starting_op_names (
List
[str
]) – List of starting op names of the model.output_op_names (
List
[str
]) – List of output op names of the model.dir_path (
str
) – Directory wherein layer-outputs will be saved.
The following API can be used to Generate Layer Outputs
- LayerOutputUtil.generate_layer_outputs(input_batch)[source]
This method captures output of every layer of a model & saves the inputs and corresponding layer-outputs to disk.
- Parameters:
input_batch (
Union
[ndarray
,List
[ndarray
],Tuple
[ndarray
]]) – Batch of inputs for which we want to obtain layer-outputs.- Returns:
None
Code Example
Imports
import tensorflow as tf
from aimet_tensorflow.quantsim import QuantizationSimModel
from aimet_tensorflow.layer_output_utils import LayerOutputUtil
Obtain Original or QuantSim model session from AIMET Export Artifacts
# Load the model into session.
tf.compat.v1.reset_default_graph()
session = tf.compat.v1.Session()
saver = tf.compat.v1.train.import_meta_graph('path/to/aimet_export_artifacts/model.meta')
saver.restore(session, 'path/to/aimet_export_artifacts/model')
# Use same arguments as that were used for the exported QuantSim model. For sake of simplicity only mandatory arguments are passed below.
quantsim = QuantizationSimModel(session, starting_op_names, output_op_names, use_cuda=False)
# Load exported encodings into quantsim object.
quantsim.load_encodings_to_sim('path/to/aimet_export_artifacts/model.encodings')
# Check whether constructed original and quantsim model are running properly before using Layer Output Generation API.
_ = session.run(None, feed_dict)
_ = quantsim.session.run(None, feed_dict)
Obtain inputs for which we want to generate intermediate layer-outputs
# Use same input pre-processing pipeline as was used for computing the quantization encodings.
input_batches = get_pre_processed_inputs()
Generate layer-outputs
# Use original session to get fp32 layer-outputs
fp32_layer_output_util = LayerOutputUtil(session, starting_op_names, output_op_names, dir_path='./fp32_layer_outputs')
# Use quantsim session to get quantsim layer-outputs
quantsim_layer_output_util = LayerOutputUtil(quantsim.session, starting_op_names, output_op_names, dir_path='./quantsim_layer_outputs')
for input_batch in input_batches:
fp32_layer_output_util.generate_layer_outputs(input_batch)
quantsim_layer_output_util.generate_layer_outputs(input_batch)