AIMET weight SVD

Weight singular value decomposition (SVD) is a technique that decomposes one large layer (in terms of MAC or memory) into two smaller layers.

Consider a neural network layer with the kernel (π‘š,𝑛,β„Ž,𝑀) where:

  • π‘š is the input channels

  • 𝑛 the output channels

  • β„Ž is the height of the kernel

  • 𝑀 is the width of the kernel

Weight SVD decomposes the kernel into one of size (π‘š,π‘˜,1,1) and another of size (π‘˜,𝑛,h,𝑀), where π‘˜ is called the rank. The smaller the value of π‘˜, larger the degree of compression.

The following figure illustrates how weight SVD decomposes the output channel dimension. Weight SVD is currently supported for convolution (Conv) and fully connected (FC) layers in AIMET.

../_images/weight_svd.png