8.1.1.8. blueoil.quantizations¶
This Package provides a set of quantizations.
How to use:
quantizer = binary_mean_scaling_quantizer() # initialize quantizer
weights = tf.get_variable("kernel", shape=[1, 2, 3, 4]) # prepare variable.
quantized_weights = quantizer(weights) # use quantizer to quantize variable
tf.nn.conv2d(inputs, quantized_weights)
-
blueoil.quantizations.linear_mid_tread_half_quantizer()¶
-
blueoil.quantizations.binary_mean_scaling_quantizer()¶
-
blueoil.quantizations.binary_channel_wise_mean_scaling_quantizer()¶
-
blueoil.quantizations.ttq_weight_quantizer()¶
-
blueoil.quantizations.twn_weight_quantizer()¶
8.1.1.8.1. Submodules¶
8.1.1.8.2. Package Contents¶
8.1.1.8.2.1. Functions¶
|
Binary channel wise mean scaling quantizer. |
|
Binary mean scaling quantizer. |
|
Linear mid tread half quantizer. |
|
Trained Ternary Quantization (TTQ) |
|
Ternary Weight Networks (TWN) |
-
blueoil.quantizations.binary_channel_wise_mean_scaling_quantizer(backward=None, dtype=tf.float32) Binary channel wise mean scaling quantizer.
This quantization creates a binary channel wise mean scaling quantizer. If backward is provided, this backward will be used in backpropagation.
This method is varient of XNOR-Net [1]_ weight quantization, the difference from XNOR-Net [1]_ is backward function.
op_type is
BinaryChannelWiseMeanScalingQuantizer.Forward is:
\[\begin{split}\begin{align} \bar{\mathbf{x}} & = \frac{1}{n}||\mathbf{X}||_{\ell1} & \text{$\bar{\mathbf{x}}$ is a $c$-channels vector} \\ & & \text{$n$ is number of elements in each channel of $\mathbf{X}$} \\\\ \mathbf{Y} & = \text{sign}\big(\mathbf{X}\big) \times \bar{\mathbf{x}} &\\ \end{align}\end{split}\]Default backward is:
\[\frac{\partial Loss}{\partial \mathbf{X}} = \frac{\partial Loss}{\partial \mathbf{Y}}\]- Parameters
backward (callable) – Be used in backpropagation.
dtype (tf.DType) – Define the data type of args of forward and backward.
- Returns
forward function (grad_func defined).
- Return type
callable
-
blueoil.quantizations.binary_mean_scaling_quantizer(backward=None, dtype=tf.float32) Binary mean scaling quantizer.
This quantization creates a binary mean scaling quantizer. If backward is provided, this backward will be used in backpropagation.
This method is DoReFa-Net 2 weight quantization.
op_type is
BinaryMeanScalingQuantizer.Forward is:
\[\begin{split}\begin{align} \bar{x} & = \frac{1}{N}||\mathbf{X}||_{\ell1} & \text{$\bar{x}$ is a scalar} \\ & & \text{$N$ is number of elements in all channels of $\mathbf{X}$}\\ \mathbf{Y} & = \text{sign}\big(\mathbf{X}\big) \cdot \bar{x} &\\ \end{align}\end{split}\]Default backward is:
\[\frac{\partial Loss}{\partial \mathbf{X}} = \frac{\partial Loss}{\partial \mathbf{Y}}\]- Parameters
backward (callable) – Be used in backpropagation.
dtype (tf.DType) – Define the data type of args of forward and backward.
- Returns
forward function (grad_func defined).
- Return type
callable
-
blueoil.quantizations.linear_mid_tread_half_quantizer(bit=None, max_value=None, backward=None, dtype=tf.float32) Linear mid tread half quantizer.
This quantization creates a linear mid tread half quantizer. If backward is provided, this backward will be used in backpropagation.
This quantization method is DoReFa-Net [1]_ activation quantization variant, the difference from DoReFa-Net [1]_ is to be able to change max_value.
op_type is
LinearMidTreadHalfQuantizer.Forward is:
\[\begin{split}\mathbf{X} & = \text{clip}\big(\mathbf{X}, 0, max\_value\big)\\ \mathbf{Y} & = \begin{cases} \mathbf{X}, & \text{if $bit$ is 32} \\ \frac{\text{round}\big(\frac{\mathbf{X}}{max\_value} \cdot (2^{bit}-1)\big)}{2^{bit}-1} \cdot max\_value, & otherwise \end{cases}\end{split}\]Default backward is:
\[\begin{split}\frac{\partial Loss}{\partial \mathbf{X}} = \begin{cases} \frac{\partial Loss}{\partial y}, & \text{if $0 < x < max\_value$}\\ 0, & otherwise \end{cases}\end{split}\]- Parameters
bit (int) – Specify the bit of quantization.
max_value (int) – Be used for shift and clip.
backward (callable) – Be used in backpropagation.
dtype (tf.DType) – Define the data type of args of forward and backward.
- Returns
forward function (grad_func defined).
- Return type
callable
-
blueoil.quantizations.ttq_weight_quantizer(threshold=0.05, dtype=tf.float32) Trained Ternary Quantization (TTQ)
-
blueoil.quantizations.twn_weight_quantizer(threshold=0.7, dtype=tf.float32) Ternary Weight Networks (TWN)