8.1.1.8.1.1. blueoil.quantizations.binary

8.1.1.8.1.1.1. Module Contents

8.1.1.8.1.1.1.1. Functions

binary_channel_wise_mean_scaling_quantizer(backward=None, dtype=tf.float32)

Binary channel wise mean scaling quantizer.

binary_mean_scaling_quantizer(backward=None, dtype=tf.float32)

Binary mean scaling quantizer.

blueoil.quantizations.binary.binary_channel_wise_mean_scaling_quantizer(backward=None, dtype=tf.float32)

Binary channel wise mean scaling quantizer.

This quantization creates a binary channel wise mean scaling quantizer. If backward is provided, this backward will be used in backpropagation.

This method is varient of XNOR-Net 1 weight quantization, the difference from XNOR-Net 1 is backward function.

op_type is BinaryChannelWiseMeanScalingQuantizer.

Forward is:

\[\begin{split}\begin{align} \bar{\mathbf{x}} & = \frac{1}{n}||\mathbf{X}||_{\ell1} & \text{$\bar{\mathbf{x}}$ is a $c$-channels vector} \\ & & \text{$n$ is number of elements in each channel of $\mathbf{X}$} \\\\ \mathbf{Y} & = \text{sign}\big(\mathbf{X}\big) \times \bar{\mathbf{x}} &\\ \end{align}\end{split}\]

Default backward is:

\[\frac{\partial Loss}{\partial \mathbf{X}} = \frac{\partial Loss}{\partial \mathbf{Y}}\]
Parameters
  • backward (callable) – Be used in backpropagation.

  • dtype (tf.DType) – Define the data type of args of forward and backward.

Returns

forward function (grad_func defined).

Return type

callable

Reference:
1(1,2)

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

blueoil.quantizations.binary.binary_mean_scaling_quantizer(backward=None, dtype=tf.float32)

Binary mean scaling quantizer.

This quantization creates a binary mean scaling quantizer. If backward is provided, this backward will be used in backpropagation.

This method is DoReFa-Net 2 weight quantization.

op_type is BinaryMeanScalingQuantizer.

Forward is:

\[\begin{split}\begin{align} \bar{x} & = \frac{1}{N}||\mathbf{X}||_{\ell1} & \text{$\bar{x}$ is a scalar} \\ & & \text{$N$ is number of elements in all channels of $\mathbf{X}$}\\ \mathbf{Y} & = \text{sign}\big(\mathbf{X}\big) \cdot \bar{x} &\\ \end{align}\end{split}\]

Default backward is:

\[\frac{\partial Loss}{\partial \mathbf{X}} = \frac{\partial Loss}{\partial \mathbf{Y}}\]
Parameters
  • backward (callable) – Be used in backpropagation.

  • dtype (tf.DType) – Define the data type of args of forward and backward.

Returns

forward function (grad_func defined).

Return type

callable

Reference:
2

DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients