Table of Notations

H̃ (l) , Θ̃ (l) Quantized activation / parameter `(H,Y) loss function of prediction H and label Y . ∇Θ` Gradient of ` w.r.t. Θ J Jacobian matrix ∂vec(H ) ∂vec(H̃ (l−1) ) K Jacobian matrix ∂vec(H ) ∂vec(Θ̃ (l) ) ∇H(l) ,∇Θ(l) ,∇Θ QAT gradient for activation / parameter ∇̂H(l) , ∇̂Θ(l) , ∇̂Θ FQT gradient for activation / parameter ∇ h (l) i , ∇̂ h (l) i i-th row of QAT / FQT activation gradient at l-th layer E [X | Y ] Conditional expectation of X given Y Var [X | Y ] Conditional variance of X given Y R(X) Dynamic range of X, i.e., maxX−minX b, B Number of quantization bits / bins