com.microsoft - InplaceClipGradNorm#
This version of the operator has been available since version 1 of domain com.microsoft.
InplaceClipGradNorm operator, taking multiple gradients as inputs (seq<tensor>). InplaceClipGradNorm should be used in conjunction with optimizers that accept seq<tensor> gradients as input, since this op takes a sequence of tensors as input and outputs a sequence of tensors there by avoiding the need for SequenceConstruct (and making any unnecessary copy).Please note that the gradient clipping happens inplace.
max_norm: Coefficient of previously accumulated gradient in running average. Default value is
norm_type: Type of normalization to perform during execution of clip grad norm.Currently, the only norm supported is the frobenius norm (which is also the default). Default value is
gradients (heterogeneous) - S_GRAD: Sequence of gradients computed in this iteration.
clipped_gradients (heterogeneous) - S_GRAD: Gradients after being clipped as per given inputs and attributes.