- InplaceClipGradNorm#

InplaceClipGradNorm - 1 (


This version of the operator has been available since version 1 of domain


InplaceClipGradNorm operator, taking multiple gradients as inputs (seq<tensor>). InplaceClipGradNorm should be used in conjunction with optimizers that accept seq<tensor> gradients as input, since this op takes a sequence of tensors as input and outputs a sequence of tensors there by avoiding the need for SequenceConstruct (and making any unnecessary copy).Please note that the gradient clipping happens inplace.


  • max_norm: Coefficient of previously accumulated gradient in running average. Default value is ?.

  • norm_type: Type of normalization to perform during execution of clip grad norm.Currently, the only norm supported is the frobenius norm (which is also the default). Default value is ?.


  • gradients (heterogeneous) - S_GRAD: Sequence of gradients computed in this iteration.


  • clipped_gradients (heterogeneous) - S_GRAD: Gradients after being clipped as per given inputs and attributes.