INNER CODE UNIT · Python
allreduce
horovod/horovod · horovod/keras/__init__.py:205
def allreduce(value, name=None, average=True, prescale_factor=1.0, postscale_factor=1.0,
op=None, compression=Compression.none):
"""
Perform an allreduce on a tensor-compatible value.
Arguments:
value: A tensor-compatible value to reduce.
The shape of the input must be identical across all ranks.
name: Optional name for the constants created by this operation.
average: If True, computes the average over all ranks.
Otherwise, computes the sum over all ranks.
prescale_factor: Multiplicative factor to scale tensor before allreduce.
postscale_factor: Multiplicative factor to scale tensor after allreduce.
op: The reduction operation to combine tensors across different ranks.
compression: Gradient compression algorithm to be used during allreduce.
Defaults to Compression.none.
"""
return _impl.allreduce(K, value, name, average, prescale_factor, postscale_factor, op, compression)