INNER CODE UNIT · Python

grad

horovod/horovod · horovod/_keras/__init__.py:196

                                        grad = tf.IndexedSlices(grad.values / float(horovod_size), grad.indices, grad.dense_shape)
                                    else:
                                        grad /= float(horovod_size)
                                    v2g[v_ref] = grad

                        return [v2g[rv.ref()] for rv in vars]
                    else:
                        for rv, rg in zip(rv, rg):
                            v2g[rv] = rg

                        if self.scale_local_gradients and len(self._local_vars):
                            # Scale local gradients by a size factor. See pull/3695 and discussions/3705 for context.
                            for v in v2g:
                                if v in self._local_vars and v2g[v] is not None:
                                    grad = v2g[v]
                                    if isinstance(grad, tf.IndexedSlices):
                                        grad = tf.IndexedSlices(grad.values / float(horovod_size), grad.indices, grad.dense_shape)
                                    else:

View source record →