Large Scale Private Learning via Low-rank Reparametrization
Da Yu 1 2 Huishuai Zhang 2 Wei Chen 2 Jian Yin 1 Tie-Yan Liu 2
Abstract
We propose a reparametrization scheme to ad-
dress the challenges of applying differentially
private SGD on large neural networks, which
are 1) the huge memory cost of storing individ-
ual gradients, 2) the added noise suffering no-
torious dimensional dependence. Specifically,
we reparametrize each weight mat ...
附件列表