Whitening and Second Order Optimization Both Make Information in the
Dataset Unusable During Training, and Can Reduce or Prevent Generalization

     Neha S. Wadia 1 Daniel Duckworth 2 Samuel S. Schoenholz 2 Ethan Dyer 2 Jascha Sohl-Dickstein 2

             Abstract                 reduction hypothesis (Attneave, 1954; Barlow, 1961; Atick
    Machine learning is predicated on the concept of       & Redlich, 1992; Dan et al., 1996; Simoncelli & Olshausen,
    generalization: a model achieving lo ...