Finite-width one hidden layer networks with multiple neurons in the readout layer display nontrivial output-output correlations that vanish in the lazy-training infinite-width limit. In this manuscript we leverage recent progress in the proportional limit of B... ...
