Learning Two-layer Neural Networks with Symmetric Inputs

We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network $$y = A \sigma(Wx) + \xi,$$ where $A,W$ are weight matrices, $\xi$ represents noise, and the number of neurons in the hidden layer is no larger than the input or output, our algorithm is guaranteed to recover the parameters $A,W$ of the ground-truth network. The only requirement on the input $x$ is that it is symmetric, which still allows highly complicated and structured input. Our algorithm is based on the method-of-moments framework and extends several results in tensor decompositions. We use spectral algorithms to avoid the complicated non-convex optimization in learning neural networks. Experiments show that our algorithm can robustly learn the ground-truth neural network with a small number of samples for many symmetric input distributions

The Necessary and Sufficient Conditions of Separability for Multipartite Pure States

In this paper we present the necessary and sufficient conditions of separability for multipartite pure states. These conditions are very simple, and they don't require Schmidt decomposition or tracing out operations. We also give a necessary condition for a local unitary equivalence class for a bipartite system in terms of the determinant of the matrix of amplitudes and explore a variance as a measure of entanglement for multipartite pure states.Comment: Submitted to PRL in Sep. 2004, the paper No is LV9637. Submitted to SIAM on computing, in Jan., 2005, the paper No. is SICOMP 44687. Under reviewing no