Statistical Mechanics of Support Vector Networks

Using methods of Statistical Physics, we investigate the generalization performance of support vector machines (SVMs), which have been recently introduced as a general alternative to neural networks. For nonlinear classification rules, the generalization error saturates on a plateau, when the number of examples is too small to properly estimate the coefficients of the nonlinear part. When trained on simple rules, we find that SVMs overfit only weakly. The performance of SVMs is strongly enhanced, when the distribution of the inputs has a gap in feature space.Comment: REVTeX, 4 pages, 2 figures, accepted by Phys. Rev. Lett (typos corrected

A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices

We consider the problem of solving TAP mean field equations by iteration for Ising model with coupling matrices that are drawn at random from general invariant ensembles. We develop an analysis of iterative algorithms using a dynamical functional approach that in the thermodynamic limit yields an effective dynamics of a single variable trajectory. Our main novel contribution is the expression for the implicit memory term of the dynamics for general invariant ensembles. By subtracting these terms, that depend on magnetizations at previous time steps, the implicit memory terms cancel making the iteration dependent on a Gaussian distributed field only. The TAP magnetizations are stable fixed points if an AT stability criterion is fulfilled. We illustrate our method explicitly for coupling matrices drawn from the random orthogonal ensemble.Comment: 27 pages, 6 Figures Published in Journal of Physics A: Mathematical and Theoretical, Volume 49, Number 11, 201

Expectation Propagation for Approximate Inference: Free Probability Framework

We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application of EP to large problem sizes. In this study, we present a rigorous analysis by means of free probability theory that allows us to overcome this computational bottleneck if specific data matrices in the problem fulfill certain properties of asymptotic freeness. We demonstrate the relevance of our approach on the gene selection problem of a microarray dataset.Comment: Both authors are co-first authors. The main body of this paper is accepted for publication in the proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT

The fabrication and test of a dual spin gas bearing reaction wheel

The design and fabrication of a dual spin gas bearing reaction wheel are discussed. Numerical analyses, data, and conclusions from performance tests are reported. The unique feature of the reaction wheel is the dual gas bearing concept in which two sets of self-acting hydrodynamic bearing are used to obtain stictionless operation and low noise around zero speed and to accommodate the momentum range from plus 6.8 N-m-s to minus 6.8 N-m-s with the potential for long life inherent in gas bearings

Integrating Research and Quality Improvement Using TeamSTEPPS: A Health Team Communication Project to Improve Hospital Discharge

Purpose/Objectives: The purpose of this article is to describe an innovative approach to the integration of quality improvement and research processes. A project with the objective of improving health team communication about hospital discharge provides an exemplar case. Description of the Project/Program: The TeamSTEPPS 10-step action planning guide provided the structure for planning, developing, and evaluating a redesign of interprofessional health team communication to improve hospital discharge led by 2 clinical nurse specialists. The redesign involved development of processes for team bedside rounding, registered nurse bedside shift reports, and briefing tools to support the rounding processes. Outcome: Using the TeamSTEPPS process, a 4-phase combined quality improvement and research project was designed and implemented. Implementation is ongoing, supported by process evaluation for continuing process improvement. Longitudinal analysis of research outcomes will follow in the future. Conclusions: Led by unit-based clinical nurse specialists, use of an integrated process of quality improvement and research creates evidence-based innovation to solve interprofessional practice problems. Incorporating research within the project design allows for data-based decisions to inform the clinical process improvement, as well as documentation of both the processes and outcomes of the local improvements that can inform replications in other sites

Phase Transitions of Neural Networks

The cooperative behaviour of interacting neurons and synapses is studied using models and methods from statistical physics. The competition between training error and entropy may lead to discontinuous properties of the neural network. This is demonstrated for a few examples: Perceptron, associative memory, learning from examples, generalization, multilayer networks, structure recognition, Bayesian estimate, on-line training, noise estimation and time series generation.Comment: Plenary talk for MINERVA workshop on mesoscopics, fractals and neural networks, Eilat, March 1997 Postscript Fil