Information Theory and its Relation to Machine Learning

In this position paper, I first describe a new perspective on machine learning (ML) by four basic problems (or levels), namely, "What to learn?", "How to learn?", "What to evaluate?", and "What to adjust?". The paper stresses more on the first level of "What to learn?", or "Learning Target Selection". Towards this primary problem within the four levels, I briefly review the existing studies about the connection between information theoretical learning (ITL [1]) and machine learning. A theorem is given on the relation between the empirically-defined similarity measure and information measures. Finally, a conjecture is proposed for pursuing a unified mathematical interpretation to learning target selection.Comment: 10 pages, 6 figures, 1 tabl

Singularity formation in three-dimensional vortex sheets

We study singularity formation of three-dimensional (3-D) vortex sheets without surface tension using a new approach. First, we derive a leading order approximation to the boundary integral equation governing the 3-D vortex sheet. This leading order equation captures the most singular contributions of the integral equation. By introducing an appropriate change of variables, we show that the leading order vortex sheet equation degenerates to a two-dimensional vortex sheet equation in the direction of the tangential velocity jump. This change of variables is guided by a careful analysis based on properties of certain singular integral operators, and is crucial in identifying the leading order singular behavior. Our result confirms that the tangential velocity jump is the physical driving force of the vortex sheet singularities. We also show that the singularity type of the three-dimensional problem is similar to that of the two-dimensional problem. Moreover, we introduce a model equation for 3-D vortex sheets. This model equation captures the leading order singularity structure of the full 3-D vortex sheet equation, and it can be computed efficiently using fast Fourier transform. This enables us to perform well-resolved calculations to study the generic type of 3-D vortex sheet singularities. We will provide detailed numerical results to support the analytic prediction, and to reveal the generic form of the vortex sheet singularity