Dynamics of Supervised Learning with Restricted Training Sets

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the relevant performance measures, incorporating the old formalism in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves to single-layer networks and realizable tasks.Comment: 36 pages, latex2e, 12 eps figures (to be publ in: Proc Newton Inst Workshop on On-Line Learning '97

Dynamical Probability Distribution Function of the SK Model at High Temperatures

The microscopic probability distribution function of the Sherrington-Kirkpatrick (SK) model of spin glasses is calculated explicitly as a function of time by a high-temperature expansion. The resulting formula to the third order of the inverse temperature shows that an assumption made by Coolen, Laughton and Sherrington in their recent theory of dynamics is violated. Deviations of their theory from exact results are estimated quantitatively. Our formula also yields explicit expressions of the time dependence of various macroscopic physical quantities when the temperature is suddenly changed within the high-temperature region.Comment: LaTeX, 6 pages, Figures upon request (here revised), To be published in J. Phys. Soc. Jpn. 65 (1996) No.

Theory of agent-based market models with controlled levels of greed and anxiety

We use generating functional analysis to study minority-game type market models with generalized strategy valuation updates that control the psychology of agents' actions. The agents' choice between trend following and contrarian trading, and their vigor in each, depends on the overall state of the market. Even in `fake history' models, the theory now involves an effective overall bid process (coupled to the effective agent process) which can exhibit profound remanence effects and new phase transitions. For some models the bid process can be solved directly, others require Maxwell-construction type approximations.Comment: 30 pages, 10 figure

Statistical Mechanics of Dilute Batch Minority Games with Random External Information

We study the dynamics and statics of a dilute batch minority game with random external information. We focus on the case in which the number of connections per agent is infinite in the thermodynamic limit. The dynamical scenario of ergodicity breaking in this model is different from the phase transition in the standard minority game and is characterised by the onset of long-term memory at finite integrated response. We demonstrate that finite memory appears at the AT-line obtained from the corresponding replica calculation, and compare the behaviour of the dilute model with the minority game with market impact correction, which is known to exhibit similar features.Comment: 22 pages, 6 figures, text modified, references updated and added, figure added, typos correcte

Dynamical and Stationary Properties of On-line Learning from Finite Training Sets

The dynamical and stationary properties of on-line learning from finite training sets are analysed using the cavity method. For large input dimensions, we derive equations for the macroscopic parameters, namely, the student-teacher correlation, the student-student autocorrelation and the learning force uctuation. This enables us to provide analytical solutions to Adaline learning as a benchmark. Theoretical predictions of training errors in transient and stationary states are obtained by a Monte Carlo sampling procedure. Generalization and training errors are found to agree with simulations. The physical origin of the critical learning rate is presented. Comparison with batch learning is discussed throughout the paper.Comment: 30 pages, 4 figure

Effects of noise and confidence thresholds in nominal and metric Axelrod dynamics of social influence

We study the effects of bounded confidence thresholds and of interaction and external noise on Axelrod's model of social influence. Our study is based on a combination of numerical simulations and an integration of the mean-field Master equation describing the system in the thermodynamic limit. We find that interaction thresholds affect the system only quantitatively, but that they do not alter the basic phase structure. The known crossover between an ordered and a disordered state in finite systems subject to external noise persists in models with general confidence threshold. Interaction noise here facilitates the dynamics and reduces relaxation times. We also study Axelrod systems with metric features, and point out similarities and differences compared to models with nominal features. Metric features are used to demonstrate that a small group of extremists can have a significant impact on the opinion dynamics of a population of Axelrod agents.Comment: 15 pages, 12 figure

How glassy are neural networks?

In this paper we continue our investigation on the high storage regime of a neural network with Gaussian patterns. Through an exact mapping between its partition function and one of a bipartite spin glass (whose parties consist of Ising and Gaussian spins respectively), we give a complete control of the whole annealed region. The strategy explored is based on an interpolation between the bipartite system and two independent spin glasses built respectively by dichotomic and Gaussian spins: Critical line, behavior of the principal thermodynamic observables and their fluctuations as well as overlap fluctuations are obtained and discussed. Then, we move further, extending such an equivalence beyond the critical line, to explore the broken ergodicity phase under the assumption of replica symmetry and we show that the quenched free energy of this (analogical) Hopfield model can be described as a linear combination of the two quenched spin-glass free energies even in the replica symmetric framework

News and price returns from threshold behaviour and vice-versa: exact solution of a simple agent-based market model

Starting from an exact relationship between news, threshold and price return distributions in the stationary state, I discuss the ability of the Ghoulmie-Cont-Nadal model of traders to produce fat-tailed price returns. Under normal conditions, this model is not able to transform Gaussian news into fat-tailed price returns. When the variance of the news so small that only the players with zero threshold can possibly react to news, this model produces Levy-distributed price returns with a -1 exponent. In the special case of super-linear price impact functions, fat-tailed returns are obtained from well-behaved news.Comment: 4 pages, 3 figures. This is quite possibly the final version. To appear in J. Phys

Statistical mechanics and stability of a model eco-system

We study a model ecosystem by means of dynamical techniques from disordered systems theory. The model describes a set of species subject to competitive interactions through a background of resources, which they feed upon. Additionally direct competitive or co-operative interaction between species may occur through a random coupling matrix. We compute the order parameters of the system in a fixed point regime, and identify the onset of instability and compute the phase diagram. We focus on the effects of variability of resources, direct interaction between species, co-operation pressure and dilution on the stability and the diversity of the ecosystem. It is shown that resources can be exploited optimally only in absence of co-operation pressure or direct interaction between species.Comment: 23 pages, 13 figures; text of paper modified, discussion extended, references adde