The Divergence of Reinforcement Learning Algorithms with Value-Iteration and Function Approximation

This paper gives specific divergence examples of value-iteration for several major Reinforcement Learning and Adaptive Dynamic Programming algorithms, when using a function approximator for the value function. These divergence examples differ from previous divergence examples in the literature, in that they are applicable for a greedy policy, i.e. in a "value iteration" scenario. Perhaps surprisingly, with a greedy policy, it is also possible to get divergence for the algorithms TD(1) and Sarsa(1). In addition to these divergences, we also achieve divergence for the Adaptive Dynamic Programming algorithms HDP, DHP and GDHP.Comment: 8 pages, 4 figures. In Proceedings of the IEEE International Joint Conference on Neural Networks, June 2012, Brisbane (IEEE IJCNN 2012), pp. 3070--307

Slope stability in slightly fissured claystones and marls

The final publication is available at Springer via http://dx.doi.org/10.1007/s10346-014-0526-5The paper deals with the behavior of some characteristic soft rocks found in the Iberian Peninsula. In geological terms, they belong to Tertiary basins, the Keuper period and the Jurassic-Cretacic transition. The discussion is organized around the following aspects: (a) the intact material and its brittle behavior; (b) the weathering action of atmospheric events; (c) the persistent discontinuities and scale effects; and (d) the modification of strength after failure. In all cases, instability phenomena are addressed in connection with several case histories. Regarding material brittleness and the initial stress state, two cases of first time failures are discussed. Practical implications concerning the selection of operative strength will be given. Field observations of the relevance of weathering and, also, on the rate of weathering, are given for a Weald claystone. Field observations emphasize the importance of sharp transitions between weathered and intact (or slightly weathered) levels. A recent long-term laboratory investigation on the nature of degradation will be summarized. Macroscopic variables such as stiffness and tensile strength have been found to be uniquely predicted by a degradation law in terms of the accumulated plastic deformations. Persistent discontinuities and, in particular, sedimentation planes play a dominant role to explain slope failures not related to the shallow failures, usually associated with weathered profiles. The strength of discontinuities in a Weald formation was investigated by means of tests performed at two scales. Finally, the evolution in time of residual strength induced by chemical actions, associated with groundwater flow, is highlighted in connection with actual field data of unstable slopes.Peer ReviewedPostprint (author's final draft

Criteria for rapid sliding I.: a review of Vaiont case

Vaiont slide has been represented by a model of two interacting evolutive wedges. Mass conservation during the motion implies that the upper wedge transfers mass to the lower one through an internal shearing plane. The model respects available in situ observations. It was formulated in dynamics terms. Outcomes of the analysis are the determination of safety factors of the valley before dam impoundment, and the calculation of run-out distance once the motion starts. Rock strength degradation as motion develops has also been included. This degradation, even if it is intense, was unable to explain the very high estimated landslide velocities.Peer ReviewedPostprint (author's final draft

Criteria for rapid sliding II.: thermo-hydro-mechanical and scale effects in Vaiont case

Thermally induced excess pore pressures have been included into a two-wedge evolutive model of Vaiont landslide. The problem requires the solution of a system of four coupled balance equations for the shear bands and the surrounding rock as well as the joint equation of motion of the entire slide. The model predicts the high velocities observed and is consistent with other data (slide geometry, residual strength, and conditions on the sliding surface). The interpretation of a sensitivity analysis suggests that there exists a threshold permeability band, in the range 10- 8 to 10- 10 m/s, which separates potentially fast motions from slow motions. This conclusion is maintained if the scale of the landslide is reduced.Peer ReviewedPostprint (author's final draft

Review. Fausto Quintana Solórzano (coord.). Sociedad global, crisis ambiental y sistemas socio-ecológicos. México, Facultad de Ciencias Políticas y Sociales, Universidad Nacional Autónoma de México, 2019

Reseña. Fausto Quintana Solórzano (coord.). Sociedad global, crisis ambiental y sistemas socio-ecológicos. México, Facultad de Ciencias Políticas y Sociales, Universidad Nacional Autónoma de México, 2019.Review. Fausto Quintana Solórzano (coord.). Sociedad global, crisis ambiental y sistemas socio-ecológicos. México, Facultad de Ciencias Políticas y Sociales, Universidad Nacional Autónoma de México, 2019

Run-out of landslides in brittle soils

One of the factors causing the acceleration of landslides is the loss of strength of the soil involved in the potential unstable mechanism. The travelled distance and the landslide velocity, a key factor in risk analysis, will be determined by the loss of resistant forces. Brittle behaviour, commonly associated with cemented soils, overconsolidated plastic clay formations and sensitive clays, lead to the progressive failure phenomenon explained by the reduction of the strength with increasing strain. In the present study, this phenomenon has been analysed in the case of a saturated slope which becomes unstable by increasing the boundary pore water pressure. A Mohr-Coulomb model with strain softening behaviour induced by increasing deviatoric plastic strain is used. The paper focusses not only on the stability of the slope but also on the post failure behaviour (run-out and sliding velocity). A coupled hydro-mechanical formulation of the material point method has been used to simulate the whole instability process. The influence of the brittleness of the material on the triggering of instability and run-out is evaluated by means of a parametric study varying peak and residual strength. The onset of the failure and the failure geometry are controlled by both peak and residual values. Good correlations between run-outs and brittleness are found. The decay of the strength determines the acceleration of the landslides and the travelled distance.Peer ReviewedPostprint (author's final draft

Landslide motion assessment including rate effects and thermal interactions: revisiting the Canelles landslide

The re-activation of a large (40 Mm3) landslide on the valley slopes of a reservoir motivated a research initiative to estimate the risk of a fast-sliding mass moving into the reservoir. A previous simplified analysis had suggested that a joint consideration of strain rate effects on friction and thermal pressurization phenomena in the sliding surface could provide a rational approach to answer the question raised. The paper describes first the capability of strain rate effects on friction to reproduce long-term creeping records of two real cases. The joint and coupled phenomena of creeping motion and thermal pressurization in shearing bands was incorporated into a material point method computational technique for hydromechanical analysis of porous materials. A representative cross section of the Canelles landslide was then analysed, profiting from previous finite element investigations of the landslide. It was found that a rapid rate of landslide acceleration could be a possibility under extreme external actions. However, it was also found that a moderate strain rate effect on the basal residual friction angle could create conditions that avoid the triggering of a fast motion.Peer ReviewedPostprint (author's final draft