On Reinforcement Learning for Full-length Game of StarCraft

StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learning approach for StarCraft II. The hierarchy involves two levels of abstraction. One is the macro-action automatically extracted from expert's trajectories, which reduces the action space in an order of magnitude yet remains effective. The other is a two-layer hierarchical architecture which is modular and easy to scale, enabling a curriculum transferring from simpler tasks to more complex tasks. The reinforcement training algorithm for this architecture is also investigated. On a 64x64 map and using restrictive units, we achieve a winning rate of more than 99\% against the difficulty level-1 built-in AI. Through the curriculum transfer learning algorithm and a mixture of combat model, we can achieve over 93\% winning rate of Protoss against the most difficult non-cheating built-in AI (level-7) of Terran, training within two days using a single machine with only 48 CPU cores and 8 K40 GPUs. It also shows strong generalization performance, when tested against never seen opponents including cheating levels built-in AI and all levels of Zerg and Protoss built-in AI. We hope this study could shed some light on the future research of large-scale reinforcement learning.Comment: Appeared in AAAI 201

Measure consumer preferences for pork attributes under different coverage in China

Media reports could help shape consumer attitudes towards food quality and safety. By introducing an information treatment with positive or negative media coverage, we study the impact on consumer preference for pork products. The hypothesis is tested by a hypothetical choice experiment with 788 samples in 15 cities in China. Attributes we take into account include traceability, farming style, brand and certificates, in addition to prices. The results indicate that the media coverage could significantly shape consumers' preference. A comparison of the two treatments indicates that the positive information treatment could yield smaller WTP values for all attributes related to food quality and safety

Induced-charge electroosmosis around conducting and Janus cylinder in microchip

The induced-charge elecetroosmosis around conducting/Janus cylinder with arbitrary Debye thickness is studied numerically, when an direct current weak electric filed is suddenly applied in a confined microchannel. It’s found that there are four large circulations around the conducting cylinder, and the total flux in the microchannel is zero; there are two smaller circulations around the Janus cylinder, and they are compressed to wall. A bulk flux, which has a parabolic relation with the applied electric field, is also predicted

Synaptic vesicle dynamics in mouse rod bipolar cells.

To better understand synaptic signaling at the mammalian rod bipolar cell terminal and pave the way for applying genetic approaches to the study of visual information processing in the mammalian retina, synaptic vesicle dynamics and intraterminal calcium were monitored in terminals of acutely isolated mouse rod bipolar cells and the number of ribbon-style active zones quantified. We identified a releasable pool, corresponding to a maximum of 7 s. The presence of a smaller, rapidly releasing pool and a small, fast component of refilling was also suggested. Following calcium channel closure, membrane surface area was restored to baseline with a time constant that ranged from 2 to 21 s depending on the magnitude of the preceding Ca2+ transient. In addition, a brief, calcium-dependent delay often preceded the start of onset of membrane recovery. Thus, several aspects of synaptic vesicle dynamics appear to be conserved between rod-dominant bipolar cells of fish and mammalian rod bipolar cells. A major difference is that the number of vesicles available for release is significantly smaller in the mouse rod bipolar cell, both as a function of the total number per neuron and on a per active zone basis

Multicolor Photometry of the Nearby Galaxy Cluster A119

This paper presents multicolor optical photometry of the nearby galaxy cluster Abell 119 (z = 0:0442) with the Beijing-Arizona-Taiwan-Connecticut (BATC) system of 15 intermediate bands. Within the BATC viewing field of 58'* 58', there are 368 galaxies with known spectroscopic redshifts, including 238 member galaxies (called sample I). Based on the spectral energy distributions (SEDs) of 1376 galaxies brighter than iBATC = 19:5, photometric redshift technique and the color-magnitude relation of earlytype galaxies are applied to select faint member galaxies. As a result, 117 faint galaxies were selected as new member galaxies. Combined with sample I, an enlarged sample (called sample II) of 355 member galaxies is obtained. Spatial distribution and localized velocity structure for two samples demonstrate that A119 is a dynamically complex cluster with at least three prominent substructures in the central region within 1 Mpc. A large velocity dispersion for the central clump indicates a merging along the line of sight. No significant evidences for morphology and luminosity segregations are found in both samples. With the evolutionary synthesis model PEGASE, environmental effect on the star formation properties is confirmed. Faint galaxies in low-density region tend to have longer time scales of star formation, smaller mean stellar ages, and lower metallicities of interstellar medium, which is in agreement with the context of hierarchical cosmological scenario.Comment: 21 pages, 11 figures and 4 tables. Accepted for publication in RA