Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems demonstrate the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.Comment: 7 pages, 5 figures, DAI 201

DC-electric-field-induced and low-frequency electromodulation second-harmonic generation spectroscopy of Si(001)-SiO2_2 interfaces

The mechanism of DC-Electric-Field-Induced Second-Harmonic (EFISH) generation at weakly nonlinear buried Si(001)-SiO$_2$ interfaces is studied experimentally in planar Si(001)-SiO$_2$-Cr MOS structures by optical second-harmonic generation (SHG) spectroscopy with a tunable Ti:sapphire femtosecond laser. The spectral dependence of the EFISH contribution near the direct two-photon $E_1$ transition of silicon is extracted. A systematic phenomenological model of the EFISH phenomenon, including a detailed description of the space charge region (SCR) at the semiconductor-dielectric interface in accumulation, depletion, and inversion regimes, has been developed. The influence of surface quantization effects, interface states, charge traps in the oxide layer, doping concentration and oxide thickness on nonlocal screening of the DC-electric field and on breaking of inversion symmetry in the SCR is considered. The model describes EFISH generation in the SCR using a Green function formalism which takes into account all retardation and absorption effects of the fundamental and second harmonic (SH) waves, optical interference between field-dependent and field-independent contributions to the SH field and multiple reflection interference in the SiO$_2$ layer. Good agreement between the phenomenological model and our recent and new EFISH spectroscopic results is demonstrated. Finally, low-frequency electromodulated EFISH is demonstrated as a useful differential spectroscopic technique for studies of the Si-SiO$_2$ interface in silicon-based MOS structures.Comment: 31 pages, 14 figures, 1 table, figures are also available at http://kali.ilc.msu.su/articles/50/efish.ht

Destabilized SMC5/6 complex leads to chromosome breakage syndrome with severe lung disease

The structural maintenance of chromosomes (SMC) family of proteins supports mitotic proliferation, meiosis, and DNA repair to control genomic stability. Impairments in chromosome maintenance are linked to rare chromosome breakage disorders. Here, we have identified a chromosome breakage syndrome associated with severe lung disease in early childhood. Four children from two unrelated kindreds died of severe pulmonary disease during infancy following viral pneumonia with evidence of combined T and B cell immunodeficiency. Whole exome sequencing revealed biallelic missense mutations in the NSMCE3 (also known as NDNL2) gene, which encodes a subunit of the SMC5/6 complex that is essential for DNA damage response and chromosome segregation. The NSMCE3 mutations disrupted interactions within the SMC5/6 complex, leading to destabilization of the complex. Patient cells showed chromosome rearrangements, micronuclei, sensitivity to replication stress and DNA damage, and defective homologous recombination. This work associates missense mutations in NSMCE3 with an autosomal recessive chromosome breakage syndrome that leads to defective T and B cell function and acute respiratory distress syndrome in early childhood

Exploring the Impact of R&D on Patenting Activity in Small Women-Owned and Minority-Owned Entrepreneurial Firms.

The relevant economics literature on the impact of R&D on patenting activity falls within two methodological areas of inquiry. The first area might be classified as a test of the Schumpeterian hypothesis. The second and lesser research area might be classified as an estimation of the knowledge production function relationship between R&D and patenting. This paper focuses on estimates of the R&D-to-patenting relationship for a random sample of small, entrepreneurial firms whose research projects were supported through the US Small Business Innovation Research (SBIR) program. Our paper contributes to the R&D-to-patenting literature in two ways. It examines empirically a unique set of small, entrepreneurial firms funded by the public sector, and it explores the effect of the gender and ethnicity of firm owners on the propensity of their firms to patent from funded research projects

The impact of public R&D investments on patenting activity: technology transfer at the U.S. Environmental Protection Agency.

This paper presents estimates of the impact of public R&D on patenting activity at the U.S. Environmental Protection Agency (EPA). Using a time series of public sector agency data, we estimate the per-capita R&D elasticity of new patent applications using a knowledge production function framework model that is an expanded version of what other scholars have used with private sector data. New patent applications are an important step in the technology transfer activities of a federal agency. We estimate this elasticity to be about 2.0. This elasticity value represents an initial estimate of the impact of EPA’s R&D investments on its technology transfer activity

Knowledge begets knowledge: university knowledge spillovers and the output of scientific papers from U.S. Small Business Innovation Research (SBIR) projects

Scientific papers submitted for publication from U.S. Small Business Innovation Research (SBIR)-funded research projects are an innovative output that has yet to be studied systematically. Using a knowledge production framework, we identify empirically covariates with the number of scientific papers resulting from SBIR projects over the period 1992 through 2001. We find empirically that when the firm involves a university in its funded project, more scientific papers result. When the form of university involvement is taken into account, we find the greatest impact on the output of scientific papers comes from the inclusion of an individual from the university who originally developed the technology being pursued by the firm in its SBIR project. In other words, the project-specific technical human capital knowledge from the university that spills over to the firm’s project begets (i.e., brings about) additional knowledge in the form of scientific papers submitted for publication

professionals met een publieke taak en de omgang met overlast, agressie en geweld als gevolg van alcohol- en/of drugsgebruik

In this research the following question is answered: 'What are (the most) effective ways for supervisors, door men, police and other public service professionals to prevent, reduce or end substance related nuisance, aggression and violence?'Werknemers met een publieke taak worden regelmatig geconfronteerd met agressie onder invloed van alchol en drugs. Over de rol van middelen hierin en hoe hier het beste mee zou kunnen worden omgegaan, is echter weinig bekend. De volgende vraagstelling ligt ten grondslag aan dit onderzoek: 'Wat zijn (meest) effectieve manieren voor toezichthouders, portiers, politie en andere personen met een publieke taak om door alchol en/of drugs veroorzaakte overlast, agressie, geweld te voorkomen, in te dammen en te beëindigen?' Ter beantwoording is een literatuuronderzoek uitgevoerd, zijn experts (uit wetenschap en praktijk) geraadpleegd, zijn zes jongeren die frequent uitgaan, geïnterviewd en is een expertmeeting gehouden omtrent de toekomstige aanpak van geweld onder invloed. INHOUD: 1. Een onderzoek naar geweld onder invloed 2. De literatuur over geweld onder invloed 3. Experts over geweld onder invloed 4. Geweld onder invloed te beïnvloeden