Comparison of Reinforcement Learning algorithms applied to the Cart Pole problem

Designing optimal controllers continues to be challenging as systems are becoming complex and are inherently nonlinear. The principal advantage of reinforcement learning (RL) is its ability to learn from the interaction with the environment and provide optimal control strategy. In this paper, RL is explored in the context of control of the benchmark cartpole dynamical system with no prior knowledge of the dynamics. RL algorithms such as temporal-difference, policy gradient actor-critic, and value function approximation are compared in this context with the standard LQR solution. Further, we propose a novel approach to integrate RL and swing-up controllers

Intravenously delivered graphene nanosheets and multiwalled carbon nanotubes induce site-specific Th2 inflammatory responses via the IL-33/ST2 axis

Carbon-based nanomaterials (CBN), such as graphene nanosheets (GNS) and multiwalled carbon nanotubes (MWCNT), have been proposed for potential nanomedicine applications such as biomedical devices and carriers for drug delivery. However, our current understanding regarding the systemic toxicity of these CBN through intravenous (iv) injection is limited. In this study, we compare the immune response resulting from GNS and MWCNT exposure. We hypothesize that iv administration of GNS and MWCNT would result in divergent systemic inflammatory responses due to physicochemical differences between these two CBN. In the lungs of C57BL/6 mice, GNS actuate a Th2 immune response 1 day following iv administration, which consists of neutrophilic influx and a significant increase in interleukin (IL)-5, IL-13, IL-33, and its soluble receptor (sST2) in the bronchoalveolar lavage fluid. MWCNT elicited a significant increase in the messenger ribonucleic acid expression of cytokines in the spleen including IL-4 and IL-33, which are associated with an increase in splenic cell differentiation (CD)4+ and CD8+ T-cells in C57BL/6 mice following iv injection. The observed Th2 responses in both the lung and spleen are absent in ST2−/− mice administrated GNS or MWCNT, suggesting a critical role for IL-33. In conclusion, the use of GNS or MWCNT as nanocarriers for drug delivery may result in Th2 immune responses that are mediated through the IL-33/ST2 axis and therefore may promote adverse allergic reactions

Defect-engineered graphene for bulk supercapacitors with high energy and power densities

The development of high-energy and high-power density supercapacitors (SCs) is critical for enabling next-generation energy storage applications. Nanocarbons are excellent SC electrode materials due to their economic viability, high-surface area, and high stability. Although nanocarbons have high theoretical surface area and hence high double layer capacitance, the net amount of energy stored in nanocarbon-SCs is much below theoretical limits due to two inherent bottlenecks: i) their low quantum capacitance and ii) limited ion-accessible surface area. Here, we demonstrate that defects in graphene could be effectively used to mitigate these bottlenecks by drastically increasing the quantum capacitance and opening new channels to facilitate ion diffusion in otherwise closed interlayer spaces. Our results support the emergence of a new energy paradigm in SCs with 250% enhancement in double layer capacitance beyond the theoretical limit. Furthermore, we demonstrate prototype defect engineered bulk SC devices with energy densities 500% higher than state-of-the-art commercial SCs without compromising the power density.Comment: 15 pages, 5 figures, and 8 supplemental figure

Defects induced ferromagnetism in Mn doped ZnO

Single phase Mn doped (2 at %) ZnO samples have been synthesized by solid-state reaction technique. Before the final sintering at 500 C, the mixed powders have been milled for different milling periods (6, 24, 48 and 96 hours). The grain sizes of the samples are very close to each other (~ 32 \pm 4 nm). However, the defective state of the samples is different from each other as manifested from the variation of magnetic properties and electrical resistivity with milling time. All the samples have been found to be ferromagnetic with clear hysteresis loops at room temperature. The maximum value for saturation magnetization (0.11 {\mu}_B / Mn atom) was achieved for 96 hours milled sample. Electrical resistivity has been found to increase with increasing milling time. The most resistive sample bears the largest saturation magnetization. Variation of average positron lifetime with milling time bears a close similarity with that of the saturation magnetization. This indicates the key role played by open volume vacancy defects, presumably zinc vacancies near grain surfaces, in inducing ferromagnetic order in Mn doped ZnO. To attain optimum defect configuration favorable for ferromagnetism in this kind of samples proper choice of milling period and annealing conditions is required.Comment: Accepted in Journal of Magnetism and Magnetic Material

EFFECTS OF SURFACE STATES, DEFECTS AND DOPANTS ON THE OPTICAL AND MAGNETIC PROPERTIES OF LOW-DIMENSIONAL MATERIALS

Nanomaterials have attracted the attention of researchers from various fields due to their unique features (that are otherwise absent in the bulk) such as quantum confinement, high surface to volume ratio, ability for surface modification etc. Since the discovery of fullerenes and carbon nanotubes, several synthesis techniques have been developed for nanomaterial growth. However, different control parameters in different synthesis techniques often result in nanostructures with varying defects that may alter their fundamental behavior. Such defects or disorder in the crystal lattice can lead to the disruption of lattice symmetry. The defect-induced symmetry lowering (or breaking) effects play a vital role in the determination of fundamental material characteristics. Thus, it is very important to characterize the defects in order to understand their effects on the nanomaterial properties. This thesis describes the effects of defects in low dimesional systems such as ZnO nanowires, graphene and carbon nanotubes are studied. Firstly, it describes the synthesis and characterization of ZnO nanostructures and discusses the effects of surface states, defects and dopants on their optical and magnetic properties. An unexpected presence of ferromagnetic (FM) ordering in nanostructured nonmagnetic metal oxides has been reported previously. Though this property was attributed to the presence of defects, systematic experimental and theoretical studies to pinpoint its origin and mechanism were lacking. While it is widely believed that oxygen vacancies are responsible for FM ordering, surprisingly annealing as-prepared samples at low temperature (high temperature) in flowing oxygen actually enhances (diminishes) the FM ordering. For these reasons, we have prepared, annealed in different environments, and measured the ensuing magnetization in micrometer and nanoscale ZnO with varying crystallinity. We further find from our magnetization measurements and ab-initio calculations that a range of magnetic properties in ZnO can result, depending on the sample preparation and annealing conditions. For example, within the same ZnO sample we have observed ferro- to para- and diamagnetic responses depending on the annealing conditions. We also explored the effects of surface states on the magnetic behavior of nanoscale ZnO through detailed calculations. In the case of grapheme, we have observed new combination modes in the range from 1650 to 2300 cm−1 in single-(SLG), bi-, few-layer and incommensurate bilayer graphene (IBLG) on silicon dioxide substrates. A peak at 1860 cm−1 (iTALO−) is observed due to a combination of the in-plane transverse acoustic (iTA) and the longitudinal optical (LO) phonons. The intensity of this peak decreases with increasing number of layers and this peak is absent for bulk graphite. The overtone of the out-of-plane transverse optical (oTO) phonon at 1750 cm−1, also called the M band, is suppressed for both SLG and IBLG. In addition, two previously unidentified modes at 2200 and 1880 cm−1 are observed in SLG. The 2220 cm−1 (1880 cm−1) mode is tentatively assigned to the combination mode of in-plane transverse optical (iTO) and TA phonons (oTO+LO phonons) around the K point in the graphene Brillouin zone. Finally, the peak frequency of the 1880 (2220) cm−1 mode is observed to increase (decrease) linearly with increasing graphene layers. Finally, we find that the high curvature in sub-nm SWCNTs leads to (i) an unusual S-like dispersion of the G-band frequency due to perturbations caused by the strong electron-phonon coupling, (ii) an activation of diameter-selective intermediate frequency modes that are as intense as the radial breathing modes (RBMs), and (iii) a clear observation of the IR modes

Animating Predator and Prey Fish Interactions

Schooling behavior is one of the most salient social and group activities among fish. They form schools for social reasons like foraging, mating and escaping from predators. Animating a school of fish is difficult because they are large in number, often swim in distinctive patterns that is they take the shape of long thin lines, squares, ovals or amoeboid and exhibit complex coordinated patterns especially when they are attacked by a predator. Previous work in computer graphics has not provided satisfactory models to simulate the many distinctive interactions between a school of prey fish and their predator, how does a predator pick its target? and how does a school of fish react to such attacks? This dissertation presents a method to simulate interactions between prey fish and predator fish in the 3D world based on the biological research findings. Firstly, a model is described by representing a school of fish as a complex network information flow with structural properties. Using this model, a predator fish targeting isolated peripheral fish is simulated. Secondly, the escape behavior state machine model and escape maneuvers exhibited by fish schools are described. The escape maneuvers include compact, avoid, fast avoid, skitter, fountain, flash, ball, split, join, herd, vacuole, and hourglass are identified in the biological studies. This proposed escape behavior animation model can free an animator from dealing with the low-level animations but instead, control the fish behavior on a higher level by modifying a state machine and a small set of system parameters. With the state machine and relatively few system parameters, the proposed system is stable, predictable, and easy to tune, which represent important properties for animators to control the outcome. This system is developed in Unity (3D). In addition, a plug-in is also developed for full-fledged graphics tool Blender software to simulate escape maneuvers. The animator has to simply select escape maneuvers, adjust parameters and work on animating predator using keyframe method. It does not deal with the state machine model. The proposed model is useful not only in generating group behaviors but also in scientific visualization tool for studying fish behavior

Retrieval of Nano-Size Particles from a Solid Substrate

The retrieval of nano-size particles from suspects and crime scenes can significantly enhance surveillance and crime investigation. The Thesis deals with extensive study of free surface energy coupled with Hertzian contact energy under different conditions between a nano-sized particle and a substrate and the various methods of dislodging the particle. Using the Johnson-Kendall-Roberts (JKR) theory, a physical spring-mass-damper model is defined. Mathematical equations are formulated and solved with non-linear equations simulating the real world situation. Using Newmark’s β method, the system is solved for response of free and forced vibrations taking moisture to be responsible for damping. Key assumptions such as the particle and substrate are perfectly elastic and damping force is linear have been applied. Then a parameter study is conducted on two types of particles. The results have been summarized along with suggestions for future work

Pharmacovigilance programme of India from the perspective of nursing students

Background: Lack of knowledge and awareness of reporting adverse drug reactions is common among the nursing staff. As the nursing staffs are more close to the patients, this study was undertaken to evaluate their knowledge, attitude and perception about pharmacovigilance and ADR reporting. The objective of this study was to assess the knowledge, attitude and perception of nursing staff about pharmacovigilance programme and finding out the ways of improving the ADR reporting rate.Methods: Across-sectional, anonymous, questionnaire based study was conducted at the Government hospital, Nizamabad among the nursing staff. A predesigned, pretested and validated questionnaire consisting of 15 questions and 8 statements on knowledge, attitude and practice aspects of Pharmacovigilance programme in India. All the nursing staffs were explained about the purpose of the study and the questionnaire was distributed. Adequate time was given to fill them. Data spread on the excel sheet and the results were analysed using Microsoft office 2007 version.Results: Out of 65 students, 74.1% were aware of the term pharmacovigilance, 44.4% of the pharmacovigilance programme in India and 70.4% of the pharmacovigilance cell in their institute. 22.2%stated that known reactions, 14.8% unknown and 63% that all ADRs are to be reported.63% stated herbal drugs are safe, 44.4% that over the counter drugs are safe. Reporting system stated was by making call/by e-mail (25.9), written form (29.6%). Underreporting was due to lack of awareness (51.9%), lack of time (22.2%), feeling of creating negative impression on heath personnel (11.1%), or due to the feeling of waste of time (14.8%).Conclusions: Lack of facilities and clinical knowledge about ADR discourages them from reporting. Educational interventions and improvement of facilities were also suggested to enhance reporting rate in the hospital

Data Governance in Data Mesh Infrastructures: The Saxo Bank Case Study

Data governance (DG) is the management of data in a manner that the value of data is maximised and data related risks are minimised. Three aspects of DG are data catalogue, data quality, and data ownership and these aim to provide transparency, foster trust, and manage access and control the data. DG solution involves change management and alignment of incentives and mere technology is not enough to address this. In this paper we aim to provide a holistic view of data governance that is a synthesis of academic and practitioner viewpoints, and conclude by giving an example of a pilot case study (Saxo Bank) where authors worked on tech and cultural interventions to address the data governance challenges