A Survey on Causal Reinforcement Learning

While Reinforcement Learning (RL) achieves tremendous success in sequential decision-making problems of many domains, it still faces key challenges of data inefficiency and the lack of interpretability. Interestingly, many researchers have leveraged insights from the causality literature recently, bringing forth flourishing works to unify the merits of causality and address well the challenges from RL. As such, it is of great necessity and significance to collate these Causal Reinforcement Learning (CRL) works, offer a review of CRL methods, and investigate the potential functionality from causality toward RL. In particular, we divide existing CRL approaches into two categories according to whether their causality-based information is given in advance or not. We further analyze each category in terms of the formalization of different models, ranging from the Markov Decision Process (MDP), Partially Observed Markov Decision Process (POMDP), Multi-Arm Bandits (MAB), and Dynamic Treatment Regime (DTR). Moreover, we summarize the evaluation matrices and open sources while we discuss emerging applications, along with promising prospects for the future development of CRL.Comment: 29 pages, 20 figure

Low-cost virtual instrumentation system of an energy-dispersive X-ray spectrometer for a scanning electron microscope

The paper describes an energy-dispersive X-ray spectrometer for a scanning electron microscope (SEM-EDXS). It was constructed using the new architecture of a virtual instrument (VI), which is low-cost, space-saving, fast and flexible way to develop the instrument. Computer-aided teaching (CAT) was used to develop the instrument and operation rather than a traditional instrument technique. The VI was designed using the object-oriented program language C++ and compact programmable logical devices (CPLD). These include spectra collection and processing, quantitative analysis and X-ray-intensity distribution analysis. The procedure is described in detail. The VI system gives an e¡ective and user-friendly human interface for the whole analytical task. Some examples are described

Auto-analysis system for graphite morphology of grey cast iron

The current method to classify graphite morphology types of grey cast iron is based on traditional subjective observation, and it cannot be used for quantitative analysis. Since microstructures have a great effect on the mechanical properties of grey cast iron and different types have totally different characters, six types of grey cast iron are discussed and an image-processing software subsystem that performs the classification and quantitative analysis automatically based on a kind of composed feature vector and artificial neural network (ANN) is described. There are three kinds of texture features: fractal dimension, roughness and two-dimension autoregression, which are used as an extracted feature input vector of ANN classifier. Compared with using only one, the checkout correct precision increased greatly. On the other hand, to achieve the quantitative analysis and show the different types clearly, the region segmentation idea was applied to the system. The percentages of the regions with different type are reported correctly. Furthermore, this paper tentatively introduces a new empirical method to decide the number of ANN hidden nodes, which are usually considered as a difficulty in ANN structure decision. It was found that the optimum hidden node number of the experimental data was the same as that obtained using the new method

Adaptive Policy with Wait-kk Model for Simultaneous Translation

Simultaneous machine translation (SiMT) requires a robust read/write policy in conjunction with a high-quality translation model. Traditional methods rely on either a fixed wait-$k$ policy coupled with a standalone wait-$k$ translation model, or an adaptive policy jointly trained with the translation model. In this study, we propose a more flexible approach by decoupling the adaptive policy model from the translation model. Our motivation stems from the observation that a standalone multi-path wait-$k$ model performs competitively with adaptive policies utilized in state-of-the-art SiMT approaches. Specifically, we introduce DaP, a divergence-based adaptive policy, that makes read/write decisions for any translation model based on the potential divergence in translation distributions resulting from future information. DaP extends a frozen wait-$k$ model with lightweight parameters, and is both memory and computation efficient. Experimental results across various benchmarks demonstrate that our approach offers an improved trade-off between translation accuracy and latency, outperforming strong baselines.Comment: Accept to EMNLP 2023 main conference. 17 pages, 12 figures, 5 table

N-doped porous carbon nanofibers embedded with TiN nanoparticles for high-performance Li–S batteries

Flexible N-doped porous carbon nanofibers embedded with TiN nanoparticles are prepared for cathode materials of lithium–sulfur (Li–S) batteries. A synergistic adsorption effect of lithium polysulfides (LiPSs) can be provided by the well-dispersed TiN nanoparticles, the heteroatomic doping, and the porous nanofibers structure. As a result, the as-prepared electrode can deliver a high reversible capacity of 990 mAh g−1 after 120 cycles at 0.2 A g−1 with coulombic efficiency approaching 100%. This work provides a simple and comprehensive strategy for the preparation of high-performance Li–S batteries.</p

Transport‐Friendly Microstructure in SSC‐MEA: Unveiling the SSC Ionomer‐Based Membrane Electrode Assemblies for Enhanced Fuel Cell Performance

The significant role of the cathodic binder in modulating mass transport within the catalyst layer (CL) of fuel cells is essential for optimizing cell performance. This investigation focuses on enhancing the membrane electrode assembly (MEA) through the utilization of a short-side-chain perfluoro-sulfonic acid (SSC-PFSA) ionomer as the cathode binder, referred to as SSC-MEA. This study meticulously visualizes the distinctive interpenetrating networks of ionomers and catalysts, and explicitly clarifies the triple-phase interface, unveiling the transport-friendly microstructure and transport mechanisms inherent in SSC-MEA. The SSC-MEA exhibits advantageous microstructural features, including a better-connected ionomer network and well-organized hierarchical porous structure, culminating in superior mass transfer properties. Relative to the MEA bonded by long-side-chain perfluoro-sulfonic acid (LSC-PFSA) ionomer, noted as LSC-MEA, SSC-MEA exhibits a notable peak power density (1.23&nbsp;W&nbsp;cm-2), efficient O2 transport, and remarkable proton conductivity (65% improvement) at 65&nbsp;°C and 70% relativity humidity (RH). These findings establish crucial insights into the intricate morphology-transport-performance relationship in the CL, thereby providing strategic guidance for developing highly efficient MEA

Targeted Sub-Attomole Cancer Biomarker Detection Based on Phase Singularity 2D Nanomaterial-Enhanced Plasmonic Biosensor

Detection of small cancer biomarkers with low molecular weight and a low concentration range has always been challenging yet urgent in many clinical applications such as diagnosing early-stage cancer, monitoring treatment and detecting relapse. Here, a highly enhanced plasmonic biosensor that can overcome this challenge is developed using atomically thin two-dimensional phase change nanomaterial. By precisely engineering the configuration with atomically thin materials, the phase singularity has been successfully achieved with a significantly enhanced lateral position shift effect. Based on our knowledge, it is the first experimental demonstration of a lateral position signal change > 340 μm at a sensing interface from all optical techniques. With this enhanced plasmonic effect, the detection limit has been experimentally demonstrated to be 10–15 mol L−1 for TNF-α cancer marker, which has been found in various human diseases including inflammatory diseases and different kinds of cancer. The as-reported novel integration of atomically thin Ge2Sb2Te5 with plasmonic substrate, which results in a phase singularity and thus a giant lateral position shift, enables the detection of cancer markers with low molecular weight at femtomolar level. These results will definitely hold promising potential in biomedical application and clinical diagnostics

Altering the Symmetry of Fe–N–C by Axial Cl‐Mediation for High‐Performance Zinc–Air Batteries

peer reviewedAbstractFe–N–C catalyst is acknowledged as a promising alternative for the state‐of‐the‐art Pt/C in oxygen reduction reaction (ORR) toward cutting‐edge electrochemical energy conversion/storage applications. Herein, a “Cl‐mediation” strategy is proposed on Fe–N–C for modulating the catalyst's electronic structure toward achieving remarkable ORR activity. By coordinating axial Cl atoms to iron phthalocyanine (FePc) molecules on carbon nanotubes (CNTs) matrix, a Cl‐modulated Fe–N–C (FePc‐Cl‐CNTs) catalyst is synthesized. The as‐prepared FePc‐Cl‐CNTs exhibit an improved ORR activity with a half‐wave potential of 0.91 V versus RHE in alkaline solution, significantly outperforming the parent FePc‐CNTs (0.88 V versus RHE). The advanced nature of the as‐prepared FePc‐Cl‐CNTs is evidenced by a configured high‐performance rechargeable Zn–air battery, which operates stably for over 150 h. The experiments and density functional theory calculations unveil that axial Cl atoms induce the transformation of FePc from its original D4h to C4v symmetry, effectively altering the electrons distribution around the Fe‐center, by which it optimizes *OH desorption and subsequently boosts the reaction kinetics. This work paves ways for resolving the dilemma of Fe–N–C catalysts’ exploration via engineering Fe–N–C configuration

Foundation models and intelligent decision-making: Progress, challenges, and perspectives

Intelligent Decision-Making (IDM) is a cornerstone of artificial intelligence (AI), designed to automate or augment decision processes. Modern IDM paradigms integrate advanced frameworks to enable intelligent agents to make effective and adaptive choices and decompose complex tasks into manageable steps, such as AI agents and high-level reinforcement learning. Recent advances in multimodal foundation-based approaches unify diverse input modalities—such as vision, language, and sensory data—into a cohesive decision-making process. Foundation Models (FMs) have become pivotal in science and industry, trans- forming decision-making and research capabilities. Their large-scale, multimodal data-processing abilities foster adaptability and interdisciplinary breakthroughs across fields such as healthcare, life sciences, and education. This survey examines IDM’s evolution, advanced paradigms with FMs, and their transformative impact on decision-making across diverse scientific and industrial domains, highlighting the challenges and opportunities in building efficient, adaptive, and ethical decision systems.<br/