Diverse configurations of columnar liquid crystals in cylindrical nano- and micropores.

Using 2D X-ray diffraction and AFM we studied the configuration, in cylindrical confinement, of hexagonal columnar phases that anchor homeotropically, i.e. with the columns normal to the pore wall. A wide range of pore diameters, from 20 nm to 100 μm, were explored by employing anodic alumina membranes and glass capillaries. The compounds used were a small discotic, hexakis(hexyloxy)triphenylene (HAT6), a large discotic hexa-peri-hexabenzocoronene (HBC), and a T-shaped bolaamphiphile, forming a honeycomb-type columnar phase. It was found that in pores up to tens of μm in diameter the columns adopt the "logpile" configuration with parallel columns crossing the pore perpendicular to its axis. Starting with 20 nm pores, with increasing pore diameter up to 5 different configurations are observed, the sequence being the same for all three compounds in spite of their structural diversity. One of the {100} planes of the hexagonal logpile starts from being parallel to the pore axis, then rotates by 90° as the pore size increases, and eventually becomes tilted to the pore axis by (8.5 ± 1)° as the pore widens further. Finally, in glass capillaries of tens of μm and beyond, the columns become axially oriented, parallel to the capillary axis. This latter finding was particularly unexpected as common sense would suggest axial columns to be favoured by planar anchoring, where in fact, it was shown to be hard to achieve. The present findings should help in the design of low-dimensional semiconductor or ionic conductor devices based on oriented columnar phases

Quantum Imitation Learning

Despite remarkable successes in solving various complex decision-making tasks, training an imitation learning (IL) algorithm with deep neural networks (DNNs) suffers from the high computation burden. In this work, we propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL. Concretely, we develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL). Q-BC is trained with a negative log-likelihood loss in an off-line manner that suits extensive expert data cases, whereas Q-GAIL works in an inverse reinforcement learning scheme, which is on-line and on-policy that is suitable for limited expert data cases. For both QIL algorithms, we adopt variational quantum circuits (VQCs) in place of DNNs for representing policies, which are modified with data re-uploading and scaling parameters to enhance the expressivity. We first encode classical data into quantum states as inputs, then perform VQCs, and finally measure quantum outputs to obtain control signals of agents. Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts, with the potential of quantum speed-up. To our knowledge, we are the first to propose the concept of QIL and conduct pilot studies, which paves the way for the quantum era.Comment: Manuscript submitted to a journal for review on January 5, 202

AI based Robot Safe Learning and Control

Introduction This open access book mainly focuses on the safe control of robot manipulators. The control schemes are mainly developed based on dynamic neural network, which is an important theoretical branch of deep reinforcement learning. In order to enhance the safety performance of robot systems, the control strategies include adaptive tracking control for robots with model uncertainties, compliance control in uncertain environments, obstacle avoidance in dynamic workspace. The idea for this book on solving safe control of robot arms was conceived during the industrial applications and the research discussion in the laboratory. Most of the materials in this book are derived from the authors’ papers published in journals, such as IEEE Transactions on Industrial Electronics, neurocomputing, etc. This book can be used as a reference book for researcher and designer of the robotic systems and AI based controllers, and can also be used as a reference book for senior undergraduate and graduate students in colleges and universities

Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets without interacting with the environment. However, the performance of existing offline RL algorithms heavily depends on the scale and state-action space coverage of datasets. Real-world data collection is often expensive and uncontrollable, leading to small and narrowly covered datasets and posing significant challenges for practical deployments of offline RL. In this paper, we provide a new insight that leveraging the fundamental symmetry of system dynamics can substantially enhance offline RL performance under small datasets. Specifically, we propose a Time-reversal symmetry (T-symmetry) enforced Dynamics Model (TDM), which establishes consistency between a pair of forward and reverse latent dynamics. TDM provides both well-behaved representations for small datasets and a new reliability measure for OOD samples based on compliance with the T-symmetry. These can be readily used to construct a new offline RL algorithm (TSRL) with less conservative policy constraints and a reliable latent space data augmentation procedure. Based on extensive experiments, we find TSRL achieves great performance on small benchmark datasets with as few as 1% of the original samples, which significantly outperforms the recent offline RL algorithms in terms of data efficiency and generalizability.Comment: The first two authors contributed equall

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Although Federated Learning (FL) enables global model training across clients without compromising their raw data, due to the unevenly distributed data among clients, existing Federated Averaging (FedAvg)-based methods suffer from the problem of low inference performance. Specifically, different data distributions among clients lead to various optimization directions of local models. Aggregating local models usually results in a low-generalized global model, which performs worse on most of the clients. To address the above issue, inspired by the observation from a geometric perspective that a well-generalized solution is located in a flat area rather than a sharp area, we propose a novel and heuristic FL paradigm named FedMR (Federated Model Recombination). The goal of FedMR is to guide the recombined models to be trained towards a flat area. Unlike conventional FedAvg-based methods, in FedMR, the cloud server recombines collected local models by shuffling each layer of them to generate multiple recombined models for local training on clients rather than an aggregated global model. Since the area of the flat area is larger than the sharp area, when local models are located in different areas, recombined models have a higher probability of locating in a flat area. When all recombined models are located in the same flat area, they are optimized towards the same direction. We theoretically analyze the convergence of model recombination. Experimental results show that, compared with state-of-the-art FL methods, FedMR can significantly improve the inference accuracy without exposing the privacy of each client.Comment: arXiv admin note: substantial text overlap with arXiv:2208.0767