End-to-End Audiovisual Fusion with LSTMs

Several end-to-end deep learning approaches have been recently presented which simultaneously extract visual features from the input images and perform visual speech classification. However, research on jointly extracting audio and visual features and performing classification is very limited. In this work, we present an end-to-end audiovisual model based on Bidirectional Long Short-Term Memory (BLSTM) networks. To the best of our knowledge, this is the first audiovisual fusion model which simultaneously learns to extract features directly from the pixels and spectrograms and perform classification of speech and nonlinguistic vocalisations. The model consists of multiple identical streams, one for each modality, which extract features directly from mouth regions and spectrograms. The temporal dynamics in each stream/modality are modeled by a BLSTM and the fusion of multiple streams/modalities takes place via another BLSTM. An absolute improvement of 1.9% in the mean F1 of 4 nonlingusitic vocalisations over audio-only classification is reported on the AVIC database. At the same time, the proposed end-to-end audiovisual fusion system improves the state-of-the-art performance on the AVIC database leading to a 9.7% absolute increase in the mean F1 measure. We also perform audiovisual speech recognition experiments on the OuluVS2 database using different views of the mouth, frontal to profile. The proposed audiovisual system significantly outperforms the audio-only model for all views when the acoustic noise is high.Comment: Accepted to AVSP 2017. arXiv admin note: substantial text overlap with arXiv:1709.00443 and text overlap with arXiv:1701.0584

A Search for Propylene Oxide and Glycine in Sagittarius B2 (LMH) and Orion

We have used the Mopra Telescope to search for glycine and the simple chiral molecule propylene oxide in the Sgr B2 (LMH) and Orion KL, in the 3-mm band. We have not detected either species, but have been able to put sensitive upper limits on the abundances of both molecules. The 3-sigma upper limits derived for glycine conformer I are 3.7 x 10^{14} cm^{-2} in both Orion-KL and Sgr B2 (LMH), comparable to the reported detections of conformer I by Kuan et al. However, as our values are 3-sigma upper limits rather than detections we conclude that this weighs against confirming the detection of Kuan et al. We find upper limits for the glycine II column density of 7.7 x 10^{12} cm^{-2} in both Orion-KL and Sgr B2 (LMH), in agreement with the results of Combes et al. The results presented here show that glycine conformer II is not present in the extended gas at the levels detected by Kuan et al. for conformer I. Our ATCA results (Jones et al.) have ruled out the detection of glycine (both conformers I and II) in the compact hot core of the LMH at the levels reported, so we conclude that it is unlikely that Kuan et al. have detected glycine in either Sgr B2 or Orion-KL. We find upper limits for propylene oxide abundance of 3.0 x 10^{14} cm^{-2} in Orion-KL and 6.7 x 10^{14} cm^{-2} in Sgr B2 (LMH). We have detected fourteen features in Sgr B2 and four features in Orion-KL which have not previously been reported in the ISM, but have not be able to plausibly assign these transitions to any carrier.Comment: 12 pages, 3 figures. Accepted by MNRAS 12th January 200

Formation of nanoscale structures by inductively coupled plasma etching

This paper will review the top down technique of ICP etching for the formation of nanometer scale structures. The increased difficulties of nanoscale etching will be described. However it will be shown and discussed that inductively coupled plasma (ICP) technology is well able to cope with the higher end of the nanoscale: features from 100nm down to about 40nm are relatively easy with current ICP technology. It is the ability of ICP to operate at low pressure yet with high plasma density and low (controllable) DC bias that helps greatly compared to simple reactive ion etching (RIE) and, though continual feature size reduction is increasingly challenging, improvements to ICP technology as well as improvements in masking are enabling sub-10nm features to be reached. Nanoscale ICP etching results will be illustrated in a range of materials and technologies. Techniques to facilitate etching (such as the use of cryogenic temperatures) and techniques to improve the mask performance will be described and illustrated