Information Losses in Neural Classifiers from Sampling

This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidden representation of that model. It then bounds this expected total variation as a function of the size of randomly sampled datasets in a fairly general setting, and without bringing in any additional dependence on model complexity. It ultimately obtains bounds on information losses that are less sensitive to input compression and in general much smaller than existing bounds. The paper then uses these bounds to explain some recent experimental findings of information compression in neural networks which cannot be explained by previous work. Finally, the paper shows that not only are these bounds much smaller than existing ones, but that they also correspond well with experiments.Comment: To be published in IEEE TNNL

Deep-sea coral distribution on seamounts, oceanic islands, and continental slopes in the Northeast Atlantic

A database of deep-water (\u3e 200 m) antipatharians, scleractinians, and gorgonians has been assembled for the NE Atlantic to determine what their distribution and diversity was before coral habitats became heavily impacted by bottom fishing gear. Benthic sampling expeditions from 1868–1985 have provided 2547 records showing the deepwater distribution of 22 species of antipatharians, 68 species of scleractinians, and 83 species of gorgonians with the majority of records found from seamounts, oceanic islands, and the continental slope of the warm temperate region. Too little is known about the coral biota of boreal and tropical seamounts to assess their levels of endemism, but on seamounts in the warm temperate region of the NE Atlantic the level endemism in antipatharian, scleractinian and gorgonian corals is low (\u3c 3%). Many of the species found on seamounts are characteristic of oceanic islands in this region and the oceanic islands have a significantly different coral fauna to that recorded at the same depths on the continental slope. Given the key role that corals can play in structuring deep-sea habitats it is hoped that our database will help inform the development of a network of marine protected areas to provide long-term protection for the differing communities found on continental slopes and isolated offshore habitats

An evaluation of the possibilities of using Second Life’s EduNation in Information Literacy training

This paper aims to identify the value of the virtual world Second Life, and the educational facilities available in EduNation within Second Life, specifically as regards their use for information literacy in Higher Education. The author identifies potential benefits and drawbacks of Second Life and EduNation, and concludes that they do have potential value

Information Losses in Neural Classifiers With Applications to Training Data Selection Strategies and Cyber Physical Systems

This dissertation considers the subject of information losses arising from finite datasets used in the training of neural classifiers. It proves a relationship between such losses and the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidden representation of that model. It then bounds this expected total variation as a function of the size of randomly sampled datasets in a fairly general setting, and without bringing in any additional dependence on model complexity. It ultimately obtains bounds on information losses that are less sensitive to input compression and much tighter than existing bounds. It then uses these bounds to explain some recent experimental findings of information compression in neural networks which cannot be explained by previous work. The dissertation goes on to provide analytical derivations for the relationship between neural architectures and the mutual information contained in their representations, which can be useful for guided architecture selection schemes. It then uses these developments to propose and illustrate a new framework for analyzing training data selection methods. The dissertation use this framework to prove that facility location methods reduce these losses, and then derive a new data dependent bound on them. This bound can be used to evaluate datasets and acts as an additional analytical tool for the study of data selection techniques. The dissertation then applies this theory to the problem of Phase Identification in power distribution systems. In particular, it focuses on improving supervised learning accuracies by exploiting some of the problem's information theoretic properties. This focus, along with the advances developed earlier in this work, helps us create two new Phase Identification techniques. The first transforms the bound on information losses into a data selection technique. This is important because phase identification data labels are difficult to obtain in practice. The second interprets the properties of distribution systems in the terms of the information losses developed earlier in the dissertation. This allows us to obtain an improvement in the representation learned by any classifier applied to the problem. Furthermore, since many problems in cyber-physical systems share similarities to the physical properties of phase identification exploited in this dissertation, the techniques can be applied to a wide range of similar problems

Artificial Intelligence, Due Process, and Criminal Sentencing

Article published in the Michigan State Law Review