Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)

Deep neural networks (DNN) have shown remarkable success in a variety of machine learning applications. The capacity of these models (i.e., number of parameters), endows them with expressive power and allows them to reach the desired performance. In recent years, there is an increasing interest in deploying DNNs to resource-constrained devices (i.e., mobile devices) with limited energy, memory, and computational budget. To address this problem, we propose Entropy-Constrained Trained Ternarization (EC2T), a general framework to create sparse and ternary neural networks which are efficient in terms of storage (e.g., at most two binary-masks and two full-precision values are required to save a weight matrix) and computation (e.g., MAC operations are reduced to a few accumulations plus two multiplications). This approach consists of two steps. First, a super-network is created by scaling the dimensions of a pre-trained model (i.e., its width and depth). Subsequently, this super-network is simultaneously pruned (using an entropy constraint) and quantized (that is, ternary values are assigned layer-wise) in a training process, resulting in a sparse and ternary network representation. We validate the proposed approach in CIFAR-10, CIFAR-100, and ImageNet datasets, showing its effectiveness in image classification tasks.Comment: Proceedings of the CVPR'20 Joint Workshop on Efficient Deep Learning in Computer Vision. Code is available at https://github.com/d-becking/efficientCNN

Модели общественных явлений и сценарные подходы в принятии решений

Предложен новый класс моделей с ассоциативной памятью для изучения явлений в больших социальных системах. Модели имеют структуру, сходную со структурой нейросетевых моделей хопфилдовского класса. Учет в предложенной концепции интеллектуальных свойств субъектов общественных процессов позволил значительно расширить круг явлений, моделирование которых становится возможным. В частности, учет способности субъектов строить прогнозы ситуаций и принимать решения на базе этих прогнозов приводит к совершенно новым свойствам решений, главное из которых — возникновение многозначных решений, что на уровне большой общественной системы приводит к появлению множества способов поведения такой системы, т.е. сценариев развития событий. Обсуждаются также некоторые аналогии с поведением квантово-механических систем.Запропоновано новий клас моделей з властивостями асоціативної пам’яті для вивчення явищ у великих соціальних системах. Моделі мають структуру, подібну до структури нейронних мереж хопфілдівського класу. Врахування в запропонованій концепції інтелектуальних властивостей, притаманних суб’єктам суспільного процесу, дозволило значно розширити коло явищ, моделювання яких стає можливим. Зокрема, спроможність суб’єктів будувати прогнози ситуацій та приймати рішення, засновані на цих прогнозах, призводять до цілком нових властивостей розв’язків, основним з яких є можливість появи багатозначних розв’язків. На рівні великої суспільної системи це призводить до появи великої кількості способів поведінки такої системи, тобто сценаріїв розвитку подій. Обговорюються також деякі аналогії з поведінкою квантово- механічних систем.A new class of models is proposed with the properties of associative memory to be used in investigation of large social systems. The models have the structure similar to neuronets models of the Hopfield type. Accounting the intellectual properties allowed extending the scope of modelable phenomena. In particular, the individual ability for forecasting and decision-making results in new properties of solutions. The main new property is possible appearance of multivalued solutions. For large social systems, this leads to the existence of many trajectories of system, that is, many scenarios of behavior. Some analogies with quantum-mechanical phenomena are discussed

Review Article: Anthropology and the Biblical Exile

Over the past three decades there have been a few attempts to use social anthropology to explore the Old Testament (OT) and interdisciplinary studies of this sort are now becoming more mainstream. Historical and archaeological data concerning ancient societies are necessarily limited in scope, and anthropological comparisons may offer insights into historical peoples who cannot be observed. This article discusses recent trends in post-exilic studies, in particular those appealing to anthropology

Change Agents in Global Initiatives

In December 2012, the President of Fort Hays State University attended a meeting with faculty and staff from the Department of Advanced Education Programs. During the meeting, he asked an important question,” What are you doing for Kansans?” The question became a challenge for a few faculty members that helped guide the direction of an initiative to provide professional development support for teachers on technology integration in the 21st Century classroom. The President’s question provided the impetus to keep going and to develop a more succinct plan to provide support for schools; especially as schools developed Google Domains and began to use the free applications offered by Google and others. As a result, 19 faculty and staff worked together to become Google Certified Educators. The process, itself, helped form a team of six individuals who went on to complete Google for Education Trainer certification. The gTrainers quickly developed a vision for what could be accomplished based on the belief that technology integration is more than an “add-on.” It is a mindset where the tools become the means for creating challenging, engaging projects and an environment for student learning

Bridging the gap between omics and earth system science to better understand how environmental change impacts marine microbes

The advent of genomic-, transcriptomic- and proteomic-based approaches has revolutionized our ability to describe marine microbial communities, including biogeography, metabolic potential and diversity, mechanisms of adaptation, and phylogeny and evolutionary history. New interdisciplinary approaches are needed to move from this descriptive level to improved quantitative, process-level understanding of the roles of marine microbes in biogeochemical cycles and of the impact of environmental change on the marine microbial ecosystem. Linking studies at levels from the genome to the organism, to ecological strategies and organism and ecosystem response, requires new modelling approaches. Key to this will be a fundamental shift in modelling scale that represents micro-organisms from the level of their macromolecular components. This will enable contact with omics data sets and allow acclimation and adaptive response at the phenotype level (i.e. traits) to be simulated as a combination of fitness maximization and evolutionary constraints. This way forward will build on ecological approaches that identify key organism traits and systems biology approaches that integrate traditional physiological measurements with new insights from omics. It will rely on developing an improved understanding of ecophysiology to understand quantitatively environmental controls on microbial growth strategies. It will also incorporate results from experimental evolution studies in the representation of adaptation. The resulting ecosystem-level models can then evaluate our level of understanding of controls on ecosystem structure and function, highlight major gaps in understanding and help prioritize areas for future research programs. Ultimately, this grand synthesis should improve predictive capability of the ecosystem response to multiple environmental drivers

Implications of the Digital Divide for Technology Integration in Schools: A White Paper

The implications of the Digital Divide in 2019 vary slightly from the onset of Internet use. Max Weber’s (1922, 1978) theory of stratification provides a foundation for understanding why the growth is slow. Defining the Digital Divide is difficult. In reality, it has moved from a situation of access to acquiring sufficient skills to use it effectively. We examines the issue from the perspective of the need for technology literate leaders in schools. Technology literate leaders not only embrace technology: they model it. The leaders provide teachers with access and training to implement the best pedagogically sound teaching strategies so that students have optimum opportunities to learn with technology

Not poles apart: Antarctic soil fungal communities show similarities to those of the distant Arctic

Antarctica's extreme environment and geographical isolation offers a useful platform for testing the relative roles of environmental selection and dispersal barriers influencing fungal communities. The former process should lead to convergence in community composition with other cold environments, such as those in the Arctic. Alternatively, dispersal limitations should minimise similarity between Antarctica and distant northern landmasses. Using high-throughput sequencing, we show that Antarctica shares significantly more fungi with the Arctic, and more fungi display a bipolar distribution, than would be expected in the absence of environmental filtering. In contrast to temperate and tropical regions, there is relatively little endemism, and a strongly bimodal distribution of range sizes. Increasing southerly latitude is associated with lower endemism and communities increasingly dominated by fungi with widespread ranges. These results suggest that micro-organisms with well-developed dispersal capabilities can inhabit opposite poles of the Earth, and dominate extreme environments over specialised local specie