The Case for Learned Index Structures

Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not. In this exploratory research paper, we start from this premise and posit that all existing index structures can be replaced with other types of models, including deep-learning models, which we term learned indexes. The key idea is that a model can learn the sort order or structure of lookup keys and use this signal to effectively predict the position or existence of records. We theoretically analyze under which conditions learned indexes outperform traditional index structures and describe the main challenges in designing learned index structures. Our initial results show, that by using neural nets we are able to outperform cache-optimized B-Trees by up to 70% in speed while saving an order-of-magnitude in memory over several real-world data sets. More importantly though, we believe that the idea of replacing core components of a data management system through learned models has far reaching implications for future systems designs and that this work just provides a glimpse of what might be possible

О параллельной обработке потока данных, адаптированной к области бит произвольной конфигурации

Предлагается модель операции свёртки арифметических многорядных двоичных кодов (МРК), которая учитывает неравномерность распределения бит данных по разрядам. На основе этой модели разрабатываются процедуры и методы свёртки МРК, которые позволяют снизить задержку на обработку.Пропонується модель операції згортки арифметичних багаторядних двійкових кодів (БРК), яка зважає на нерівномірність розподілу біт даних за разрядами. На основі цієї моделі розроблюються процедури і методи згортки БРК, які дозволяють зменшити затримку на обробку.Model of the compressing operation of arithmetic multi-row binary codes (MRC) is offered. In this model irregularity allocation of data bit per digits is considered. Procedures and methods of compressing MRC based on this model allow diminish delay of processing are designed (developed)

Insights into the regulation of DMSP synthesis in the diatom Thalassiosira pseudonana through APR activity, proteomics and gene expression analyses on cells acclimating to changes in salinity, light and nitrogen

Despite the importance of dimethylsulphoniopropionate (DMSP) in the global sulphur cycle and climate regulation, the biological pathways underpinning its synthesis in marine phytoplankton remain poorly understood. The intracellular concentration of DMSP increases with increased salinity, increased light intensity and nitrogen starvation in the diatom Thalassiosira pseudonana. We used these conditions to investigate DMSP synthesis at the cellular level via analysis of enzyme activity, gene expression and proteome comparison. The activity of the key sulphur assimilatory enzyme, adenosine 5′- phosphosulphate reductase was not coordinated with increasing intracellular DMSP concentration. Under all three treatments coordination in the expression of sulphur assimilation genes was limited to increases in sulphite reductase transcripts. Similarly, proteomic 2D gel analysis only revealed an increase in phosphoenolpyruvate carboxylase following increases in DMSP concentration. Our findings suggest that increased sulphur assimilation might not be required for increased DMSP synthesis, instead the availability of carbon and nitrogen substrates may be important in the regulation of this pathway. This contrasts with the regulation of sulphur metabolism in higher plants, which generally involves upregulation of several sulphur assimilatory enzymes. In T. pseudonana changes relating to sulphur metabolism were specific to the individual treatments and, given that little coordination was seen in transcript and protein responses across the three growth conditions, different patterns of regulation might be responsible for the increase in DMSP concentration seen under each treatment

Critical analysis of vendor lock-in and its impact on cloud computing migration: a business perspective

Vendor lock-in is a major barrier to the adoption of cloud computing, due to the lack of standardization. Current solutions and efforts tackling the vendor lock-in problem are predominantly technology-oriented. Limited studies exist to analyse and highlight the complexity of vendor lock-in problem in the cloud environment. Consequently, most customers are unaware of proprietary standards which inhibit interoperability and portability of applications when taking services from vendors. This paper provides a critical analysis of the vendor lock-in problem, from a business perspective. A survey based on qualitative and quantitative approaches conducted in this study has identified the main risk factors that give rise to lock-in situations. The analysis of our survey of 114 participants shows that, as computing resources migrate from on-premise to the cloud, the vendor lock-in problem is exacerbated. Furthermore, the findings exemplify the importance of interoperability, portability and standards in cloud computing. A number of strategies are proposed on how to avoid and mitigate lock-in risks when migrating to cloud computing. The strategies relate to contracts, selection of vendors that support standardised formats and protocols regarding standard data structures and APIs, developing awareness of commonalities and dependencies among cloud-based solutions. We strongly believe that the implementation of these strategies has a great potential to reduce the risks of vendor lock-in

Reference deployment models for eliminating user concerns on cloud security

Cloud computing has become a hot topic both in research and in industry, and when making decisions on deploying/adopting cloud computing related solutions, security has always been a major concern. This article summarizes security related issues in cloud computing and proposes five service deployment models to address these issues. The proposed models provide different security related features to address different requirements and scenarios and can serve as reference models for deployment

A Digital Repository and Execution Platform for Interactive Scholarly Publications in Neuroscience

The CARMEN Virtual Laboratory (VL) is a cloud-based platform which allows neuroscientists to store, share, develop, execute, reproduce and publicise their work. This paper describes new functionality in the CARMEN VL: an interactive publications repository. This new facility allows users to link data and software to publications. This enables other users to examine data and software associated with the publication and execute the associated software within the VL using the same data as the authors used in the publication. The cloud-based architecture and SaaS (Software as a Service) framework allows vast data sets to be uploaded and analysed using software services. Thus, this new interactive publications facility allows others to build on research results through reuse. This aligns with recent developments by funding agencies, institutions, and publishers with a move to open access research. Open access provides reproducibility and verification of research resources and results. Publications and their associated data and software will be assured of long-term preservation and curation in the repository. Further, analysing research data and the evaluations described in publications frequently requires a number of execution stages many of which are iterative. The VL provides a scientific workflow environment to combine software services into a processing tree. These workflows can also be associated with publications and executed by users. The VL also provides a secure environment where users can decide the access rights for each resource to ensure copyright and privacy restrictions are met

Optimal deployment of components of cloud-hosted application for guaranteeing multitenancy isolation

One of the challenges of deploying multitenant cloud-hosted services that are designed to use (or be integrated with) several components is how to implement the required degree of isolation between the components when there is a change in the workload. Achieving the highest degree of isolation implies deploying a component exclusively for one tenant; which leads to high resource consumption and running cost per component. A low degree of isolation allows sharing of resources which could possibly reduce cost, but with known limitations of performance and security interference. This paper presents a model-based algorithm together with four variants of a metaheuristic that can be used with it, to provide near-optimal solutions for deploying components of a cloud-hosted application in a way that guarantees multitenancy isolation. When the workload changes, the model based algorithm solves an open multiclass QN model to determine the average number of requests that can access the components and then uses a metaheuristic to provide near-optimal solutions for deploying the components. Performance evaluation showed that the obtained solutions had low variability and percent deviation when compared to the reference/optimal solution. We also provide recommendations and best practice guidelines for deploying components in a way that guarantees the required degree of isolation