Speeding up Context-based Sentence Representation Learning with Non-autoregressive Convolutional Decoding

Context plays an important role in human language understanding, thus it may also be useful for machines learning vector representations of language. In this paper, we explore an asymmetric encoder-decoder structure for unsupervised context-based sentence representation learning. We carefully designed experiments to show that neither an autoregressive decoder nor an RNN decoder is required. After that, we designed a model which still keeps an RNN as the encoder, while using a non-autoregressive convolutional decoder. We further combine a suite of effective designs to significantly improve model efficiency while also achieving better performance. Our model is trained on two different large unlabelled corpora, and in both cases the transferability is evaluated on a set of downstream NLP tasks. We empirically show that our model is simple and fast while producing rich sentence representations that excel in downstream tasks

Rethinking Skip-thought: A Neighborhood based Approach

We study the skip-thought model with neighborhood information as weak supervision. More specifically, we propose a skip-thought neighbor model to consider the adjacent sentences as a neighborhood. We train our skip-thought neighbor model on a large corpus with continuous sentences, and then evaluate the trained model on 7 tasks, which include semantic relatedness, paraphrase detection, and classification benchmarks. Both quantitative comparison and qualitative investigation are conducted. We empirically show that, our skip-thought neighbor model performs as well as the skip-thought model on evaluation tasks. In addition, we found that, incorporating an autoencoder path in our model didn't aid our model to perform better, while it hurts the performance of the skip-thought model

Pericardial effusion after open heart surgery for congenital heart disease

Objectives: To determine the prevalence and time course of pericardial effusion after open heart surgery for congenital heart diseases and to identify predisposing risk factors. Design and patients: Prospective assessment of development of pericardial effusion in 336 patients (163 males) undergoing open heart surgery for congenital heart disease by serial echocardiography on days 5, 7, 14, 21, and 28 postoperatively. Setting: Tertiary paediatric cardiac centre. Results: The prevalence of pericardial effusion was 23% (77 of 336). Of the 77 patients who developed effusion, 43 (56%) had moderate to large effusions and 18 (23%) were symptomatic. Patients who had a large amount of effusion were more likely to be symptomatic than those with only a small to moderate amount (47.4% v 15.5%, p = 0.01). The mean (SD) onset of pericardial effusion was 11 (7) days after surgery, with 97% (75 of 77) of cases being diagnosed on or before day 28 after surgery. The prevalence of effusion after Fontan-type procedures (60%, 6 of 10) was significantly higher than that after other types of cardiac surgery: repair of left to right shunts (22.1%, 43 of 125), repair of lesions with right ventricular outflow tract obstruction (22.6%, 19 of 84), arterial switch operation (6.7%, 1 of 15), and miscellaneous procedures (25%, 8 of 32) (p = 0.037). Univariate analyses showed that female patients (p = 0.009) and those receiving warfarin (p = 0.002) had increased risk of postoperative pericardial effusion. A greater pericardial drain output in the first four hours after surgery also tended to be significant (p = 0.056). Multivariate logistic regression similarly identified warfarin treatment (β = 1.73, p = 0.009) and female sex (β for male = -0.63, p = 0.037) as significant determinants. Conclusions: Pericardial effusion occurs commonly after open heart surgery for congenital heart disease. Serial echocardiographic monitoring up to 28 days postoperatively is indicated in selected high risk patients such as those with symptoms of postpericardiotomy syndrome and those given warfarin.published_or_final_versio

Efficacy of High Dose Vitamin D Supplements for Elite Athletes.

PURPOSE: Supplementation with dietary forms of vitamin D is commonplace in clinical medicine, elite athletic cohorts and the general population, yet the response of all major vitamin D metabolites to high doses of vitamin D is poorly characterized. We aimed to identify the responses of all major vitamin D metabolites to moderate and high dose supplemental vitamin D3. METHODS: A repeated measures design was implemented in which 46 elite professional European athletes were block randomized based on their basal 25[OH]D concentration into two treatment groups. Athletes received either 35,000 or 70,000 IU.week vitamin D3 for 12 weeks and 42 athletes completed the trial. Blood samples were collected over 18 weeks to monitor the response to supplementation and withdrawal from supplementation. RESULTS: Both doses led to significant increases in serum 25[OH]D and 1,25[OH]2D3. 70,000 IU.week also resulted in a significant increase of the metabolite 24,25[OH]2D at weeks 6 and 12 that persisted following supplementation withdrawal at week 18, despite a marked decrease in 1,25[OH]2D3. Intact PTH was decreased in both groups by week 6 and remained suppressed throughout the trial. CONCLUSIONS: High dose vitamin D3 supplementation (70,000 IU.week) may be detrimental for its intended purposes due to increased 24,25[OH]2D production. Rapid withdrawal from high dose supplementation may inhibit the bioactivity of 1,25[OH]2D3 as a consequence of sustained increases in 24,25[OH]2D that persist as 25[OH]D and 1,25[OH]2D concentrations decrease. These data imply that lower doses of vitamin D3 ingested frequently may be most appropriate and gradual withdrawal from supplementation as opposed to rapid withdrawal may be favorable