基于约束的神经机器翻译

神经机器翻译是近几年出现并快速发展的一种深度学习驱动的新型机器翻译模式,目前已成为机器翻译学术和工业界广为接受的主流技术.本文总结了我们在神经机器翻译方面的工作,特别是在各种信息和知识约束条件下提出的一系列神经机器翻译模型和方法,具体包括隐变量约束的变分神经机器翻译模型、单词与短语级统计机器翻译译文推荐与约束模型、源端句法结构约束模型.除此之外,本文也对神经机器翻译未来发展进行了初步思考和展望.国家自然科学基金优秀青年基金(批准号:61622209)资助项

Neural Machine Translation with Deep Attention

该论文提出一种深层的注意机制，用于融合深层编码器和深层解码器之间的语义信息，从而进一步增强翻译系统建模源语言和目标语言之间翻译关系的能力。该论文提出的模型可以利用低层注意机制学习到的上下文信息，自动地判定如何从相应的编码层中提取、过滤源端语义信息并融入到相应的解码层之中，从而使高层注意机制拥有更充分的信息来建模深层次的翻译关系，并促使模型的隐层表示更适合目标词汇的预测。在中英、英德和英法三个翻译任务上，新模型取得了近乎最先进的翻译结果。该研究工作由我校软件学院苏劲松老师团队和天津大学熊德意老师团队合作完成。通讯作者为我校软件学院苏劲松副教授，第一作者为我校软件学院硕士生张飚。【Abstract】Deepening neural models has been proven very successful in improving the model's capacity when solving complex learning tasks, such as the machine translation task. Previous efforts on deep neural machine translation mainly focus on the encoder and the decoder, while little on the attention mechanism. However, the attention mechanism is of vital importance to induce the translation correspondence between different languages where shallow neural networks are relatively insufficient, especially when the encoder and decoder are deep. In this paper, we propose a deep attention model (DeepAtt). Based on the low-level attention information, DeepAtt is capable of automatically determining what should be passed or suppressed from the corresponding encoder layer so as to make the distributed representation appropriate for high-level attention and translation. We conduct experiments on NIST Chinese-English, WMT English-German and WMT English-French translation tasks, where, with 5 attention layers, DeepAtt yields very competitive performance against the state-of-the-art results. We empirically find that with an adequate increase of attention layers, DeepAtt tends to produce more accurate attention weights. An in-depth analysis on the translation of important context words further reveals that DeepAtt significantly improves the faithfulness of system translations.The authors were supported by National Natural Science Foundation of China (Nos. 61672440 and 61622209), the Fundamental Research Funds for the Central Universities (Grant No. ZK1024),and Scientific Research Project of National Language Committee of China (Grant No. YB135-49). Biao Zhang greatly acknowledges the support of the Baidu Scholarship. 该项研究得到了国家自然科学基金（Nos. 61672440, 61622209）、中央高校基础科研基金（No. ZK1024）、国家语委科研项目（No. YB13549）、百度奖学金等的资助

Current Statistical Machine Translation Research in China

2005年7月13日至15日,中国科学院自动化研究所、计算技术研究所和厦门大学计算机系联合举办了我国首届统计机器翻译研讨班。本文主要介绍本次研讨班参加单位的测试系统和实验结果,并给出相应的分析。测试结果表明,我国的统计机器翻译研究起步虽晚,但已有快速进展,参评系统在短期内得到了较好的翻译质量,与往年参加863评测的基于规则方法的系统相比性能虽还有差距,但差距已经不大。从目前国际统计机器翻译研究的现状和发展趋势来看,随着数据资源规模的不断扩大和计算机性能的迅速提高,统计机器翻译还有很大的发展空间。在未来几年内,在基于短语的主流统计翻译方法中融入句法、语义信息,必将成为机器翻译发展的趋势。Institute of Automation,Institute of Computing Technology of Chinese Academy of Sciences,and Department of Computer Science of Xiamen University held the first Statistical Machine Translation Workshop in China together,from July 13th to 15th in 2005.This paper describes the tested systems of involved institutions,and analyzes the results of their experiments.The test results show that although the research of statistical machine translation started late in China,it develops rapidly.The tested systems got quite good results in a short period.Compared with the rule-based systems reported in the formal "863"evaluation,the performance is somewhat lower;however,the difference is small.According to the state of art and the trend of international statistic machine translation research,we believe that there is still great space for the improvement of statistic machine translation,with larger-scale data resources and more powerful hardware.In near future,phrase-based method incorporated with syntax and semantic information will become the mainstream of statistical machine translation.国家自然科学基金资助项目(60272041