The course introduces the foundations and the recent advancements in machine translation (MT), probably the most prolific application sector in computational linguistics. Machine translation deals with the automatic translation of speech or text between two languages. This technology is considered strategic for the integration of Europe as well as for the global market. Most internet companies (Google, Microsoft, Facebook, eBay, ...) own now MT research teams in order to support their geographic expansion. This course focuses on the statistical or machine learning approach which, after a long hegemony of phrase-based models, recently moved to neural network based models. Hence, the course will focus on neural MT, in particular modelling, training, and decoding. We will also look at use cases of MT in the translation industry, ranging from enterprise dedicated MT to the support of professional translators. The following is a tentative content table of the course: 1. Introduction to MT, problem, approaches, phrase-based versus neural MT, quality evaluation. 2. Fundamentals of neural networks, feed-forward networks, activation functions, loss functions, training criteria, stochastic gradient descent, learning rate policies, computation graph, back propagation algorithm. 3. Recurrent neural networks, time unfolded representation, back-propagation through time, vanishing and exploding gradient problems, long-short term memory units, gated recurrent units; 4. Neural MT, encoder-decoder architecture, attention model, beam search, model variations, large vocabulary methods, beam search, ensamble decoding. 5. MT Evaluation, manual and automatic MT evaluation, in depth comparison of neural vs phrase-based MT. 6. Training criteria for NMT, crosse-entropy , data as demonstrator, reinforcement learning, bandit learning, minimum risk training, curriculum learning.; 7. Convolutional NMT, convolutional networks, convolutional and, transformer models; 8. MT deployment, post-editing and online learning, interactive MT, topic and domain adaptation, automatic post-editing, multilingual NMT, the ModernMT project.
