The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding

TL;DR

This document provides comprehensive submission and formatting guidelines for ACL 2020, standardizing anonymous two-column manuscripts, templates, and references to ensure consistent, accessible, and reproducible reviews. It covers everything from anonymous submission and page limits to font choices, figure placement, citations, DOIs, and appendices, including LaTeX-specific instructions and troubleshooting. By detailing templates, submission rules, and accessibility considerations, the guidelines facilitate uniform presentation and efficient evaluation across submissions. The practical impact is to streamline authors' preparation process and improve the quality and comparability of accepted papers in the ACL venue ecosystem.

Abstract

We present MT-DNN, an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models. Built upon PyTorch and Transformers, MT-DNN is designed to facilitate rapid customization for a broad spectrum of NLU tasks, using a variety of objectives (classification, regression, structured prediction) and text encoders (e.g., RNNs, BERT, RoBERTa, UniLM). A unique feature of MT-DNN is its built-in support for robust and transferable learning using the adversarial multi-task learning paradigm. To enable efficient production deployment, MT-DNN supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. We demonstrate the effectiveness of MT-DNN on a wide range of NLU applications across general and biomedical domains. The software and pre-trained models will be publicly available at https://github.com/namisan/mt-dnn.