Turing Natural Language Generation (T-NLG) is a groundbreaking language model developed by Microsoft. With a staggering 17 billion parameters, it stands out as one of the world’s largest and most powerful generative language models. T-NLG is designed to outperform existing models in a variety of natural language processing (NLP) tasks, including freeform generation, question answering, and summarization. The model is a part of Microsoft’s Project Turing, an initiative aimed at integrating deep learning advancements into Microsoft’s product suite.
Features of Turing-NLG
- Generative Capabilities: T-NLG is a Transformer-based generative language model. This means it can generate words to complete open-ended textual tasks. Whether it’s finishing an incomplete sentence, providing direct answers to questions, or summarizing input documents, T-NLG is equipped to handle it.
- Human-like Responses: The primary goal of T-NLG is to respond as directly, accurately, and fluently as humans in various situations. Earlier systems for tasks like question answering and summarization often relied on extracting content from documents, which sometimes resulted in unnatural or incoherent responses. T-NLG, on the other hand, can generate natural summaries or answers, enhancing user experience.
- Efficient Training: The larger the model and the more diverse its pretraining data, the better it generalizes to multiple tasks, even with fewer training examples. This efficiency means that a single large multi-task model like T-NLG can be more beneficial than training individual models for each task.
- Hardware and Software Breakthroughs: Training such a massive model requires significant computational resources. Microsoft leveraged NVIDIA DGX-2 hardware setups, tensor slicing, and the DeepSpeed library with ZeRO optimizer to efficiently train T-NLG.
- Direct Question Answering: T-NLG can provide direct answers to questions without relying on context passages. This “zero shot” question answering capability allows the model to generate answers based solely on its pretraining knowledge.
- Abstractive Summarization: T-NLG is designed to produce human-like abstractive summaries for a wide range of text documents. This capability is especially valuable given the lack of supervised training data for various document types.
- Integration with Microsoft Products: T-NLG’s advancements offer new opportunities for enhancing user experiences across the Microsoft Office suite. This includes providing writing assistance, answering document-related questions, and more.
- Fluent Chatbots and Digital Assistants: The model paves the way for more fluent chatbots and digital assistants, which can significantly improve customer relationship management and sales by engaging in natural conversations with users.
- Part of Project Turing: T-NLG is a component of Project Turing, an applied research group at Microsoft. This group focuses on evolving Microsoft products through the adoption of deep learning for both text and image processing.