About Gopher by DeepMind

DeepMind’s Gopher is a groundbreaking language model that pushes the boundaries of AI research. It’s a 280 billion parameter transformer language model that was developed as part of a broader portfolio of AI research. The model was trained on a series of transformer language models of different sizes, ranging from 44 million parameters to 280 billion parameters.

Here are four key features of Gopher

  1. Scale and Performance: Gopher is a large-scale language model with 280 billion parameters. It has shown superior performance in areas like reading comprehension, fact-checking, and the identification of toxic language.
  2. Massive Multitask Language Understanding (MMLU) Benchmark: Gopher has demonstrated significant advancements towards human expert performance over prior work on the MMLU benchmark.
  3. Dialogue Interaction: When prompted towards a dialogue interaction, Gopher can provide surprisingly coherent responses. It can discuss complex topics like cell biology and provide correct citations, despite no specific dialogue fine-tuning.
  4. Failure Mode Analysis: DeepMind has conducted thorough analysis of Gopher’s failure modes, which include tendencies for repetition, reflection of stereotypical biases, and the confident propagation of incorrect information. This analysis is crucial for understanding how large language models could lead to downstream harms and where mitigation efforts should focus.