About GLM-130B

GLM-130B is an open-source bilingual pre-trained model developed by THUDM. It is designed to support inference tasks with its 130 billion parameters on a single A100 (40G * 8) or V100 (32G * 8) server. The model has been trained on over 400 billion text tokens, half in English and half in Chinese.

Here are four key features of GLM-130B:

  1. Bilingual Support: GLM-130B supports both English and Chinese languages, making it versatile for various applications.
  2. Superior Performance: The model outperforms GPT-3 175B, OPT-175B, and BLOOM-176B on LAMBADA and MMLU for English. For Chinese, it performs significantly better than ERNIE TITAN 3.0 260B on 7 zero-shot CLUE datasets and 5 zero-shot FewCLUE datasets.
  3. Fast Inference: GLM-130B supports fast inference on both SAT and FasterTransformer, achieving up to 2.5X speedup with a single A100 server.
  4. Reproducibility and Cross-Platform Support: All results from over 30 tasks can be easily reproduced with the open-sourced code and model checkpoints. The model supports training and inference on NVIDIA, Hygon DCU, Ascend 910, and Sunway platforms.