About GPT-2 Output Detector
The GPT-2 Output Dataset is a resource developed by OpenAI for research in detection, biases, and more. It’s hosted on GitHub and provides a rich dataset for exploring the outputs of GPT-2 models.
- Extensive Dataset: The dataset contains 250K documents from the WebText test set. For each GPT-2 model trained on the WebText training set, it includes 250K random samples and 250K samples generated with Top-K 40 truncation.
- Model Variations: The dataset covers outputs from different GPT-2 models, including small (117M), medium (345M), large (762M), and extra-large (1542M) models. Each model’s output is available with and without Top-K 40 truncation.
- Finetuned Model Samples: The repository encourages research on detection of finetuned models. It has released data with samples from a GPT-2 full model finetuned to output Amazon reviews.
- Detectability Baselines: OpenAI is interested in research on the detectability of GPT-2 model family generations. They provide initial analysis of two baselines and code for the better baseline. They have achieved accuracies in the mid-90s for Top-K 40 generations, and mid-70s to high-80s for random generations.