SimpleQA

lastmod 2024-11-06created 2024-11-061 edits

OpenAI's fact-checking benchmark

SimpleQA is a new benchmark for measuring the factuality of language models
Training models that generate factually grounded responses is still an unsolved problem in artificial intelligence
Current language models sometimes produce false outputs or responses not backed by evidence. These are called "hallucinations"
Language models that are more accurate and hallucinate less are more trustworthy and can be used in a wider range of applications
By open-sourcing SimpleQA, the aim is to measure the factuality of language models

SimpleQA benchmark features

Because factuality is a hard topic to measure, SimpleQA focuses on short, fact-seeking queries
Goals of SimpleQA:
- High correctness: reference answers are backed by sources from two independent AI trainers, and questions are written so that predicted answers are easy to grade
- Diversity: covers a wide range of topics, from science and technology to TV shows and video games
- Challenging for frontier models: unlike older benchmarks such as TriviaQA or NQ, SimpleQA was built to be a bigger challenge for frontier models (e.g., GPT-4 scores below 40%)
- Researcher-friendly UX: thanks to concise questions and answers, SimpleQA is fast and simple to run. Grading through the OpenAI API or other frontier model APIs is also efficient. With 4,326 questions, it is expected to have relatively low variance as an evaluation benchmark

AI trainers browsed the web and wrote short, fact-seeking questions with their answers
To be included in the dataset, each question had to meet strict criteria:
- It must have a single, indisputable answer that is easy to grade
- The answer to the question must not change over time
- Most questions had to induce hallucinations from GPT-4 or GPT-3.5
To further improve dataset quality, a second, independent AI trainer answered each question without seeing the original response
Only questions where the two AI trainers' answers agreed were included

For final verification, 1,000 questions were randomly sampled from the dataset and answered by a third AI trainer
The third AI trainer's answers matched the original agreed answers 94.4% of the time. 5.6% disagreed
Manual inspection of the disagreements found:
- 2.8% of the 5.6% were due to grader false negatives or human error by the third trainer (e.g., incomplete answers, misreading sources)
- The other 2.8% were due to real issues with the questions themselves (e.g., ambiguous questions, conflicting answers across websites)
Based on this, the inherent error rate of the dataset is estimated at about 3%

Questions are graded using a ChatGPT classifier that sees both the model's predicted answer and the ground-truth answer
The classifier grades a predicted answer as "correct", "incorrect", or "not attempted"
The table below shows the definition of each grade with corresponding examples
- "Correct": the predicted answer fully contains the ground-truth answer and does not contradict it
- "Incorrect": the predicted answer contradicts the ground-truth answer in any way (even with hedging)
- "Not attempted": the ground-truth target is not fully given in the answer, and there is no contradiction with the ground-truth answer
Ideally, a model answers as many questions as possible (the highest number of correct) while minimizing the number of incorrect answers

A factuality benchmark like SimpleQA can measure whether a model "knows what it knows"
This is called calibration, and it can be measured by directly asking the language model to state its confidence in its answer as a percentage
The correlation between the model's stated confidence and its actual accuracy can then be plotted
A perfectly calibrated model would have stated confidence equal to actual accuracy
The figure below shows these results:
- The positive correlation between stated confidence and accuracy is a good sign that models have some notion of confidence
- o1-preview is better calibrated than o1-mini, and gpt4 is better calibrated than gpt4-mini
- However, performance falling well below the y=x line means models consistently overstate their confidence
- So there is a lot of room to improve the calibration of large language models in terms of stated confidence

SimpleQA is a simple but challenging benchmark for evaluating the factuality of frontier models
The main limitation of SimpleQA is its scope. SimpleQA is accurate, but it measures factuality only in the constrained setting of short, fact-seeking queries with a single, verifiable answer
Whether the ability to give short factual answers correlates with the ability to write long responses full of many facts is still an open research question
The hope is that open-sourcing SimpleQA will drive more trustworthy and reliable AI research, and that researchers will evaluate the factuality of language models with SimpleQA and give feedback

SimpleQA is an interesting and needed benchmark that measures language model factuality with short fact-based questions. Ultimately, making AI more trustworthy requires improving its ability to generate factually grounded responses
However, SimpleQA measures factuality only in a constrained setting, so it cannot fully reflect language model factuality in real-world use scenarios. Further research seems needed to evaluate factuality in more varied settings
Also, since the SimpleQA dataset itself is about 97% accurate, it seems hard for language model performance to go beyond that. The quality of the dataset itself should also keep improving
Other benchmarks with similar goals to SimpleQA include TruthfulQA and HonestQA. Comparing against them would make the strengths and weaknesses of SimpleQA clearer
To improve language model factuality, pre-training on large amounts of high-quality data matters, but the ability to use external knowledge or self-correct at inference time also seems necessary. Hopefully related research will be active

author Zer0 sitemap