South Korean researchers have developed a new method to make artificial intelligence models acknowledge when they do not know something, addressing a key limitation known as AI overconfidence. The breakthrough, from the Korea Advanced Institute of Science and Technology (KAIST), could improve reliability in fields such as autonomous driving and medicine.
Common AI models like OpenAI’s ChatGPT often “hallucinate” or fabricate facts because they are incentivised to guess rather than admit ignorance. The researchers found that overconfidence stems from how neural networks are initialised: random data input during this phase leads to high confidence despite no actual learning.
Mimicking the human brain, which generates signals before birth to handle uncertainty, the team developed a warm-up process. The AI’s neural network undergoes brief pre-training with random noise inputs before real learning, setting its initial confidence to a low level near chance. This helps the model first learn the state of “I don’t know anything yet”.
In tests, models with warm-up training showed a clear improvement in lowering confidence and recognising unfamiliar data, compared to conventional models that give incorrect answers with high confidence. The study, published in Nature Machine Intelligence, demonstrates that incorporating brain development principles can help AI distinguish what it knows from what it does not.
“This is important because it helps AI understand when it is uncertain or might be mistaken, not just improve how often it gives the right answer,” said study author Se-Bum Paik.
