A study test has revealed that the technology powering ChatGPT provides better advice on eye problems compared to non-specialist doctors.

Researchers suggest that while AI won't replace doctors, tools powered by artificial intelligence could potentially have a transformative impact on NHS waiting lists. This conclusion comes after a surprising set of findings from a study that evaluated doctors of various backgrounds.

A recent study conducted by the University of Cambridge revealed that the technology behind ChatGPT, known as GPT-4, outperformed non-specialist doctors in assessing eye problems and offering advice. The large language model (LLM) developed by OpenAI demonstrated nearly comparable performance to specialist eye doctors in a written multiple-choice test.

In the study, the AI model, renowned for its ability to generate text based on extensive training data, was pitted against doctors at various career levels. This included junior doctors lacking specialization, as well as trainees and seasoned expert eye doctors. Each group was presented with numerous patient scenarios involving distinct eye issues and tasked with providing a diagnosis or treatment advice by choosing from four options.

The examination was constructed around written inquiries drawn from a textbook utilized for evaluating aspiring eye doctors, encompassing a spectrum of ocular issues such as light sensitivity, diminished vision, lesions, and ocular irritation.

Since the textbook's contents are not publicly accessible, the researchers surmise that the large language model likely hasn't been trained on its material. In comparison to junior doctors, whose expertise level aligns with general practitioners, GPT-4 outperformed significantly on the assessment.

While achieving comparable scores to trainee and proficient eye doctors, GPT-4 fell short of surpassing the top-performing experts. The investigation, conducted last year utilizing the latest available large language models, also evaluated earlier versions of OpenAI's model (GPT-3.5), Google's PaLM2, and Meta's LLaMA using the same set of queries. GPT-4 exhibited superior accuracy compared to any other model tested.

The researchers emphasized that large language models are not poised to replace doctors but could potentially enhance the healthcare system. They suggest that these models might aid in reducing waiting lists by enabling doctors to attend to more patients within the same timeframe.

Source: Sky News