Novel AI Framework Unites Generalist and Specialist Models for Precision Healthcare

For years, medical AI has faced a dilemma: “generalist” models are flexible but lack pinpoint diagnostic precision, while “specialist” models are highly accurate but too narrow and costly to scale. 

Now, a breakthrough from HKUST has elegantly solved this puzzle, paving the way for AI to enter everyday clinical workflows seamlessly and safely. The vision of a hospital where AI collaborates like a seasoned medical board is now one step closer to reality.

Spearheading this initiative is Prof. CHEN Hao, Assistant Professor in the Department of Computer Science and Engineering and the Department of Chemical and Biological Engineering. He is the Director of the Collaborative Center for Medical and Engineering Innovation at HKUST. 

Leading an international consortium that includes Harvard Medical School, Weill Cornell Medicine, and top local universities, Prof. Chen’s team recently published their groundbreaking “Generalist–Specialist Collaboration” (GSCo) framework in the prestigious journal Nature Biomedical Engineering. 

This achievement cements HKUST’s role as a global vanguard in translational medical AI. At the heart of this breakthrough is MedDr, an open-source generalist AI model developed at HKUST and trained on over two million diverse medical samples. But the strength lies in the GSCo framework. Instead of forcing one massive AI to know everything, GSCo pairs MedDr with a suite of lightweight “specialist” models. 

Think of MedDr as the chief physician: during diagnosis, the lightweight specialists—trainable on a single consumer-grade graphics card—analyze the image, retrieve similar historical cases, and offer expert predictions. MedDr then synthesizes this advice with its own vast medical knowledge to make the final, arbitrated call.

“The core value of GSCo lies in facilitating their synergistic collaboration, where the generalist model acts as the decision-maker, integrating expert advice with its own broader medical knowledge,” Professor Chen explained. 

The results are transformative. Evaluated across 32 public datasets comprising 260,000 medical images, GSCo secured the top overall ranking, outperforming ten state-of-the-art models. In stringent skin lesion tests on previously unseen data, GSCo achieved a remarkable diagnostic score of 0.8420, surpassing both standalone models. 

In human evaluations, six out of seven board-certified radiologists preferred GSCo’s radiology reports over those of existing specialist models. Crucially, stress tests proved MedDr’s robustness: even when fed systematically biased data suggesting all images were normal, it still correctly identified 67.6% of tumors.

This innovation perfectly embodies HKUST’s mission to drive interdisciplinary research that delivers sustainable, real-world societal impact. By keeping specialist training local, GSCo ensures sensitive patient data never leaves the hospital, solving major privacy and regulatory hurdles. 

Moreover, it slashes the development costs of adapting AI to new clinical tasks by up to 100-fold. As Prof. Chen notes, this offers a practical pathway for healthcare settings with limited computational resources to benefit from AI innovation.

“We hope this novel framework offers a more practical and sustainable pathway for integrating medical foundation models into everyday clinical workflows, particularly enabling healthcare settings with limited AI computational resources to share in the benefits of AI-powered medical innovation,” Prof. Chen said.

With the MedDr and GSCo frameworks now publicly open-source, and future expansions into 3D imaging (like CT and MRI) and medical video on the horizon, HKUST continues to push the boundaries of what is possible—making advanced, equitable, and intelligent healthcare truly boundless.