Researchers say they’ve developed a new AI foundation model trained on more than 40 million facial photographs that can help predict mortality risk in patients with cancer.
asserted
that → say → cancer
Recommended Videos
The technology is based on work from scientists at Mass General Brigham in Boston, which had earlier created a system called FaceAge that is designed to estimate a person’s biological age – indicative of physiological condition rather than chronological age – from a facial photograph.
asserted
that → base → photograph
The new system is called FAHR-FaceSurvival and is detailed in a study that was published Monday in the Lancet Digital Health.
asserted
that → call → Health
“This is the first time that we’ve shown that we can actually predict directly your mortality risk – not just your biological age, but your mortality risk – from a photograph alone,” said Raymond Mak, a radiation oncologist at Mass General Brigham Cancer Institute and an author of the study.
asserted
Mak → show → study
Findings showed that among cancer patients with FAHR-FaceSurvival scores in the highest 10 percent, about half died within 90 days, versus about 2 percent of those in the lowest 10 percent.
asserted
half → show → percent
Researchers used the biological-age estimates to help predict survival outcomes of people with cancer and found that patients whose biological age exceeded their chronological age tended to have poorer outcomes.
“The original FaceAge study that was reported on a year ago was a proof of principle that showed that we could train an algorithm to predict a biological age from a face photograph,” Mak said.
uncertain
Mak → use → photograph
FAHR-FaceAge was “more accurate than our original FaceAge” both at estimating age and as a prognostic marker in cancer patients, Mak said.
asserted
Mak → estimate → patients
The other system, FAHR-FaceSurvival, directly predicts mortality risk in patients with cancer.
asserted
system → predict → cancer
One of the most important findings, he said, was that researchers could use one face photograph and a foundation model to develop multiple algorithms to detect different health signals just from that simple image.
uncertain
researchers → say → image
Using the new FAHR-Face foundation model, Mak and his team were able to draw on existing datasets containing photos of more than 30,000 cancer patients, along with information about their medical outcomes – an approach Mak described as novel.
asserted
Mak → use → approach
“We could take those photographs and the known medical outcome – so their survival time, whether they died after their cancer diagnosis and when they died, or if they survived,” Mak said.
uncertain
Mak → take → diagnosis
The team used that information to train FAHR-FaceSurvival.
asserted
team → use → FaceSurvival
In the study, attention maps were generated from the trained models to understand which regions of the face the models focused on in making patient predictions.
asserted
models → generate → predictions
FAHR-FaceSurvival homed in on the area below the eyes, including the nasal bridge.
asserted
FaceSurvival → home → bridge
FAHR-FaceAge predominantly highlighted the nasolabial folds, forehead and temporalis area, underlining the importance of age-related changes in these areas.
asserted
FaceAge → highlight → areas
What’s not understood is exactly what these algorithms detected in those areas or how they formed conclusions.
asserted
they → understand → conclusions
“One of the fascinating unanswered questions is what exactly the algorithm is seeing,” Suchi Saria, a professor and director of the John Hopkins University Machine Learning and Healthcare Lab who was not involved in the study, said in an email.
asserted
who → see → email
“But we don’t yet know what physiological features in those regions are carrying the signal.
asserted
features → know → signal
Understanding that would help us determine whether the model is discovering a genuinely new biomarker or quantifying visible manifestations of illness that clinicians already recognize.”
asserted
clinicians → understand → that
When asked about possible theories as to what AI is picking up, Mak recalled Hippocrates’ writings about what he called “the face of death” in the Book of Prognostics, written in about 400 B.C.: “a sharp nose, hollow eyes, collapsed temples.”
asserted
he → ask → B.C.
As an oncologist, Mak said, he has seen patients nearing death develop a characteristic facial appearance, including hollowing around the temples, sunken eyes and loss of soft tissue around the nose.
asserted
patients → say → nose
As a clinician, he suspects AI might be “looking at that loss of soft tissue, loss of muscle mass, but that’s speculation.”
uncertain
that → suspect → mass
He clarified that the algorithms are “complete computer vision only” and are not reaching out across the internet for clues.
asserted
algorithms → clarify → clues
They have only seen a face photograph and either the age of that person or the date of death or survival time.
asserted
They → see → death
In other words, AI hasn’t had access to the observations of Hippocrates or anyone else.
asserted
AI → have → Hippocrates
Emma Pierson, an assistant professor of computer science and core faculty in the Computational Precision Health program at the University of California at Berkeley who was not involved in the study, described the findings as “scientifically interesting.”
asserted
who → involve → findings
But she questioned whether doctors are already implicitly assessing parts of a patient’s face – as Mak does as a clinician – even if not in a particularly systematic way.
asserted
Mak → question → way
Before the model is used in clinical practice, Pierson would want to see clear evidence that it improves on predictions in a way that otherwise could not be captured.
uncertain
that → use → way
“Showing that a photograph predicts survival is not the same as showing that it tells us something a clinician doesn’t already know,” Saria said, “and that in turn is not the same as showing that using the prediction improves patient care. …
asserted
using → show → care
The next experiment I’d want to see is: Take the best prediction you can make using the information already available to the clinician, then add the facial model and ask how much better the prediction gets.
asserted
prediction → want → model
Pierson also noted that the study accounted for factors such as age, sex, race or ethnicity, cancer site and aspects of treatment.
asserted
study → note → treatment
Based on her preliminary review, however, she did not see comprehensive measures of frailty – a patient’s level of physical vulnerability and ability to endure illness and treatment.
asserted
she → base → illness
Although formal frailty assessments are not included in the new study’s survival analyses, Mak described a separate clinical trial of older lung cancer patients receiving radiation treatment; researchers are comparing FaceAge estimates with assessments of patients’ strength and physical reserve.
asserted
researchers → include → strength
Pierson raised concerns about AI potentially generating racial bias.
asserted
AI → raise → bias
“Any time you train something on someone’s face, you are implicitly including patient race in the model’s predictions,” she said, and the use of race in medical predictions is “enormously controversial.”
asserted
use → train → predictions
For instance, if patients from a particular racial group have historically received poorer cancer treatment, AI trained on facial photographs could be “encoding racial bias into your model,” she said.
uncertain
she → receive → model
Researchers noted study limitations that include cancer-only cohorts from mainly high-income countries and called for broader validation and more representative patient populations.
asserted
that → note → validation
Prospective trials are needed to determine whether the algorithms provide benefits when used alongside existing prognostic models.
asserted
algorithms → need → models
In a separate study published earlier this year, Mak and colleagues analyzed images taken over a period of years to assess how biological age estimates changed over time and how those changes were associated with survival outcomes.
asserted
changes → publish → outcomes
Beyond cancer care, researchers also see broader possibilities using relatively limited clinical data.
asserted
researchers → see → data
…and 4 more, not listed.