Sending a “selfie” to the doctor could be a cheap and simple way of detecting heart disease, according to the authors of a new study published today in the European Heart Journal.
The study is the first to show that it’s possible to use a deep learning computer algorithm to detect coronary artery disease (CAD) by analysing four photographs of a person’s face.
Although the algorithm needs to be developed further and tested in larger groups of people from different ethnic backgrounds, the researchers say it has the potential to be used as a screening tool that could identify possible heart disease in people in the general population or in high-risk groups, who could be referred for further clinical investigations.
“To our knowledge, this is the first work demonstrating that artificial intelligence can be used to analyse faces to detect heart disease.
It is a step towards the development of a deep learning-based tool that could be used to assess the risk of heart disease, either in outpatient clinics or by means of patients taking ‘selfies’ to perform their own screening.
This could guide further diagnostic testing or a clinical visit,” said Professor Zhe Zheng, who led the research and is vice director of the National Center for Cardiovascular Diseases and vice president of Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People’s Republic of China.
He continued: “Our ultimate goal is to develop a self-reported application for high risk communities to assess heart disease risk in advance of visiting a clinic.
This could be a cheap, simple and effective of identifying patients who need further investigation. However, the algorithm requires further refinement and external validation in other populations and ethnicities.”
It is known already that certain facial features are associated with an increased risk of heart disease. These include thinning or grey hair, wrinkles, ear lobe crease, xanthelasmata (small, yellow deposits of cholesterol underneath the skin, usually around the eyelids) and arcus corneae (fat and cholesterol deposits that appear as a hazy white, grey or blue opaque ring in the outer edges of the cornea).
However, they are difficult for humans to use successfully to predict and quantify heart disease risk.
Prof. Zheng, Professor Xiang-Yang Ji, who is director of the Brain and Cognition Institute in the Department of Automation at Tsinghua University, Beijing, and other colleagues enrolled 5,796 patients from eight hospitals in China to the study between July 2017 and March 2019.
The patients were undergoing imaging procedures to investigate their blood vessels, such as coronary angiography or coronary computed tomography angiography (CCTA). They were divided randomly into training (5,216 patients, 90%) or validation (580, 10%) groups.
Trained research nurses took four facial photos with digital cameras: one frontal, two profiles and one view of the top of the head.
They also interviewed the patients to collect data on socioeconomic status, lifestyle and medical history. Radiologists reviewed the patients’ angiograms and assessed the degree of heart disease depending on how many blood vessels were narrowed by 50% or more (≥ 50% stenosis), and their location. This information was used to create, train and validate the deep learning algorithm.
The researchers then tested the algorithm on a further 1,013 patients from nine hospitals in China, enrolled between April 2019 and July 2019. The majority of patients in all the groups were of Han Chinese ethnicity.
They found that the algorithm out-performed existing methods of predicting heart disease risk (Diamond-Forrester model and the CAD consortium clinical score).
In the validation group of patients, the algorithm correctly detected heart disease in 80% of cases (the true positive rate or ‘sensitivity’) and correctly detected heart disease was not present in 61% of cases (the true negative rate or ‘specificity’). In the test group, the sensitivity was 80% and specificity was 54%.
Prof. Ji said: “The algorithm had a moderate performance, and additional clinical information did not improve its performance, which means it could be used easily to predict potential heart disease based on facial photos alone.
The cheek, forehead and nose contributed more information to the algorithm than other facial areas. However, we need to improve the specificity as a false positive rate of as much as 46% may cause anxiety and inconvenience to patients, as well as potentially overloading clinics with patients requiring unnecessary tests.”
As well as requiring testing in other ethnic groups, limitations of the study include the fact that only one centre in the test group was different to those centres which provided patients for developing the algorithm, which may further limit its generalisabilty to other populations.
In an accompanying editorial, Charalambos Antoniades, Professor of Cardiovascular Medicine at the University of Oxford, UK, and Dr Christos Kotanidis, a DPhil student working under Prof. Antoniades at Oxford, write: “Overall, the study by Lin et al. highlights a new potential in medical diagnostics.
The robustness of the approach of Lin et al. lies in the fact that their deep learning algorithm requires simply a facial image as the sole data input, rendering it highly and easily applicable at large scale.”
They continue: “Using selfies as a screening method can enable a simple yet efficient way to filter the general population towards more comprehensive clinical evaluation.
Such an approach can also be highly relevant to regions of the globe that are underfunded and have weak screening programmes for cardiovascular disease. A selection process that can be done as easily as taking a selfie will allow for a stratified flow of people that are fed into healthcare systems for first-line diagnostic testing with CCTA.
Indeed, the ‘high risk’ individuals could have a CCTA, which would allow reliable risk stratification with the use of the new, AI-powered methodologies for CCTA image analysis.”
They highlight some of the limitations that Prof. Zheng and Prof. Ji also include in their paper. These include the low specificity of the test, that the test needs to be improved and validated in larger populations, and that it raises ethical questions about “misuse of information for discriminatory purposes.
Unwanted dissemination of sensitive health record data, that can easily be extracted from a facial photo, renders technologies such as that discussed here a significant threat to personal data protection, potentially affecting insurance options. Such fears have already been expressed over misuse of genetic data, and should be extensively revisited regarding the use of AI in medicine”.
The authors of the research paper agree on this point. Prof. Zheng said: “Ethical issues in developing and applying these novel technologies is of key importance. We believe that future research on clinical tools should pay attention to the privacy, insurance and other social implications to ensure that the tool is used only for medical purposes.”
Prof. Antoniades and Dr. Kotanidis also write in their editorial that defining CAD as ≥ 50% stenosis in one major coronary artery “may be a simplistic and rather crude classification as it pools in the non-CAD group individuals that are truly healthy, but also people who have already developed the disease but are still at early stages (which might explain the low specificity observed)”.
Coronary artery disease (CAD) remains the leading cause of death and chronic disability in cardiovascular diseases for all regions of the world.1 Precise, practical and cost-effective tools to screen CAD are urgently needed.
Except for conventional prediction models based on clinical risk factors,2–9 some facial features were associated with increased risk of CAD, which might provide a potential means for disease screening.10 For instance, alopecia, grey hair, facial wrinkle, earlobe crease, xanthelasmata, and arcus corneae were found to be probably associated with increased risk of CAD and poor cardiovascular health.11–13
Further studies demonstrated that these facial features may have a fair performance in identifying CAD or improve the performance of traditional prediction model.10,14
However, use of such facial features for CAD screening has been limited by the (i) few categories and low prevalence of facial features, (ii) lack of specific definitions and quantifiable severity grading, and (iii) poor reproducibility in human identification.10,11,15
A tool to integrate all facial features associated with CAD for disease screening is therefore warranted. As artificial intelligence has evolved, the deep learning algorithm has become a promising tool for disease diagnosis and prediction based on facial photos, especially for genetic and endocrine diseases.16,17
Thus, we hypothesized that this novel approach may help to integrate facial features for detecting CAD. And this study aimed to develop and validate a deep learning algorithm to detect CAD using facial photos.
References
1Roth GA , Johnson C , Abajobir A , Abd-Allah F , Abera SF , Abyu G , Ahmed M , Aksut B , Alam T , Alam K , Alla F , Alvis-Guzman N , Amrock S , Ansari H , Arnlov J , Asayesh H , Atey TM , Avila-Burgos L , Awasthi A , Banerjee A , Barac A , Barnighausen T , Barregard L , Bedi N , Belay Ketema E , Bennett D , Berhe G , Bhutta Z , Bitew S , Carapetis J , Carrero JJ , Malta DC , Castaneda-Orjuela CA , Castillo-Rivas J , Catala-Lopez F , Choi JY , Christensen H , Cirillo M , Cooper LJr , Criqui M , Cundiff D , Damasceno A , Dandona L , Dandona R , Davletov K , Dharmaratne S , Dorairaj P , Dubey M , Ehrenkranz R , El Sayed Zaki M , Faraon EJA , Esteghamati A , Farid T , Farvid M , Feigin V , Ding EL , Fowkes G , Gebrehiwot T , Gillum R , Gold A , Gona P , Gupta R , Habtewold TD , Hafezi-Nejad N , Hailu T , Hailu GB , Hankey G , Hassen HY , Abate KH , Havmoeller R , Hay SI , Horino M , Hotez PJ , Jacobsen K , James S , Javanbakht M , Jeemon P , John D , Jonas J , Kalkonde Y , Karimkhani C , Kasaeian A , Khader Y , Khan A , Khang YH , Khera S , Khoja AT , Khubchandani J , Kim D , Kolte D , Kosen S , Krohn KJ , Kumar GA , Kwan GF , Lal DK , Larsson A , Linn S , Lopez A , Lotufo PA , El Razek HMA , Malekzadeh R , Mazidi M , Meier T , Meles KG , Mensah G , Meretoja A , Mezgebe H , Miller T , Mirrakhimov E , Mohammed S , Moran AE , Musa KI , Narula J , Neal B , Ngalesoni F , Nguyen G , Obermeyer CM , Owolabi M , Patton G , Pedro J , Qato D , Qorbani M , Rahimi K , Rai RK , Rawaf S , Ribeiro A , Safiri S , Salomon JA , Santos I , Santric Milicevic M , Sartorius B , Schutte A , Sepanlou S , Shaikh MA , Shin MJ , Shishehbor M , Shore H , Silva DAS , Sobngwi E , Stranges S , Swaminathan S , Tabares-Seisdedos R , Tadele Atnafu N , Tesfay F , Thakur JS , Thrift A , Topor-Madry R , Truelsen T , Tyrovolas S , Ukwaja KN , Uthman O , Vasankari T , Vlassov V , Vollset SE , Wakayo T , Watkins D , Weintraub R , Werdecker A , Westerman R , Wiysonge CS , Wolfe C , Workicho A , Xu G , Yano Y , Yip P , Yonemoto N , Younis M , Yu C , Vos T , Naghavi M , Murray C. Regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol 2017;70:1–25.
2Peter WF , Ralph BD , Daniel L , Belanger AM , Silbershatz H , Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837–1847.
3Diamond GA , Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 1979;300:1350–1358.
4Pryor DB , Harrell FEJr , Lee KL , Califf RM , Rosati RA. Estimating the likelihood of significant coronary artery disease. Am J Med 1983;75:771–780.
5Genders TS , Steyerberg EW , Alkadhi H , Leschka S , Desbiolles L , Nieman K , Galema TW , Meijboom WB , Mollet NR , de Feyter PJ , Cademartiri F , Maffei E , Dewey M , Zimmermann E , Laule M , Pugliese F , Barbagallo R , Sinitsyn V , Bogaert J , Goetschalckx K , Schoepf UJ , Rowe GW , Schuijf JD , Bax JJ , de Graaf FR , Knuuti J , Kajander S , van Mieghem CA , Meijs MF , Cramer MJ , Gopalan D , Feuchtner G , Friedrich G , Krestin GP , Hunink MG , The CAD Consortium. A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur Heart J 2011;32:1316–1330.
6Bittencourt MS , Hulten E , Polonsky TS , Hoffman U , Nasir K , Abbara S , Di Carli M , Blankstein R. European Society of Cardiology-recommended coronary artery disease consortium pretest probability scores more accurately predict obstructive coronary disease and cardiovascular events than the Diamond and Forrester score: the Partners Registry. Circulation 2016;134:201–211.
7Almeida J , Fonseca P , Dias T , Ladeiras-Lopes R , Bettencourt N , Ribeiro J , Gama V. Comparison of coronary artery disease consortium 1 and 2 scores and Duke clinical score to predict obstructive coronary disease by invasive coronary angiography. Clin Cardiol 2016;39:223–228.
8Fordyce CB , Douglas PS , Roberts RS , Hoffmann U , Al-Khalidi HR , Patel MR , Granger CB , Kostis J , Mark DB , Lee KL , Udelson JE , for the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) Investigators. Identification of patients with stable chest pain deriving minimal value from noninvasive testing: the PROMISE minimal-risk tool, A secondary analysis of a randomized clinical trial. JAMA Cardiol 2017;2:400–408.
9Genders TS , Steyerberg EW , Hunink MG , Nieman K , Galema TW , Mollet NR , de Feyter PJ , Krestin GP , Alkadhi H , Leschka S , Desbiolles L , Meijs MF , Cramer MJ , Knuuti J , Kajander S , Bogaert J , Goetschalckx K , Cademartiri F , Maffei E , Martini C , Seitun S , Aldrovandi A , Wildermuth S , Stinn B , Fornaro J , Feuchtner G , De Zordo T , Auer T , Plank F , Friedrich G , Pugliese F , Petersen SE , Davies LC , Schoepf UJ , Rowe GW , van Mieghem CA , van Driessche L , Sinitsyn V , Gopalan D , Nikolaou K , Bamberg F , Cury RC , Battle J , Maurovich-Horvat P , Bartykowszki A , Merkely B , Becker D , Hadamitzky M , Hausleiter J , Dewey M , Zimmermann E , Laule M. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ 2012;344:e3485.
10Christoffersen M , Tybjærg-Hansen A. Visible aging signs as risk markers for ischemic heart disease: epidemiology, pathogenesis and clinical implications. Ageing Res Rev 2016;25:24–41.
11Christoffersen M , Frikke-Schmidt R , Schnohr P , Jensen GB , Nordestgaard BG , Tybjærg-Hansen A. Visible age-related signs and risk of ischemic heart disease in the general population a prospective cohort study. Circulation 2014;129:990–998.
12Schnohr P , Lange P , Nyboe J , Appleyard M , Jensen G. Gray hair, baldness, and wrinkles in relation to myocardial infarction: the Copenhagen city heart study. Am Heart J 1995;130:1003–1010.
13Ang M , Wong W , Park J , Wu R , Lavanya R , Zheng Y , Cajucom-Uy H , Tai ES , Wong TY. Corneal arcus is a sign of cardiovascular disease, even in low-risk persons. Am J Ophthalmol 2011;152:864–871.e1.
14Wang Y , Mao LH , Jia EZ , Li ZY , Ding XQ , Ge PC , Liu Z , Zhu TB , Wang LS , Li CJ , Ma WZ , Yang ZJ. Relationship between diagonal earlobe creases and coronary artery disease as determined via angiography. BMJ Open 2016;6:e008558.
15Gunn DA , Murray PG , Tomlin CC , Rexbye H , Christensen K , Mayes AE. Perceived age as a biomarker of ageing: a clinical methodology. Biogerontology 2008;9:357–364.
16Kosilek RP , Frohner R , Würtz RP , Berr CM , Schopohl J , Reincke M , Schneider HJ. Diagnostic use of facial photo analysis software in endocrine and genetic disorders: review, current results and future perspectives. Eur J Endocrinol 2015;173:M39–M44.
Google ScholarCrossrefPubMed 17Gurovich Y , Hanani Y , Bar O , Nadav G , Fleischer N , Gelbman D , Basel-Salmon L , Krawitz PM , Kamphausen SB , Zenker M , Bird LM , Gripp KW. Identifying facial phenotypes of genetic disorders using deep learning. Nat Med 2019;25:60–64.
18Buderer NM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med 1996;3:895–900.
19Stephen ID , Hiew V , Coetzee V , Tiddeman BP , Perrett DI. Facial shape analysis identifies valid cues to aspects of physiological health in Caucasian, Asian, and African populations. Front Psychol 2017;8:1883.
20Coetzee V , Perrett DI , Stephen ID. Facial adiposity: a cue to health? Perception 2009;38:1700–1711.
21Eric NR , Robert MH , Karen CS. Predicting adult health and mortality from adolescent facial characteristics in yearbook photographs. Demography 2009;46:27–41.
22Christensen K , Thinggaard M , McGue M , Rexbye H , Hjelmborg JV , Aviv A , Gunn D , van der Ouderaa F , Vaupel JW. Perceived age as clinically useful biomarker of ageing: cohort study. BMJ 2009;339:b5262.
23Zhao C , Li GZ , Li F , Wang Z , Liu C. Qualitative and quantitative analysis for facial complexion in traditional Chinese medicine. Biomed Res Int 2014;2014:1–17. 24Reeh J , Therming CB , Heitmann M , Højberg S , Sørum C , Bech J , Husum D , Dominguez H , Sehestedt T , Hermann T , Hansen KW , Simonsen L , Galatius S , Prescott E. Prediction of obstructive coronary artery disease and prognosis in patients with suspected stable angina. Eur Heart J 2019;40:1426–1435.
25Miller JM , Rochitte CE , Dewey M , Arbab-Zadeh A , Niinuma H , Gottlieb I , Paul N , Clouse ME , Shapiro EP , Hoe J , Lardo AC , Bush DE , de Roos A , Cox C , Brinker J , Lima JAC. Diagnostic performance of coronary angiography by 64-row CT. N Engl J Med 2008;359:2324–2336.
Source:
ESC
