EPFL researchers have developed an artificial intelligence-based system that can listen to your cough and indicate whether you have COVID-19.
With the new Coughvid app, developed by five researchers at EPFL’s Embedded Systems Laboratory (ESL), you can record your cough on a smartphone and find out whether you might have COVID-19.
So how can a smartphone app detect the new coronavirus?
“According to the World Health Organization, 67.7% of people who have the virus present with a dry cough – producing no mucus – as opposed to the wet cough typical of a cold or allergy,” says David Atienza, a professor at EPFL’s School of Engineering who is also the head of ESL and a member of the Coughvid development team.
The app is still being developed and will be released in the next few weeks.
Free and anonymous
Once the app is available, users will simply need to install it and record their cough – the results will appear immediately.
“We wanted to develop a reliable, easy-to-use system that could be deployed for large-scale testing,” says Atienza.
“It’s an alternative to conventional tests.” In addition to being easy to use, the app has the advantage of being non-invasive, free and anonymous.
“The app has a 70% accuracy rate,” he adds.
“That said, people who think they may have the disease should still go see their doctor. Coughvid is not a substitute for a medical exam.”
Using artificial intelligence to help patients
Coughvid uses artificial intelligence to distinguish between different types of coughs based on their sound. “The idea is not new.
Doctors already listen to their patients’ coughs to diagnose whooping cough, asthma and pneumonia,” says Atienza.
Right now his team is collecting as much data as possible to train the app to distinguish between the coughs of people with COVID-19, healthy people, and people with other kinds of respiratory ailments.
“We’ll release the app once we’ve accumulated enough data. It could take a few more weeks,” says Atienza.
In the meantime, COVID-19 patients who would like to contribute to the development work can record their cough at https://coughvid.epfl.ch/ or on the Coughvid mobile app.
IS COVID-19 COUGH UNIQUE ENOUGH TO YIELD AI BASED DIAGNOSIS?
Unfortunately, cough is a very common symptom of over a dozen medical conditions caused by either bacterial or viral respiratory infections not related to COVID-19. Several non- respiratory conditions can also cause cough.
Table 1 summarizes the non-COVID-19 medical conditions all of which are known to cause cough. Theoretically, a cough based COVID-19 diagnosis, therefore, must take into account the cough sound data associated with all of the conditions listed in the Table 1.
Human ear is definitely not capable to differentiate cough from all the conditions listed in Table 1. If there are no unique latent features in the cough sound of COVID-19 patients, there is a risk for a cough-based AI diagnosis tool to confuse cough caused by any of the diseases identified in Table 1 with the cough caused by COVID-19.
A brute force-based approach to evaluate this risk would require gathering cough data from a large number of patients for each of the conditions listed in Table 1.
This deluge of data can be then used to train a powerful AI engine such as very deep neural network to see if it can differentiate COVID-19 cough from that caused by all of other medical conditions listed in Table 1.
This approach is not practical at the moment given gathering such all-encompassing data will take too much time, rendering this approach of no help for the current pandemic.
To ensure that our developed solution works in practice with useful accuracy while being trainable with timely available data, we take another approach that we call domain-aware AI-design.
Domain-aware here refers to the fact that the proposed AI engine does not solely rely on blind big data churning, e.g., through a deep neural network. Instead it relies on the deep domain knowledge of medical researchers trained in respiratory and infectious diseases to assess and narrow down the hypothesis testing space, and minimize the amount of data needed to test our hypothesis.
To this end, the medical researcher in our team began by analyzing in-depth the pathomorphological changes caused by COVID-19 in respiratory system by examining the data reported in numerous recent X-rays [13], [18]–[20] and CT- scans based studies [14], [21], [22] of COVID-19 patients.
The goal here is to see if the pathomorphological alter- ations caused by COVID-19 are distinct from that of other common medical conditions, particularly the ones identified in Table 1, that are well known to cause cough.
If this turns out to be the case, then in cough caused by COVID- 19 we should have latent features distinct from the cough caused by the other medical conditions. An appropriately designed AI should then be able to pick these cough feature idiosyncratic to COVID-19 infection and yield a reliable diagnosis, given enough labeled data.
If no such differences exist at pathomorphological level, the idea of cough based COVID-19 diagnosis should be dropped. Because in that case, any AI based diagnosis yielded from cough is more likely to be frivolous correlation and not a meaningful causal relationship. Such-AI based diagnosis will be an artifact of the training data rather than unique latent features of COVID-19 caused cough. Such domain oblivious solution irrespective of its performance in lab, will not be useful in practice.
DISTINCT PATHOMORPHOLOGICAL ALTERNATIONS IN RESPIRATORY SYSTEM CAUSED BY COVID-19
Recent studies show that in COVID-19 infected people, there are distinct early pulmonary pathological signs even before the onset of the symptoms of COVID-19 such as dry cough, fever and some difficulty in breathing [23].
Early histological changes include evident alveolar damage with alveolar edema and proteinaceous exudates in alveolar spaces, with granules; inflammatory clusters with fibrinoid material and multinucleated giant cells; vascular congestion.
Reactive alveolar epithelial hyperplasia and fibroblastic pro- liferation (fibroblast plugs) was indicative of early organi- zation [23].
CT scan based studies shows that on the early stage of COVID-19 disease, it mainly manifests as inflammatory in- filtration restricted to the subpleural or peribronchovascular regions of one lung or both lungs, exhibiting patchy or segmental pure ground-glass opacities (GGOs) with vascular dilation.
There is an increasing range of pure GGOs and involvement of multiple lobes of lung, consolidation of lesions and crazy-paving patterns during the progressive stage. There are diffuse exudative lesions and lung “white- out” during advanced stage [24].
In some patients, COVID-19 leads to onset of pneumonia and pneumonia is marked by a peculiar cough. However, pneumonia can also be caused by many other factors includ- ing non-COVID-19 viral or bacterial infections.
Therefore, the question arises: is there a difference between COVID-19 caused pneumonia and other types of pneumonia that can be expected to translate into a difference in associated cough’s latent features?
Researches show that compared to non- COVID-19 related pneumonia, COVID-19 related pneumo- nia on chest CT scan was more likely to have a peripheral distribution (80% vs. 57%), ground-glass opacity (91% vs. 68%), vascular thickening (59% vs. 22%), reverse halo sign (11% vs. 9%) and less likely to have a central+peripheral distribution (14% vs. 35%), air bronchogram (14% vs. 23%), pleural thickening (15% vs. 33%), pleural effusion (4% vs.39%) and lymphadenopathy (2.7% vs. 10.2%) [25].
These findings suggest that cough sound signatures with COVID-19 caused pneumonia are likely to have some idiosyncrasies stemming from the distinct underlying path- omorphological alterations.
Furthermore, AI based analyses of X-ray [13] and CT scan [14] of the respiratory system have also shown to exploit the differences in pathomorphological alternations caused by COVID-19 to perform differential diagnosis among bacterial infection, non-COVID-19 viral infection and COVID-19 viral infection, with remarkable accuracy.
This further implies that COVID-19 affects the respiratory system in a fairly distinct way compared to other respiratory infections. Therefore, it is logical to hypothesize that the sound waves of cough produced by COVID-19 infected respiratory system is also likely to have unique latent features and the risk of these features overlapping with those associated with other respiratory infections is low.
This hypothesis is supported not only by our own prior studies for diagnosing several common respiratory diseases using cough [15], but also in a recent clinically validated and widely publicized study [26].
In [26], a large team of researchers showed that cough alone can be used to diagnose asthma, pneumonia, bronchiolitis, croup, and lower respiratory tract infections with over 80% sensitivity and specificity. Another very recent study shows that cough can be used to successfully diagnose influenza [27].
However, to the best of authors’ knowledge, no study so far has leveraged cough for diagnosis of COVID-19 while presenting a mobile app base solution for anytime, anywhere virtual testing and pre-screening for COVID-19.
- THE AI4COVID-19 AI-ENGINE
In this section we explain the system architecture and the details of a two stage solution that we developed for:
1) detection of cough sound from mixed cough, non-cough, and noisy sounds; and
2) diagnosis of COVID-19 from the cough sound.
A. SYSTEM ARCHITECTURE
The overall system architecture is illustrated in Figure 2. The smartphone app records sound/cough when prompted with the press and release button.
The recorded sounds are transmitted to the server when the diagnosis button is pressed. At the server, the sounds are first fed into the cough detector.
In case, the sound is not detected to be cough, the server commands the app to prompt so. In case, the sound is detected to be a cough, the sound is forwarded to the three parallel, independent classifier systems: Deep Learning-based Multi Class classifier (DL-MC), Classical Machine Learning-based Multi Class classifier (CML-MC), Deep Learning-based Binary Class classifier (DL-BC).
The results of the all three classifiers are then passed on to a mediator. The app reports a diagnosis only if all three independent classifiers return identical classification results. If the classifiers do not agree, the app returns ‘test inconclusive’.
This tri-pronged mediator centered novel archi- tecture is designed to effectively minimize the probability of misdiagnosis. With this architecture, results show that AI4COVID-19 engine predicting ‘COVID-19 likely’ when
the subject is not suffering from COVID-19 or vice-versa is extremely low, almost zero on the testing data available at the time of writing. This idea is inspired from the “second opinion” practice in health care. The added caution here is that three independent opinions (diagnosis) are solicited, each with veto power. How this novel architecture manages to reduce the overall misdiagnosis rate of the AI4COVID-19 despite the relatively higher misdiagnoses rate of individual classifiers is further explained in Section V-C through (9) and (10).
The details of detection and diagnosis classifiers are presented below.
COUGH DETECTION
The recorded cough sample is forwarded to our network server where the cough detector engine first computes its Mel-spectrogram (as explained in Section III-C), converts it into grayscale to unify the intensity scaling and reduce the image dimensions, and feed it into the Convolutional Neural Network (CNN) to decide whether the recorded sound is of cough or not.
An overview of our used CNN structure is shown in Figure 3. As the input Mel spectrogram image is of high dimensions, it’s first passed through a 2 x 2 max-pooling layer to reduce the overall model complexity before proceeding.
This relatively smaller image is then passed to two convolutional layers consisting of 32 filters and 5 x 5 kernel size with a max-pooling layer in between to learn the complex features, which are then passed to a 2 x 2 max-pooling to represent the learned features in lower dimensions.
The image is flattened to 1 dimension and then passed to two fully connected layers with 128 neurons each.
0.5 dropout is used in both of these to avoid overfitting.
The final layer is the softmax classification layer with 2 neurons to distinguish between cough and not cough for the given input. The number of convolutional and fully connected layers are kept low to minimize potential overfitting issues. Since ReLU is the current standard for
CNNs, it is used for the activation functions of this model, while Adam [29] is used as the optimizer due to its relatively better efficiency and flexibility. A binary cross entropy loss function completes the detection model.
COVID-19 DIAGNOSIS
In the case of cough detection, the cough sound is forwarded to our tri-pronged mediator-centered AI engine to diagnose between COVID-19 and non-COVID-19 coughs. In order to produce results with maximum reliability, with the limited data available at the moment, the three classifiers used in the system use different approaches and are designed independently by three teams and cross-validated. The three classification approaches are described below.
Deep Learning-based Multi Class classifier (DL-MC)
The first solution leverages a CNN based four class clas- sifiers using Mel spectrograms (described above) as input. The four classes here are cough caused by:
1) COVID-19,
2) pertussis,
3) bronchitis, or
4) normal person with no known infection.
The same type of machine learning categorization procedure used for binary-class cough detection is applied to the labeled cough data originating from several different illnesses.
However, due to the differences between two coughs of differing illnesses typically being subtler than a cough and non-cough event, a more complex set of layering is required for this version of the CNN. This classifier architecture is illustrated in Figure 4.
The diagnosis CNN begins with a 2 x 2 max-pooling layer which is followed by two blocks
of layers, each block comprising two convolutional layers followed by a 2 2 max pooling layer and a 0.15 dropout.
All the convolutional layers have 16 filters and a 3 x 2 kernel size. The complex learned features from these 4 convolutional layers are flattened and then passed to a fully connected layer of 256 neurons followed by a 0.15 dropout layer to prevent overfitting. Finally, the output layer with 4 neurons and softmax activation function is used to classify
the input between 4 possible diseases.
Classical Machine Learning-based Multi Class classifier (CML-MC)
A second parallel diagnosis test uses classic machine learning instead of deep learning. It begins with a different preprocessing of cough sounds. Instead of using spectrogram like the first classifier, it uses MFCC and PCA based feature extraction as explained in Section III-C. These smart features are then fed into a multi-class support vector machine for classification.
Class balance is achieved by sampling from each class randomly such that the number of samples equals to the number of minority class samples, i.e., class with the lowest number of samples. Using the concatenated feature matrix (of mean MFCC and top few PCAs) as input, we perform SVM with k-fold validation for 100,000 iterations.
This approach is illustrated in Figure 5.
Deep Learning-based Binary Class classifier (DL-BC)
The third parallel diagnosis test also uses deep learning based CNN on the Mel spectrogram image of the input cough samples, similar to the first branch of the AI engine, but performs only binary classification of the same input, i.e., is the cough associated COVID-19 or not. The CNN structure used for this technique is the same as the one used for cough detector, elaborated in Figure 3.
Source:
EFPL
