Dermatologists typically classify skin lesions based on several sources of data. Algorithms that merge information can support this classification. An international research team has now developed an algorithm that classifies skin lesions more accurately than previous algorithms using an improved data fusion process.
Many people around the world suffer from skin diseases. For the diagnosis, doctors often combine several sources of information. These include, for example, clinical images, microscopic images and metadata such as patient age and gender. Deep learning algorithms can support the classification of skin lesions by merging all information and evaluating it. Several such algorithms are already under development. However, to apply these learning algorithms in the clinic, they still need to be improved to achieve higher diagnostic accuracy.
New Data Fusion Method Improves Diagnostic Accuracy
A research team led by PD Dr. Tobias Lasser of the Munich Institute of Biomedical Engineering (MIBE) at the Technical University of Munich (TUM) has now developed a new learning algorithm – FusionM4Net – which displays an accuracy of higher mean diagnosis than previous algorithms. The code for FusionM4Net is freely available (https://ciip.in.tum.de/software.html). The new algorithm uses a so-called multi-step multi-modal data fusion process for multi-label classification of skin lesions.
- Multimodal: The learning algorithm includes three different types of data: clinical images, microscopic images of the suspicious skin lesion, and patient metadata.
- Multi-label: The researchers trained the multi-label skin classification algorithm, that is, it can differentiate between five different categories of skin lesions.
- Multi-step: the new algorithm merges the available image data first, then the patient metadata. This two-step process allows the image data and metadata to be weighted in the algorithm’s decision-making process. This significantly distinguishes FusionM4Net from previous algorithms in this field, which merge all data at once.
To assess the diagnostic accuracy of an algorithm, it can be compared to the best existing classification for the dataset used, which is assigned the value 100%. The average diagnostic accuracy of FusionM4Net improved to 78.5% due to the multi-step fusion process, outperforming all other leading algorithms it was compared with.
Towards a clinical application
To aid reproducibility, a publicly available dataset was used to train the algorithm. However, in dermatology, data sets are not standardized everywhere. Depending on the clinic, different types of images and patient information may be available. Thus, for a real clinical deployment, the algorithm must be able to handle the type of data available in each specific clinic.
Together with the Department of Dermatology and Allergology at the University Hospital of LMU Munich, the research team is working intensively to make the algorithm operational for future clinical routine. To this end, the team is currently integrating many datasets that have been standardized for this clinic.
“Future routine clinical use of high diagnostic accuracy algorithms could help ensure that rare diseases are also detected by less experienced physicians and could mitigate decisions affected by stress or fatigue,” says PD Dr. Tobias Lasser. Thus, learning algorithms could help improve the overall level of medical care.
Peng Tang, Xintong Yan, Yang Nan, Shao Xiang, Sebastian Krammer, Tobias Lasser. FusionM4Net: a multi-step multi-modal learning algorithm for multi-label classification of skin lesions. Analysis of medical images, November 22, 2021. DOI: 10.1016/j.media.2021.102307 https://doi.org/10.1016/j.media.2021.102307
- The raw code of FusionM4Net is publicly available at https://ciip.in.tum.de/software.html.
- The Department of Dermatology and Allergology of the University Hospital of the LMU in Munich and the company Avelios Medical GmbH in Munich are partners of TUM in this research project.
- The research project is partially funded by the German Federal Ministry of Health. The China Scholarship Council funds a research grant for one of the authors.
- PD Dr. Tobias Lasser is a senior researcher at the Munich Institute for Biomedical Engineering (MIBE). MIBE is an integrative research institute within the Technical University of Munich (TUM) in Germany. At MIBE, researchers specializing in medicine, natural sciences and engineering come together to develop new methods for diagnosing, preventing and treating disease and to work on technologies that compensate for physical disabilities. Activities cover the entire development process – beginning with the study of basic scientific principles and ending with their application in new medical devices, drugs and software.
Analysis of medical images
The title of the article
FusionM4Net: A Multi-Step Multi-Modal Learning Algorithm for Multi-Label Classification of Skin Lesions
Publication date of articles
Warning: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.