3D technology to monitor changes in the skin to prevent melanomas.
- gm2055
- Jul 10, 2022
- 2 min read
Project description
Pigmentation changes in existing skin markers or the appearance of new markers are important indicators for the early diagnosis of melanoma, which can appear anywhere on the body with a high incidence on the face.
Several companies in the world have developed professional equipment for monitoring skin markers using whole body photography for the prediction of potential skin cancers. Links 1-2 below show examples of medical institutions that use whole body photography.
In this project we are using new advances in 3D technology to track skin markers in the facial region over time.
The team at MediaPipe (link 3 below) released "Face Mesh" to estimate 3D face shape in real time from a single photograph using a 468-point mesh. This technology is usually applied to mobile devices, but it is also possible to develop applications in Python language for other operating systems. "MediaPipe Face Mesh" produces an excellent 3D reconstruction of the face from a single image, as shown in the figure above left, in which the 3D reconstruction has been superimposed on the 2D image, and it is impossible to detect the overlap. The figure on the top right has been rotated showing that the figure on the left actually contains the 3D model generated with Mediapipe.
The figure below shows a real case study. The image has been enlarged to the maximum to protect the privacy of the patient. The images have been taken in July 2019 and March 2020, 8 months apart, and are part of a total of 10 studies of the same patient. The circles represent skin changes that have been noted by a specialist. The crosses are our algorithm's predictions of the position of the circles in the previous study, 8 months earlier. The 3D mesh is displayed as triangular cells. Expressed in everyday words, our algorithm predicts the position of the crosses by drawing them in the same cell, and in the same position within each cell, in the previous study. A positioning error is observed in the highest cross because the originating circle is outside the Mediapipe mesh. This shows that the precision in the other markers has been achieved thanks to Mediapipe.
In conclusion, an algorithm has been designed for backward tracking of affected skin regions. This is important to assess the evolution of a skin disorder over time.
Comments