How can camera images be used during minimally invasive surgeries to assess whether the operated organ is sufficiently perfused with blood? Scientists from the German Cancer Research Center (DKFZ) and the Städtisches Klinikum Karlsruhe have now succeeded in automatic monitoring of ischemia during kidney surgery using only the optical properties of the tissue and not relying on injection of a contrast agent. To do this, they combined a special imaging technique with artificial intelligence methods. The new method has now been used for the first time in patients with kidney tumours.
Minimally invasive surgical techniques, also known as “keyhole surgery”, are increasingly replacing conventional open operations—also in cancer care. For example, certain tumours of the internal organs can be removed in such laparoscopic procedures. This involves the use of an instrument known as a laparoscope, which is inserted into the abdomen through a small incision in the skin and then allows the physician to view the affected organs.
The video camera connected to the laparoscope delivers an RGB image signal. However, it has been shown that human tissue has optical properties that cannot be represented with this image information alone. In the scope of the European Research Council (ERC) project NEURAL spicing and in collaboration with the Städtisches Klinikum Karlsruhe, Lena Maier-Hein (DKFZ) and her team, therefore, have investigated the use of multispectral imaging in this context. These multispectral camera systems not only capture RGB, but provide a multispectral measurement covering over 16 wavelength ranges. This makes it possible to visualise certain functional properties of the tissue that are invisible to conventional camera systems, such as the blood perfusion status of an organ.
This is relevant, for example, in patients with kidney cancer when the tumour has to be removed surgically. During this intervention, the blood flow to the affected areas must be interrupted by clamping the arteries. Only relying on the conventional laparoscopic RGB video makes it virtually impossible to assess the state of tissue perfusion. Only by using a fluorescent contrast agent, which is injected into the patient and subsequently accumulates in the perfused tissues, is the physician able to see whether the correct segments of the organ have been cut off from the blood supply. If this was not successful, the contrast medium has to be washed out for about 30 minutes so that the procedure can then be repeated.
“We were able to show that blood perfusion in the kidney can be visualised without contrast agents and in real time using a combination of multispectral imaging and artificial intelligence”, says Lena Meier-Hein. The new technology has now been successfully used in a study in ten patients who had to undergo partial nephrectomy. “Because our approach allows the blood perfusion status of the tissue to be monitored live, the clamping of the blood vessels can be corrected immediately if necessary. This reduces the duration of the procedure, making it safer for patients”, says Leonardo Ayala, lead author of the study. The new technology also eliminates the risk of an allergic reaction to the contrast agent.
Two aspects played a decisive role in the acquisition and processing of the image data. The use of multispectral imaging systems in minimally invasive surgery has failed so far, among other things, due to the time required by the systems to acquire and process the data. If several seconds are needed to calculate a new image, the information is practically unusable during the procedure. By using novel multispectral camera technology, the researchers have now been able to solve this problem. Their newly developed system delivers 25 frames per second and thus a fluid video image. On the other hand, the perfusion estimation for the first time could be accomplished using AI methods that do not rely on large amounts of training data. The special feature here is that the AI learns to distinguish perfused from non-perfused tissue for each patient anew. “This is necessary because the tissue is not the same in every patient, so the optical properties can also be very different”, explains Leonardo Ayala. Therefore, at the beginning of a surgery, a short video sequence of the kidney is recorded for each patient with the help of a laparoscope. “The algorithm then calculates, based on the patient’s individual image data, how the state of blood flow affects the optical properties of the tissue, so that the degree of blood flow can be displayed during surgery”, Tim Adler added.
Comparable methods could also be used for other surgical issues in the future. Dogu Teber, clinical lead of the project, sees great potential for this new technological approach: “Spectral imaging in combination with new AI-based analysis tools could become an important tool for fast, reliable and safe functional imaging in minimally invasive surgery.”
