Hepa-Navi: Planning and Navigation System for Liver Resections

“Hepa-Navi: Planning and Navigation System for Liver Resections” project was initially to hire both a PostDoc and a clinical PhD. But, due to initial funding limitations, only a Postdoc was hired that led to some changes in the initial objectives. However, the major objective of Hepa-Navi project, i.e. to provide a complete planning and navigation system, always remained. Hepa-Navi’s initial objectives with both PostDoc and PhD: O1) Novel MR protocols for better visualization of liver anatomy O2) Clinician friendly system for segmentation of liver, blood vessels and tumor O3) Integration of laparoscopic navigated ultrasound and Intra-operative model update using ultrasound. O4) Clinical evaluation and validation of tools The PostDoc hired through the project started by implementing a user-friendly liver blood vessel segmentation method by creating a module in the 3D-Slicer. Later, also helped in integration of liver resection planning software into the 3D Slicer, which led to the development of the first version of Hepa-Navi Planning. With the NORMIT project (normit.no), the system was later made as NORMIT-plan, making it a national level planning system for liver resections. Since the automatic segmentation of different regions was more complicated enough to be PhD topics by itself, through the EU project HiPerNav (coordinated by Ole Jakob Elle), the PostDoc supervised and collaborated with multiple PhDs for these tasks. The PostDoc is, through the HiPerNav project, the main technical supervisor for Pravda, working on deep learning based automatic liver parenchyma segmentation; and co-supervisor to Shanmugapriya, working on deep learning based automatic segmentation of liver blood vessels. Also, the PostDoc is the main supervisor to Yuliia, funded by the Akkuttklinikkan through project proposal by the PostDoc himself, working on automatic tumor segmentation, classification and early prediction algorithm. All these works were part of objective O2. Due to the lack of a clinical PhD funded through this project, the objective O1 was rethought as providing a good visualization tool for liver anatomy. This was made possible with introduction of mixed reality system, Microsoft HoloLens, where physical and digital objects co-exist and interact in real time providing better 3D depth information to the surgeons. Through projects HoloViz and HoloNav, innovation projects led by the PostDoc and funded by Helse Sør Øst, Postdoc continued to work on providing the complete planning and navigation system in the HoloLens. The resection planning from NORMIT-plan was reimplemented for HoloLens. The HoloLens projects were in industrial collaboration with SopraSteria that provided the development behind the apps. Building a navigation system as per the objective O3 was not realized, as it relied on a prototype of laparoscopic navigation ultrasound to be provided from Trondhiem that was yet not provided. Thus, the objective was made as a task of having a general navigation system for laparoscopic liver resection. Through the HoloNav project, the Postdoc worked to provide a navigation system for liver surgery with the use of Polaris tracking systems and HoloLens. Since liver is soft tissue making it difficult for navigation tasks, the PostDoc worked with another Innovation project OrthoNav where the same technology was used for orthopedic navigation. This system was tested on patients as a diagnostic tool to provide X-ray vision to the clinicians to dynamically view the movement of bones within the body. Further, the Postdoc also collaborated with Andrea, PhD from HiPerNav working on liver navigation, in understanding the need of intraoperative imaging for use of navigation for laparoscopic liver surgery. For the objective O4, the Postdoc worked closely with clinical Postdoc Henrik and clinical PhD Egidijus(HiPerNav) on clinically evaluating the effect of the use of mixed reality for clinical purposes for both liver and congenital heart use. Primary liver cancer, which consists predominantly of Hepatocellular Carcinoma (HCC), is the fifth most common cancer worldwide and the third most common cause of cancer mortality. The incidence of primary liver cancer is increasing in several developed countries and expected to continue increasing. The liver is also a frequent target of metastases from other cancer origins, like colorectal, with an estimated 550,000 cases per year. In Norway, the most frequent liver neoplasms evaluated for surgical treatment are colorectal metastases. Liver resection is also the treatment of choice with hepatic colorectal metastases, even in recurrent cases. Surgical resection of malignant liver tumors is the only curative therapy. The introduction of laparoscopic liver resection is also changing many aspects of liver surgery, not least for the patients. However, operating laparoscopically on a solid organ as the liver is very demanding, and necessitates meticulous studies of the patient’s anatomy in order to preoperatively create a safe plan for the operation. Thus, planning is an integral part of laparoscopic resection workflow, where the surgeons decide beforehand on the resection plan for the surgery. 3D visualisation of liver anatomy with interactive tracing of instruments, i.e. navigation, will be a giant leap forward for laparoscopic liver surgery. The Hepa-Navi project had worked diligently in solving many of the bottlenecks in providing a complete planning and navigation system for laparoscopic liver surgery. The results from O1 has resulted in direct use of mixed reality visualisation for regularly at MDT meetings, and liver and heart surgery planning at the OUS-Rikshospitalet. Also, results from O2 will help in providing a fast automatic segmentation of organs which will help in fast visualisation of medical images in 3D. Thereby, increasing the possibility of using intraoperative imaging and navigation for surgeries. Finally, the results from O3 will help in providing navigation tool that could be used during surgery to easily locate the tumor in livers for laparoscopic liver resection surgery. The project led to multiple successful proposal fundings, such as 1 EU project - HiPerNav (16 PhDs, 2016), 2 Helse Sør-Øst innovation projects (HoloViz-2016 & HoloNav-2017), one Akkuttklikkan OUS PhD funding (2018), 1 Forskningsrådet BIA funding (75million NOK, 2019). Being the first to work on mixed reality for medicine in Norway, we were first featured in the NRK’s news segment. We also received the Microsoft Global Health Innovation Award 2017, and later, won the Computer World's eHelsePrisen 2017 and second prize for the Inven2 IdePrisen 2017. We also got a lot of attention over the years from many media sources. The HoloViz project lead by the Postdoc was proudly selected as one in 18 projects from whole of Norway to be in report for Norwegian Ministry of Health on research and innovation for patient’s best. The Postdoc has also resulted in submission of multiple DOFIs, which were taken under a single umbrella of DOFIs. These DOFIs led to the founding of a unique med-tech company HoloCare AS, with public-private partnership between Inven2 representing OUS, and SopraSteria. HoloCare AS aims provide clinicians with solutions that support them in reducing risk in operations. The formation of HoloCare AS guarantees better and faster use of results from HepaNavi project into daily clinical routines in Norway and around the world.