Deep learning segmentation has been found to be effective and accurate for a variety of tasks including auto-contouring for radiotherapy and root phenotyping for agricultural and plant physiological research. Prior work has shown that existing models may not generalise well to new datasets, resulting in an unmet need for a method allowing clinical personnel and biological research labs to conveniently train their own models for new segmentation tasks.
We aimed to investigate the effectiveness of Interactive Machine Learning (IML) for the rapid training of models for use in both plant science and radiotherapy. IML puts the human-in-the-loop in the model training process, allowing them to directly observe and influence the characteristics of the model whilst annotating, yet there is little research investigating the potential of IML for deep learning segmentation model training.
We implemented and investigated various corrective-annotation approaches for multiple segmentation tasks and found that IML via corrective-annotation provides a way to rapidly train fully-automatic models for diverse datasets in plant image segmentation in under two hours of interactive training. In the field of radiotherapy, we found the IML process can be directly utilised for task completion, with the model and humans collaborating whilst the model learns from human feedback. The auto-contouring model was found to continuously provide improvements in contouring time for a heart contouring task, eventually out-performing the existing clinical delineation software. We also quantified agreement of the generated heart contours in terms of radiation dose, finding that for the vast majority of automatic contours there is little deviation between the model predicted and human corrected contours, yet in some rare cases there can be large differences with potentially catastrophic consequences, highlighting the need for continuous monitoring and guidance from a human observer, an intrinsic property of the IML Workflow for task completion.
We revealed new dynamics related to IML model training, finding the combination with interactive-segmentation can exacerbate what is known as the cold-start problem, leading to a longer period of interactive training before the approach becomes competitive with manual correction of contours. In spite of this slower convergence the combined method demonstrates a capability that eventually out-performs the more manual alternative.
The IML methods in the thesis are evaluated using real world image segmentation tasks with real human annotators, enabling immediate translation to several subsequent image-analysis based research projects in various domains.