Last year I had the pleasure of supervising Dylan Dophemont, then a student game & media technology and now a game developer at Organiq. During his project Dylan created a game for medical image annotation. In this blog post I summarize this project.
Annotating airways
The game addresses the problem of annotating airways in chest CT images. The measurements of airways are important for diagnosis and monitoring of different lung diseases. Currently this is done manually, by looking at 2D slices of the 3D chest image, and outlining the airway – a dark ellipse with a lighter ring around it. The dark part is the inside of the airway, and the lighter ring is the airway wall.
This is very time-consuming – annotating 1 CT image can cost an expert the whole day. Machine learning algorithms for this purpose exist too, but they are not yet robust enough because there are not enough annotated images that are available for training.
Crowdsourcing and games
To address the problem, prior to this project we experimented with using crowdsourcing on Amazon Mechanical Turk to annotate the airways. This was challenging, as it turned out that most people did not read the instructions!
Most did try to annotate airways, but only annotated only the inside or the outside of the airway, not both as we wanted. Nevertheless, it was easy to filter out the annotations that weren’t usable. For the remaining annotations, the measurements correlated well with the expert measurements of the airways – see our paper about it (also on arXiV).
In Dylan’s project the goal was to gamify this process. After creating several game concepts, he settled on ValleyWatch. This is a casual game, where the world is generated by the lung image – notice several round valleys (round valleys) in the screenshot. The players have to take care of the world by sending rangers to put out forest fires. To put out a fire, the player switches to a screen where he or she can outline the valley – and therefore, the airway!
Dylan conducted several playtests for this prototype, both with game design students and medical imaging researchers. Although it was not always a clear to the players what to do, the game was received enthusiastically by both. Here, too, we saw that reading the instructions paid off – the players who did this created accurate annotations!
More?
Next steps in this project would be to extend the prototype by adding a tutorial, add functionality to “change world” (i.e. load a lung iamge of a different person), investigate how different game elements affect the quality of annotations, and more. If you are interested in a student project on this or related topics, please get in touch!