Instance segmentation of crystalline cones from x-ray microCT of insect eye
- Abstract number
- 43
- Presentation Form
- Poster Flash Talk + Poster
- Corresponding Email
- [email protected]
- Session
- Stream 4: Diamond Light Source Session 2
- Authors
- Dr. Tunhe Zhou (2), Pierre Ticht (1), Dr. Hans Martin Kjer (4), Assoc. Prof. Vedrana A. Dahl (4), Prof. Anders B. Dahl (4), Assoc. Prof. Emily Baird (3)
- Affiliations
-
1. Lund University
2. Stockholm Univeristy
3. Stockholm University
4. Technical University of Denmark
- Keywords
X-ray microCT, Segmentation, Image analysis, Arthropods, Compound eye, Crystalline cones
- Abstract text
Insects play significant roles in the world ecology and contribute majorly to the biodiversity. Vision is one the most important sensory modalities that they rely on to perform essential activities, such as flight control [1] and collisions avoidance [2], etc. In order to fully understand how insects interact with their environment, it is important to carry out anatomical and functional investigations of their eyes.
Most insects have a pair of compound eyes. The repeated units in a compound eye are called ommatidia – typically hexagonal as we often see. An ommatidium typically consists of cornea, crystalline cone and rhabdom [3]. The knowledge of the orientation of crystalline cones is important for an accurate estimation of the visual interommatidial angles [4], which decides the angular resolution and hence spatial resolution of the insect vision.
X-ray micro-tomography (microCT) has been demonstrated to be an efficient tool to provide external and internal 3D anatomical structure of the eyes [5]. The optical properties across the field of view of one compound eye can be simulated using the 3D information of the cornea extracted from the microCT data [6]. A typical X-ray microCT scan takes a few minutes in synchrotron radiation and a few hours in laboratory system, and produces a 3D image dataset of around 20003 voxels. Such high throughput and high resolution of X-ray microCT in turn demand adequately efficient image analysis tools in order to extract useful information from the extensive quantity of data.
In this project, we developed an instance segmentation tool [7], InSegtCone, that automatically segments individual crystalline cones in insect compound eyes after a limited annotation from the user. The design of the tool is shown in Fig. 1, including eye surface modelling, sub-volume unfolding, cone segmentation and back-transformation. By dividing the highly curved eye surface into several subregions, and modelling them mathematically, the 3D volume of the eye can be resampled and unfolded, which allows for slice-wise texture-based segmentation [8]. The segmented cone labels are back-transformed into the original geometries in the end to allow further analysis.
We have demonstrated InSegtCone on microCT images, acquired both at synchrotron radiation and laboratory X-ray system, of three insect species with differently shaped compound eyes: the Western honeybee Apis mellifera, the buff-tailed bumblebee Bombus terrestris, and the green-veined white butterfly Pieris napi. The result of the segmentation of the honeybee is shown in Fig. 2 as an example. Our tool has successfully extracted 60%-80% of the estimated total number of about 6000 crystalline cones and is about 250 times faster than manual labelling of the individual cones. We believe that InSegtCone can be an important tool for peer scientists for extensive studies of the diversity of anatomies of the compound eyes and vision of insects. The code for InSegtCone and test dataset are available at Github [9].
Fig. 1 The segmentation tool workflow, with examples of one subset of the compound eye of a honeybee.
Fig. 2 Segmented cones of one honeybee compound eye
- References
1. E. Baird, T. Kornfeldt, and M. Dacke, "Minimum viewing angle for visually guided ground speed control in bumblebees," The Journal of Experimental Biology 213, 1625-1632 (2010).
2. N. Linander, M. Dacke, and E. Baird, "Bumblebees measure optic flow for position and speed control flexibly within the frontal visual field," The Journal of Experimental Biology 218, 1051-1059 (2015).
3. M. F. Land, and D.-E. Nilsson, Animal Eyes (Oxford University Press, 2012).
4. D. G. Stavenga, Pseudopupils of Compound Eyes (1979).
5. E. Baird, and G. Taylor, "X-ray micro computed-tomography," Curr. Biol. 27, R289-r291 (2017).
6. G. J. Taylor, P. Tichit, M. D. Schmidt, A. J. Bodey, C. Rau, and E. Baird, "Bumblebee visual allometry results in locally improved resolution and globally improved sensitivity," Elife 8 (2019).
7. P. Tichit, T. Zhou, H. M. Kjer, V. A. Dahl, A. B. Dahl, and E. Baird, "<em>InSegtCone</em>: Interactive Segmentation of crystalline Cones in compound eyes," bioRxiv, 2020.2012.2015.422850 (2020).
8. V. A. Dahl, M. J. Emerson, C. H. Trinderup, and A. B. Dahl, "Content-based Propagation of User Markings for Interactive Segmentation of Patterned Images," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)(2020), pp. 4280-4288.
9. "InSegtCone," https://github.com/zhoutunhe/InSegtCone.git.
