CoDA: Interactive Segmentation and Morphological Analysis of Dendroid Structures Exemplified on Stony Cold-Water Corals (2024)

I Introduction

Complex dendroidal structures are widespread in nature and commonly formed by various organisms comprising plants, e.g., trees, grass, including their roots, coralline algae (e.g.,[35, 22, 24, 46, 27, 10]), and (marine) animals, e.g., corals, including the herein investigated cold-water corals (CWC), or bryozoans ([22, 30, 1]).Often, these organisms are key players in ecosystems, such as trees in forests or corals in coral reefs, by forming multiple ecological niches beneficial for other organisms, so that these ecosystems represent biodiversity hotspots and provide important ecosystem services (e.g., [42, 2, 13, 21]).

Coral colonies consist of multiple coral polyps that originate from asexual budding.Each polyp produces a skeleton called corallite with a central cavity, called calyx, in which it lives(2(a)).Coral colonies exhibit various shapes (e.g., encrusting, hemispherical, tabular, corymbose and branching; [14]) but colonial stony (scleactinian) cold-water corals are predominantly branching (dendroidal; [33]).Some dendroid cold-water corals commonly show secondary joints, regions in the colony where two different coral branches have grown together, and skeletal strengthening as well as further associated secondary joints related to the presence of worms of the genus Eunice [29].Both represent important features in the analysis of cold-water coral colonies as they influence significantly the morphology of the colonies, which is the result of the interaction of the organism with its ambient environment.However, an in-depth understanding of this interaction is often hindered by the limited methodologies to characterize three-dimensional morphologies.Often, morphologies are only qualitatively described (e.g., for CWC: [20, 9, 11]) or characterized by some basic measurements (e.g., for CWC:[37, 36]).

This work presents a new visual analytics suite called CoDAthat allows a detailed three-dimensional quantitative assessment of dendroidal structures.A major component of CoDAis CoDA.Graph, a browser-based application for showing 2D plots like scatter plots, SPLOMs, maps and graph layouts.CoDA.Graphis not only suited for the analysis of CWCs but any kind of dendroid structures.We tested CoDAon colony fragments of the reef-forming CWCs Lophelia pertusa, Madrepora oculata and Goniocorella dumosa.An overview of the analysis workflow using CoDAis shown in Figure1.Note that in this work, we visualize all binary and instance segmentations as voxelized data for two reasons:(1)It gives the reader a better feeling for the resolution of the data.(2)It is the one used in CoDAas it is more interactive than using a surface representation that needs recomputation with each change.

II Segmentation Workflow

Input to our segmentation pipeline is a 3D image resulting from scanning individual specimens containing one or several coral colony fragments using computed tomography.An iso-surface rendering of such a 3D image is shown in Figure1a.

II-A Initial Calyx Segmentation

The first step in the processing of a 3D image containing coral colonies is the computation of the calyx (polyp cavity) segmentation.This is accomplished with the ambient occlusion and ambient curvature-based method developed by Schmitt et al.[39], which makes use of two observations.Firstly, the cavities are more occluded than regions outside the skeleton[3, 44].Secondly, coral cavities are mostly convex on the outside and concave on the inside.Combining both information in a statistical model for background and foreground, i.e., cavity and non-cavity, gives a soft segmentation of the polyp cavities(calices).The final segmentation is then obtained by computing a minimal partition of the soft segmentation.The result for a small colony fragment is shown in Figure1b.

In the next step, the initial calyx instance segmentation is computed by a classical contour-tree segmentation[5] on the random-walk distance transform[23] of the calyx segmentation.Although a classical Euclidean distance transform may be used as well, we share the findings of Baum et al.[4] that the random-walk distance transform yields slightly better results for the calyx instance segmentation.The persistence value in the contour-tree segmentation is interactively chosen such that most of the small, young corals are segmented correctly.This choice, however, comes with an over-segmentation of larger, adult corals.We find this tradeoff easier to work with than the other way around, since over-segmentations are easier to spot and correct.The initial calyx instance segmentation of the same colony fragment as in Figure1b is shown in Figure3a.

At this point, the initial calyx instance segmentation can already be refined with the tools described in subsectionII-C.The final result is depicted in Figure3b.

II-B Automatic Colony-Tree Computation

Among other information, we are particularly interested in the skeleton tree that describes the mother-daughter corallite relationship in a coral colony, that is, a graph with a vertex for each corallite and a directed edge going from the mother corallite to the daughter corallite.Since corals reproduce asexually within a colony, the graph will be a tree.

The automatic tree computation starts by propagating the calyx instance segmentation, like the one shown in Figure3b, onto the corals skeleton.The propagation algorithm assigns each voxel in the skeleton mask the label of the closest calyx.The result is a corallite instance segmentation as the one shown in Figure4a.

Scanning through the corallite instance segmentation and tracking which instances touch each other yields a region adjacency graph (RAG) as shown in Figure4b.This graph has one vertex for each corallite and one edge for each pair of neighbouring corallites.Unfortunately, the region adjacency graph is too large in the sense that it also contains edges between corallites that are adjacent but do not describe a proper mother-daughter relationship.

In order to prune these superfluous edges, we first orient the corallites.We start by fitting a parabola to the point cloud given by all voxels occupied by the corresponding calyx.Let $\mathcal{V}\subset\mathbb{R}^{3}$ denote the point cloud of voxels belonging to a calyx.Then we seek to find a rotation matrix $R\in O(3)$, an anchor point $a\in\mathbb{R}^{3}$, the curvature of the parabola $\alpha\in\mathbb{R}_{\geq 0}$, as well as the curve parameters $t_{x}\in\mathbb{R}$ for all $x\in\mathcal{V}$, such that the distance between the spatial parabola

$$\gamma(t)=R\begin{pmatrix}t\\\alpha t^{2}\\0\end{pmatrix}+a$$

and the calyx point cloud is minimized. In more formal terms:

$$\min_{\begin{subarray}{c}R\in O(3)\\a\in\mathbb{R}^{3}\\\alpha\in\mathbb{R}\\t_{x}\in\mathbb{R}^{3}\end{subarray}}\sum_{x\in\mathcal{V}}\left\lVert\gamma(t$$

In short, the parameters are initialized by first computing a principal component analysis (PCA) and projecting all voxels onto the first two principal axes.The rotation matrix coincides, up to the sign of the last column, with the projection matrix of the PCA, while the anchor and curve parameters are found by linear regression.The parameters are then refined by alternating the minimization with respect to the parabola parameters $t_{x}$ and all other parameters.Figure4c shows the parabola fit for the A2W specimen.

The parabola fit is finalized by swapping the start $t_{\text{min}}=\min_{x\in\mathcal{V}}t_{x}$ and end $t_{\text{max}}=\max_{x\in\mathcal{V}}t_{x}$, if necessary, such that more points in the point cloud project onto the second half of the parabola, i.e.,

$$\underbrace{\left\lvert\{t_{x}:t_{x}<t_{\text{mid}}\right\}\rvert}_{\text{$$

with $t_{\text{mid}}=\frac{t_{\text{min}}+t_{\text{max}}}{2}$.This condition is inspired by the observation that the coral grows in breadth.Figure4d shows the distribution of the parameters $t_{x}$ and the distance of the voxel $x$ to the parabola for a single calyx.

Based on the oriented parabola, a simple measure describing whether a voxel is closer to the bottom or top of a calyx is defined as

$$h(x)=\begin{cases}0&,\quad\text{if }t_{x}<t_{\text{min}}\\\frac{t_{x}-t_{\text{min}}}{t_{\text{max}}-t_{\text{min}}}&,\quad\text{if }t_{$$

where $t_{x}$ again minimizes the distance of $x$ to the parabola, i.e.,

$$t_{x}=\mathrm{argmin}_{t\in\mathbb{R}}\lVert\gamma(t)-x\rVert_{2}^{2}.$$

An edge $e=(A,B)$ connecting two calices A and B in the region adjacency graph is now oriented such that

$$h_{A}\left(x_{A}\right)\geq h_{B}\left(x_{B}\right),$$

(1)

where $x_{A}\in V_{A}$ is the closest point in calyx A to calyx B, and $x_{B}\in V_{B}$ is analogously the closest point in calyx B to calyx A:

$$\lVert x_{A}-x_{B}\rVert_{2}=\text{dist}(V_{A},V_{B}).$$

These points are easily obtained by tracking the touching points during the corallite instance segmentation.The condition Equation1 orients the edges such that they start at the top of the source corallite (mother) and end at the bottom of the target corallite (daughter).

Most of the remaining superfluous edges connect daughters of the same mother corallite, giving rise to the following configuration:

A simplification procedure prunes the edge between the daughters $\text{B}_{1}$ and $\text{B}_{2}$.

The pruned graph already gives a good estimate for the skeleton tree.However, it may still contain edges for secondary joints.Formally, secondary joints are constituted by the subset of edges within the region adjacency graph that are not part of the skeleton tree.

The automatically computed skeleton graph and the final skeleton tree, after manual refinement of the skeleton graph, are shown in Figure5.

II-C Refinement Tools

The initial instance segmentation of the colony into calices and the associated skeleton graph are not perfect, that is, they may still have errors which require manual intervention.CoDAprovides suitable tools for correcting over-segmentations, under-segmentations, wrongly oriented edges, as well as adding missing and removing superfluous edges.All operations acting on the instance segmentation keep the skeleton graph in sync by inserting or removing edges and vertices when needed.

Over-segmentations are corrected with the merge tool as shown in Figure6.The user marks the instances that should be merged by drawing a line over them.The voxels in the touched connected components are then collected and assigned to a new, common label.Their vertices in the spatial graph are merged as well and internal edges are removed while keeping the edges to non-affected vertices.

Under-segmentations are corrected by drawing a cutting line across the instance that is too large.We fit a plane to the line and the viewing direction, and assign each half-space of the connected component a new label ID, resulting in a clean, scissor-like cut into two segments.The edges of the original calyx are distributed between the two new instances such that they maximize the orientation condition stated in Equation1.Additionally, a new edge is inserted between the two new components.If no orientation of the view allows a clear cut, then the merge and cut tools can be used multiple times until a satisfactory result is achieved.

The vertices of the graph can only be changed by merging, cutting or reassigning the labels of the instance segmentation.With the help of graph editing tools, the user can add missing edges by selecting the source and target vertices in the 3D view.Similarly, superfluous edges are removed by selecting the edge itself and deleting it (Figure6c) as part of the merge workflow, or by selecting the source and target vertices and disconnecting them.

From time to time, the heuristic described in subsectionII-B yields incorrect orientations, especially for small corals and for calices where top and bottom are close by.In those cases, the direction of the adjacent edges can be corrected by selecting and flipping them.

II-D Proofreading

Proofreading in CoDAconsists of two steps.The first step concerns the calyx instance segmentation itself.Here, a major part of wrongly segmented calices are identified by looking at the feature plots in CoDA.Graph, e.g., for volume, surface area, length or the mean distance to the parabolas.Once all outliers are taken care of, a step-through process presents the user with all coral instances one after the other.Initially, all instances are marked as unseen.The user then either marks them as good or corrects them with the proper refinement tools described in subsectionII-C.After refinement, all affected calyx instances are again marked as unseen and placed into the proofreading queue.

Problems with occlusions, for example in inner, dense areas of a colony, are taken care of by showing only the closest, adjacent corals by cropping to a centered region of interest (RoI).Notably, the order in which the user steps through the instances is not random but visits the next, closest, unchecked calyx.Thus larger jumps within the 3D volume image are avoided.The camera of the view is aligned with the principal axes of the calyx to obtain a standardized view.

The second proofreading step concerns the edges of the skeleton graph.The cycle detection highlights the shortest cycle in the graph which can be traced to find and remove the edge that was caused by a secondary joint.Each vertex must also have an in-degree of at most one since each calyx/corallite can have only one or no predecessor.These two hints are the most helpful for identifying wrong edges and are shown in Figure7.Other helpful features during this proofreading stage are the edge orientation criterion in Equation1, the budding angles, edge lengths or contact area widths.It is important to note that the visual hints only work for calices and edges that are in the current segmentation, and that we cannot add visual hints for edges that are not there anymore, once they have been removed.Thus, also for this step, we prefer that the initial instance segmentation is an over-segmentation and contains more calices as well as more edges due to secondary joints.

Similarly to the proofreading of the calyx instance segmentation, the user is provided with a step-by-step tool for proofreading all edges.This guides the user through all vertices (calices).The user marks a vertex as good if all edges starting or ending in the vertex are correct and none are missing.

The proofreading state is just another attribute of the vertices and edges and can thus also be used as source for the color- and glyph-map.This allows the user to easily see which vertices and edges are still marked as unseen or as already been taken care of.

III Implementation

The whole proofreading, analysis and exploration framework is implemented within two different software stacks that communicate via a simple file-based mechanism.The image processing and refinement tools are implemented in Amira[41] and are available as custom modules and extensions to the built-in 3D visual analysis tools.Within Amira, aggregated features are extracted for the individual calices and related corallites, as well as their relationships, and are exported to CoDA.Graph, a web application written in Python using the Bokeh[8] library.CoDA.Graphextends Amira by a rich set of classical 2D visualizations like SPLOMs, scatter plots, histograms, and graph layouts.

III-A Linking between Amira and CoDA.Graph

CoDA.Graphand Amira are entirely decoupled and can be used independently.However, to facilitate an interactive and seamless exploration of the data, a simple communication protocol based on .csv files and a canonical naming scheme is implemented.

On startup, a temporary folder with a known prefix amira_coda_{random_number}/ is created by Amira.CoDA.Graphautomatically detects folders with this prefix and watches their content.Whenever a file in the shared folder is created, deleted or modified, the change is detected automatically by both Amira and CoDA, which will trigger a reload of the respective spreadsheets.To avoid race conditions or undefined behaviour, a spreadsheet is only ever written to by either Amira or CoDAand read by the other, never by both.The origin of the data is again indicated with a prefix.

• •

  amira_vertex_{name}.csv
  A spreadsheet with features exported from Amira in which each row contains data for one vertex (calyx/corallite).All vertex spreadsheets must have the same number of rows.

• •

  amira_edge_{name}.csv
  A spreadsheet with features exported from Amira in which each row contains data for one edge.All edge spreadsheets must have the same number of rows.Additionally, at least one spreadsheet must have two columns with the index of the start and end vertices.

• •

  amira_vertex_colormap.csv
  A spreadsheet with the current color for every vertex (calyx/corallite).The colormap can be used in CoDA.Graphso that the user is presented with the same color scheme in all views.

• •

  amira_edge_colormap.csv
  The same as amira_vertex_colormap.csv but here, each row contains the color for an edge.

• •

  coda_vertex_selection.csv
  A spreadsheet containing a single column in which each row indicates whether a vertex is currently selected in CoDA.Graphor not.This information is used in Amira to show the currently selected calices/corallites in CoDA.Graph, enabling a seamless link-and-brush experience.

• •

  coda_edge_selection.csv
  The same as cora_edge_selection.csv but each row indicates if an edge is currently selected in CoDA.Graphor not.

• •

  coda_vertex_colormap.csv
  The same as amira_vertex_colormap.csv but the colors are the ones displayed in CoDA.Graph.

• •

  coda_edge_colormap.csv
  The same as amira_edge_colormap.csv but the colors are the ones displayed in CoDA.Graph.

The data flow is visualized in Figure8.

At this point, it is worth mentioning that the naming scheme and choice of .csv files for the communication make it easy to use CoDA.Graphwithout Amira.Similarly, an integration of CoDA.Graphinto other applications is just a matter of adhering to the naming scheme within the shared folder.

III-C Exploration and Selection Tools

Selections can be made in any of the CoDA.Graphviews and will be propagated to Amira.Bokeh comes with several useful selection tools out of the box, for example, the lasso, box and tap selection tools, which already allow for a freehand, unconstrained selection in all views.

For the ontogenetic analysis and exploration of the spatial growth patterns of the coral colonies, we implemented custom tools in the graph view.The descendant selection tool selects the sub-tree containing all descendants of the selected calyx/corallite;the ancestor selection tool selects all ancestors of a single corallite;and the connected component selection tool selects all corallites within a connected component, which are all calices/corallites within the same colony fragment.These tools allow the exploration of a corallite’s ancestry within a specimen and to focus on a single colony fragment within a dataset.An overview of the selection tools is shown in Figure9.

IV Case studies

In order to showcase the suitability of CoDAfor the analysis of coral colonies, case studies are presented using the five specimens described in subsectionI-B.The chosen samples differ in size, corallite number, and corallite and colony morphology, so that the study covers many use cases.

IV-A Segmentation

Sample	(I) A2W	(II) C1W	(III) SaM-ID43148	(IV) GeoB127471-1	(V) NIWA-148046
Species	Lophelia pertusa	Lophelia pertusa	Lophelia pertusa	Madrepora oculata	Goniocorella dumosa
Voxels	$257\times 237\times 392$	$275\times 145\times 276$	$1024\times 729\times 1215$	$279\times 115\times 335$	$829\times 455\times 317$
Resolution $[\text{mm}^{3}]$	$0.351\times 0.351\times 0.3$	$0.351\times 0.351\times 0.3$	$0.3125\times 0.3125\times 0.3$	$0.351\times 0.351\times 0.3$	$0.146\times 0.146\times 0.3$
Colony fragments	5	1	67	1	9
Calices/Corallites	114	30	1725	66	262
Buddings
Median	1	1	1	1	0
75% Quantile	2	2	2	2	1
Maximum	4	4	7	3	20
Edges
RAG	148	36	3592	86	409
Initialisation	115	29	2152	67	361
Final	109	29	1658	65	253
RAG $\rightarrow$ Initialisation(automatic)
Computation time	25s	9s	420s	5s	86s
Unchanged edges	68	0	734	29	218
Pruned edges	32	7	1440	19	48
Initialisation $\rightarrow$ Final(manual)
Unchanged edges	107	29	1378	55	172
Removed edges	8	0	711	9	125
Added edges	2	0	217	7	17
Flipped edges	0	0	63	3	64

The initial instance segmentation was carried out similarly for all specimens and follows the workflow and recommendations from sectionII.Separating a segment into two segments may require multiple cuts and subsequent merges since it is sometimes hard to find a cutting plane that separates both segments clearly, while merging two segments only requires a scribble roughly drawn on top of them.Additionally, we find over-segmentations easier to spot than under-segmentations.Two segments differ in color, while an under-segmented area is only recognizable by its shape or as an outlier in one of the morphological feature plots.

The comparatively small specimens of Lophelia pertusa, A2W (I), C1W (II), and Madrepora oculata, GeoB12747-1 (IV), were processed almost fully automatically; the whole process from calyx segmentation to the final colony-tree and instance segmentation took no longer than an hour each.These specimens contain 114, 30 and 64 individual calices/corallites, respectively.The fact that these specimens only contain a few or no secondary joints simplified the processing and manual interaction.Due to their small size and morphology, the calices/corallites do not occlude others within the colony and it was possible to obtain a perfect segmentation without using the RoI centering and cropping tool in the proofreader.

The instance segmentation, skeleton-tree computation and proofreading of the largest sample, SaM-ID43148 (III), which motivated the development of CoDA, took longer than every other specimen with roughly two working weeks.After the initial, contour-tree-based calyx instance segmentation, we used CoDA.Graphto filter small instances with less than $0.5\mathrm{mm}^{3}$ and removed them.Such small cavities belong to small holes in the ambient occlusion mask that is used to obtain the polyp-cavity segmentation on which the calyx instance segmentation is based on, or artefacts due to the scanning resolution.Even if they belong to young corallites, they are too small to be part of a meaningful morphological analysis, so that they would be discarded from further analysis anyway.Next, we examined the initial calyx instance segmentation with the 3D visualizations and corrected all segments that we could spot at first glance.Afterwards, we switched to the proofreading tool for the correction and validation of the calyx instance segmentation.Especially the canonical view with the automatically centered RoI proved to be useful in the inner, dense areas of the specimen.Notably, the calyx segmentation contained many tubes of Eunice worms, which had to be removed.Using the proofread instance segmentation, we computed the initial colony skeleton graph and pruned secondary joints with help of the cycle detection tool, of which SaM-ID43148 (III) contains many.A second proofreading was carried out, in which a few errors that we missed earlier were corrected, both in the calyx instance segmentation, as well as the colony-tree graph.The final instance segmentation contains 1724 individual corals, 1658 edges and 66 colony fragments.

The processing of the Goniocorella dumosa specimen NIWA-148046 (V) proved to be more involved than the others due to its fundamental different morphology.Each corallite of this species has intermediate floors (dissepiments) and budding occurs multiple times during corallite growth.Hence, a single corallite corresponds to multiple calyx subcavities.Figure11a shows an iso-surface rendering and a slice of the original CT intensity image of such a region.In addition to the unique growth pattern, the calyx subcavities are more cylindrical in shape, other than Lophelia pertusa or Madrepora oculata, which are more conic.Thus, they do not exhibit a strong anisotropy in growth direction which hampered the automatic computation of their orientation and the subsequent automatic estimation of the coral colony tree.Contrary to the other corals, here, we used the Euclidean distance transform as scalar field for the contour-tree segmentation, which gave slightly better results due to the more cylindrical shape and already good separated calyx subcavities due to the intermediate floors.The major bottleneck in the proofreading and refinement of this specimen was the merging of the many calyx subcavities belonging to the same corallites.From start to end, the first refinement and proofreading process took roughly two work days and resulted in an instance segmentation with 262 corallites.

IV-B Visual Exploration

After finishing the segmentation of all specimens, we started the visual exploration by trying to confirm some basic hypotheses and questions.

The SPLOM plots of the volume and surface area of both calices and corallites colored by their generations showed us that the corals need a growth-period of up to four generations to reach their adult stage, visible in Figure9d.3 since they are predominantly smaller in size and length than the older generations.After this period, morphological features are indistinguishable even in different branches of the same colony and between colony fragments in the same specimen.

The dendrograms also showed a skeleton tree that grows more in breadth rather than in depth for the Lophelia pertusa specimens compared to the Madrepora oculata specimen.Taking a look at the statistics panel, we could also see that the maximum number of buds (three) in the Madrepora oculata specimen is lower than the maximum number of buds (seven) in the Lophelia pertusa specimens.Note that these are first impression on morphological differences between two species that must be confirmed by the analysis of many more specimens.

In all cases, the colony-tree graph revealed more components, i.e., colony fragments, within a single specimen than identified by looking at the real specimens or the CT images.

By coloring the calices depending on their colony fragment in the SaM-ID43148 (III) sample, we can easily make out the principal directions of growth visually, as shown in Figure12d.While younger generations tend to start growing in a flat area near the ocean floor into many directions, all later generations tend to follow the same direction towards the top left corner in the image.

CoDA.Graphalso allows the exploration of multiple colonies at once.We use this feature of CoDA.Graphto compare the morphology and the ontogenetic development between different specimens/species.All datasets are annotated with location attributes and their ID, which we use to compare the morphologies with respect to their species and location.As shown in Figure13, the scatter plot of the calyx volume and calyx surface area reveal different growth patterns and shapes for different species.The same plot also reveals that the correlation between volume and surface area depends on the species.

V Discussion

Three-dimensional dendroidal structures are quite common in nature (e.g., trees, coral frameworks) and structure habitats, such as forests or coral reefs, increase their complexity which results in an increase of ecological niches and consequently enhanced biodiversity (e.g., [32, 25]).While the provided ecological niches and biodiversity in these habitats are quite well studied, the influence of the actual shape of the three-dimensional dendroidal organisms on the habitat complexity, the ecological niches and biodiversity are nearly unstudied.One reason for this lack of knowledge is the predominantly qualitative or semi-quantitative description of these complex structures.The herein presented CoDAframework provides tools to assess such structures.

Currently, the presented segmentation pipeline and tools are optimised for dendroidal cold-water coral colonies with different corallite and colony morphologies.Even though the development process used only specimens of Lophelia pertusa, CoDAworks well for two other cold-water coral species, i.e., Madrepora oculata and Goniocorella dumosa.However, the different corallite morphology of Goniocorella dumosa with its tube-shaped corallites with multiple intermediate floors (dissepiments) [40] compared to the cone-shaped corallites of Lophelia pertusa and Madrepora oculata) clearly shows the sensitivity of the colony skeleton-tree construction on the calyx/corallite instance segmentation and the subsequent ontogenetic interpretation of the results.In Lophelia pertusa and Madrepora oculata, corallite budding occurs at a specific time of corallite development and usually is a single or double budding (maximal budding up to 7, see TableI).Budding numbers greater than two occur predominantly in Goniocorella dumosa.Its tube-shaped corallites exhibit up to $>20$ consecutive budding events (Figure11, FigureS1), which has fundamental importance for the ontogenetic interpretation of its colony skeleton tree.For an ontogenetic analysis of a colony skeleton tree of Goniocorella dumosa, the calyx/corallite instance segmentation should not be based on the complete corallites (as shown herein) but on the calyx subvolumes enclosed by the intermediate floors (dissepiments; Figure11a).

After a meaningful calyx/corallite instance segmentation, the biggest challenge and prerequisite for the ontogenetic analysis of coral colonies is the correct reconstruction of the colony skeleton tree.A single wrong edge will cause the generation number in a whole branch to be wrong and multiple errors will accumulate fast.With CoDA, we now have a systematic approach and intuitive tools at hand for correcting and proofreading the colony skeleton tree so that the number of errors can be minimized.

Having proofread data for the calyx/corallite instance segmentations and methods to compute it in a timely manner enable us to gather enough training data to leverage machine learning methods, for example, a simple convolutional neural network like a U-Net[34], for learning the corallite borders.The application of this may reduce the number of initial errors such that manual correction of errors will require less time.Up until now, no sufficient training data had been available.The coral colony datasets analyzed within this paper comprise almost 2200 corallites and their adjacencies, making it already a feasible training dataset for modern machine learning methods.

The initial automatic colony skeleton graph computation greatly reduces the amount of superfluous edges in the region adjacency graph.However, some edges are falsely removed, mostly in dense areas within a colony with many secondary joints.Often, these regions are interesting by themselves so that a closer look and manual introspection is required nonetheless.Similarly to the proofread calyx/corallite instance segmentations, the now available colony skeleton trees can be used to bootstrap other methods for classifying the adjacency between corallites, e.g., deep learning-based methods, which will work directly on the original image data and will potentially be able to identify the corallite orientation and mother-daughter relationship as well as the secondary joints even more accurately.Another option might be to employ graph neural networks for link prediction[48].

The parabolas fitted to the calices are not only useful for the colony skeleton tree computation, but provide new morphological features.For example, the mean, standard deviation and skew of the distance distribution give a simple description for the anisotropy of a calyx, that is, the skew of the distribution will be larger for a cone-shaped calyx than for a tubular-shaped one.Each parabola itself provides a robust estimate for the calyx (and corallite) centerline, its curvature provides a measure for the bending of the calyx/corallite, and the arc length provides a better estimate for the calyx’ length than the one based on principal axes (PCA).

Although CoDA.Graphprovides only minor contributions on its own, it combines many proven visualization techniques with a simple user interface and allows the exploration of cold-water corals in a novel, intuitive way.Being able to take a look at the region in the original volume image on which a data point in a plot is based on, may it be a classical measurement or otherwise computed feature, proved to be a very handy and convenient tool during the exploration and analysis of the coral colonies.Especially the linked 3D visualization in Amira and the 2D graph-layout in CoDA.Graphwere remarkably useful during the analysis of the corallite ancestry and provided a way to explore the relationships that would not have been possible within a single 2D or 3D visualization.

VI Conclusion and Future Work

This study provides the technological basis to study complex three-dimensional dendroidal structures systematically.Exemplified on dendroid stony cold-water corals, CoDAand CoDA.Graphare capable to disassemble fragments of dendroidal coral colonies into their principle building blocks (corallites) and construct an ontogenetic meaningful colony skeleton graph – the basis for a quantitative and reproducible analysis of the corallite and coral-colony morphology.Based on this decomposition of coral colonies, an automated classification of colonial cold-water corals, the investigation of intra- and interspecies coral morphoplasticity, local or regional spatial preferences of the distribution of specific morphologies, environmental influences on the coral morphology and its influence of reef-forming capacity of specific morphologies become possible, previously hindered by the qualitative to semi-quantitative acquisition of morphological data of cold-water corals.Our next goal is the statistical and morphological analysis of the most important reef-forming cold-water coral colonies to provide an automated CT-based species identification and investigate similarities and differences in their ontogenetic colony formation, corallite and coral colony morphology and their species-specific morphoplasticity.

The preliminary results presented herein regarding the quantifiable morphological differences between species (Figure13), are already very promising.However, these analyses must currently be restricted to younger parts of colonies as older parts are impacted by bioerosion, especially by bioeroding sponges that produce large cavities that change the shape of the coral skeleton considerably (e.g., [6, 7]).To overcome this restriction, we plan to use the data gathered from the analysis of intact coral colonies, like the ones presented in this paper, to develop and validate methods that deal with bioerosion-altered coral skeletons.Furthermore, the software provides the first step to investigate cold-water coral deposits in more detail.So far, CT scans of sediment cores retrieved from cold-water coral reefs/mounds were only used to reconstruct the coral content, the coral clast size and the clast orientation ([16, 43, 45]).The incorporation of morphological analyses into these analyses following the approach of CoDAwill potentially allow the differentiation of coral species and identification of specific coral morphotypes, which will considerably improve the reconstruction of the temporal development of these cold-water coral reefs/mounds.

We make CoDA.Graphavailable to the public in the hope that it is not only useful for the visual analysis of cold-water coral colonies but also for other dendroid organisms and objects with an inherent tree-like structure.

VII Data and Code Availability

The most recent version of CoDA.Graphis available at https://github.com/zibamira/CTCoral_CoDA.The repository contains the documentation for the link and brush interface described in subsectionIII-A, as well as some of the extracted features for the A2W (I) specimen.

Additionally, we provide the source code for the Amira modules interfacing with CoDA.Graphat https://github.com/zibamira/CTCoral_hxcoda, the shared library for Amira 2024.core is also provided.

Acknowledgements

Arne-Jörn Lemke and Christian Timann, Klinikum Bremen-Mitte, Gesundheit Nord (Bremen), are acknowledged for performing the CT scans and their support during the measurements.We thank Claudia Wienberg (MARUM) for providing access to the GeoB12738-1 specimen, and André Freiwald and Giovanni Sanna (both Senckenberg am Meer) for providing access to SaM-ID43148.Armin Form (GEOMAR) is acknowledged for providing CT scans of the specimens C1W and A2W (BMBF project BIOACID II (FKZ 03F0655A)).Furthermore, we thank the nautical and scientific crews and especially the chief scientists of RV POSEIDON cruise POS455, PV PELAGIA cruise 64PE284, and RV METEOR cruise M70-1 during which the samples were retrieved.Di Tracey and Sadie Mills (both from the National Institute of Water and Atmospheric Research (NIWA, New Zealand) and the NIWA Invertebrate Collection, are gratefully acknowledged for providing the Goniocorella dumosa colony NIWA-148046.The specimen was collected by NIWA as part of their research project “Resilience Of deep-sea Benthic fauna to the Effects of Sedimentation” (ROBES) funded by the New Zealand Ministry of Business, Innovation and Employment (MBIE).Furthermore, we thank Leiss Abdal Al for helping with the segmentation of the SaM-ID-43148 specimen.The work in this paper is part of the project “CTcoral – CyberTaxonomic Classification and Morphological Characterisation of Cold-Water Corals” funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation; Ba5042/6-1; Ti706/6-1) as project 490665120.

References

• [1]Katerina Achilleos and AbigailM. Smith.Proof of concept for measuring growth of shelf marine calcifiers:‘a bryozoan odyssey’.Marine and Freshwater Research, 74(14):1262–1273, 2023.
• [2]ClaireW. Armstrong, NaomiS. Foley, Viktoria Kahui, and Anthony Grehan.Cold water coral reef management from an ecosystem serviceperspective.Marine Policy, 50:126–134, 2014.
• [3]Daniel Baum and Jürgen tit*chack.Cavity and Pore Segmentation in 3D Images with Ambient Occlusion.In Enrico Bertini, Niklas Elmqvist, and Thomas Wischgoll, editors,EuroVis 2016 - Short Papers. The Eurographics Association, 2016.
• [4]Daniel Baum, JamesC Weaver, Igor Zlotnikov, David Knötel, Lara Tomholt,and MasonN Dean.High-throughput segmentation of tiled biological structures usingrandom-walk distance transforms.Integrative and Comparative Biology, 59(6):1700–1712, 2019.
• [5]Serge Beucher and Christian Lantuéjoul.Use of watersheds in contour detection.volume 132, 01 1979.
• [6]L.Beuck and A.Freiwald.Bioerosion patterns in a deep-water Lophelia pertusa(Sleractinia) thicket (Propeller Mound, northern Porcupine Seabight),volume1 of Erlangen Earth Conference Series, pages 915–936.Springer, Berlin, 2005.
• [7]Lydia Beuck, Agostina Vertino, Elizaveta Stepina, Marek Karolczak, and OlafPfannkuche.Skeletal response of lophelia pertusa (scleractinia) to bioerodingsponge infestation visualised with micro-computed tomography.Facies, 53:157–176, 2007.
• [8]Bokeh Development Team.Bokeh: Python library for interactive visualization, 2018.
• [9]Sandra Brooke and CraigM. Young.In situ measurement of survival and growth of lophelia pertusa in thenorthern gulf of mexico.Marine Ecology Progress Series, 397:153–161, 2009.
• [10]Andrea Cabrito, Silvia deJuan, Hilmar Hinz, and Francesc Maynou.Morphological insights into the three-dimensional complexity ofrhodolith beds.Marine Biology, 171(6):127, 2024.
• [11]L.Chapron, N.LeBris, X.Durrieude Madron, E.Peru, P.E. Galand, andF.Lartaud.Long term monitoring of cold-water coral growth shows response toepisodic meteorological events in the nw mediterranean.Deep Sea Research Part I: Oceanographic Research Papers,160:103255, 2020.
• [12]JunYoung Choi, Sang-Eun Lee, YeIn Lee, Eunji Cho, Sunghoe Chang, and Won-KiJeong.Dxplorer: A unified visualization framework for interactive dendriticspine analysis using 3d morphological features.IEEE Transactions on Visualization and Computer Graphics,29(2):1424–1437, 2023.
• [13]ErikE. Cordes, Amanda W.J. Demopoulos, AndrewJ. Davies, Ryan Gasbarro,AlexandriaC. Rhoads, Elizabeth Lobecker, Derek Sowers, JasonD. Chaytor,CherylL. Morrison, AlexisM. Weinnig, Sandra Brooke, JayJ. Lunden, FuruMienis, SamanthaB. Joye, AndreaM. Quattrini, TraceyT. Sutton, CatherineS.McFadden, JillR. Bourque, JenniferP. McClain-Counts, BrianD. Andrews,MelissaJ. Betters, PeterJ. Etnoyer, GaryA. Wolff, BernieB. Bernard,JamesM. Brooks, MichaelK. Rasser, and Caitlin Adams.Expanding our view of the cold-water coral niche and accounting ofthe ecosystem services of the reef habitat.Scientific Reports, 13(1):19482, 2023.
• [14]AnnaK. Cresswell, DamianP. Thomson, Michael D.E. Haywood, and MichaelRenton.Frequent hydrodynamic disturbances decrease the morphologicaldiversity and structural complexity of 3d simulated coral communities.Coral Reefs, 39(4):1147–1161, 2020.
• [15]Vincent Dercksen, Hans-Christian Hege, and Marcel Oberlaender.The filament editor: An interactive software environment forvisualization, proof-editing and analysis of 3d neuron morphology.Neuroinformatics, 12, 12 2013.
• [16]Mélanie Douarin, DanielJ. Sinclair, Mary Elliot, Lea-Anne Henry, David Long,Freya Mitchison, and J.Murray Roberts.Changes in fossil assemblage in sediment cores from mingulay reefcomplex (ne atlantic): Implications for coral reef build-up.Deep-Sea Research Part II: Topical Studies in Oceanography,99:286–296, 2014.
• [17]John Ellson, Emden Gansner, Lefteris Koutsofios, StephenC. North, and GordonWoodhull.Graphviz— open source graph drawing tools.In Petra Mutzel, Michael Jünger, and Sebastian Leipert, editors,Graph Drawing, pages 483–484, Berlin, Heidelberg, 2002. SpringerBerlin Heidelberg.
• [18]C.Fouard, G.Malandain, S.Prohaska, and M.Westerhoff.Blockwise processing applied to brain microvascular network study.IEEE Transactions on Medical Imaging, 25(10):1319–1328, 2006.
• [19]AFreiwald and RHenrich.The sula reef project. unpublished cruise report of r/v victor hensencruise vh94-95.Report, 1995.
• [20]A.Freiwald, R.Henrich, and J.Pätzold.Anatomy of a deep-water coral reef mound from Stjernsund,West-Finnmark, northern Norway, volume56 of SEPM Special Publication,pages 141–161.SEPM (Society for Sedimentary Geology), Tulsa, 1997.
• [21]ViniciusJ. Giglio, AnaideW. Aued, Cesar A. M.M. Cordeiro, Linda Eggertsen,Débora S.Ferrari, LeandraR. Gonçalves, Natalia Hanazaki, OsmarJ. Luiz,AndréL. Luza, ThiagoC. Mendes, HudsonT. Pinheiro, Bárbara Segal,LuizaS. Waechter, and MarianaG. Bender.A global systematic literature review of ecosystem services in reefenvironments.Environmental Management, 73(3):634–645, 2024.
• [22]Mathieu Gonin, Isai Salas-González, David Gopaulchan, JuanP. Frene, StijnRoden, BramVan dePoel, DavidE. Salt, and Gabriel Castrillo.Plant microbiota controls an alternative root branching regulatorymechanism in plants.Proceedings of the National Academy of Sciences,120(15):e2301054120, 2023.
• [23]L.Gorelick, M.Galun, E.Sharon, R.Basri, and A.Brandt.Shape representation and classification using the poisson equation.In Proceedings of the 2004 IEEE Computer Society Conference onComputer Vision and Pattern Recognition, 2004. CVPR 2004., volume2, pagesII–II, 2004.
• [24]Dirk Granse, Jürgen tit*chack, Malika Ainouche, Kai Jensen, and KetilKoop-Jakobsen.Subsurface aeration of tidal wetland soils: Root-system structure andaerenchyma connectivity in spartina (poaceae).Science of The Total Environment, 802:149771, 2022.
• [25]Siria Gámez and NyeemaC. Harris.Conceptualizing the 3d niche and vertical space use.Trends in Ecology & Evolution, 37(11):953–962, 2022.doi: 10.1016/j.tree.2022.06.012.
• [26]D.Hebbeln.Report and preliminary results of rv pelagia cruise 64pe284.cold-water corals in the gulf of cádiz and on coral patch seamount (neatlantic). portimão - portimão, 18.02. - 09.03.2008, 2008.
• [27]EmilyRebecca Lines, FabianJörg Fischer, Harry JonFoord Owen, and TommasoJucker.The shape of trees: Reimagining forest ecology in three dimensionswith remote sensing.Journal of Ecology, 110(8):1730–1745, 2022.
• [28]Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger.Umap: Uniform manifold approximation and projection.Journal of Open Source Software, 3(29):861, 2018.
• [29]ChristinaE. Mueller, Tomas Lundälv, JackJ. Middelburg, and Dick van Oevelen.The symbiosis between lophelia pertusa and eunice norvegicastimulates coral calcification and worm assimilation.PLOS ONE, 8:e58660, 2013.
• [30]Covadonga Orejas and Carlos Jiménez.The Builders of the Oceans – Part I: Coral Architecture fromthe Tropics to the Poles, from the Shallow to the Deep, pages 627–655.Springer International Publishing, Cham, 2017.
• [31]Karran Pandey, Talha BinMasood, Saurabh Singh, Ingrid Hotz, Vijay Natarajan,and Tejas Murthy.Morse theory-based segmentation and fabric quantification of granularmaterials.Granular Matter, 24, 02 2022.
• [32]DanielR. Pygas, Renata Ferrari, and WillF. Figueira.Review and meta-analysis of the importance of remotely sensed habitatstructural complexity in marine ecology.Estuarine, Coastal and Shelf Science, 235:106468, 2020.
• [33]J.M. Roberts, A.J. Wheeler, A.Freiwald, and S.Cairns.Cold-Water Corals: The Biology and Geology of Deep-Sea CoralHabitats.Cambridge University Press, Cambridge, 2009.
• [34]Olaf Ronneberger, Philipp Fischer, and Thomas Brox.U-net: Convolutional networks for biomedical image segmentation.In Nassir Navab, Joachim Hornegger, WilliamM. Wells, andAlejandroF. Frangi, editors, Medical Image Computing andComputer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015.Springer International Publishing.
• [35]Anita Roth-Nebelsick, Tatiana Miranda, Martin Ebner, Wilfried Konrad, andChristopher Traiser.From tree to architecture: how functional morphology of arborescenceconnects plant biology, evolution and physics.Palaeobiodiversity and Palaeoenvironments, 101:267–284, 2021.
• [36]Giovanni Sanna, JaninaV. Büscher, and André Freiwald.Cold-water coral framework architecture is selectively shaped bybottom current flow.Coral Reefs, 2023.
• [37]Giovanni Sanna and André Freiwald.Deciphering the composite morphological diversity of Lopheliapertusa, a cosmopolitan deep-water ecosystem engineer.Ecosphere, 12(11):e03802, 2021.
• [38]Mie Sato, Ingmar Bitter, Michael Bender, Arie Kaufman, and Masayuki Nakajima.Teasar: Tree-structure extraction algorithm for accurate and robustskeletons.volume54, pages 281 – 449, 02 2000.
• [39]Kira Schmitt, Jürgen tit*chack, and Daniel Baum.Polyp-Cavity Segmentation of Cold-Water Corals guided by AmbientOcclusion and Ambient Curvature.In RenataG. Raidou, Björn Sommer, TorstenW. Kuhlen, Michael Krone,Thomas Schultz, and Hsiang-Yun Wu, editors, Eurographics Workshop onVisual Computing for Biology and Medicine. The Eurographics Association,2022.
• [40]DonaldF. Squires.Scleractinian corals from the norfolk island cable.Records of the Auckland Institute and Museum, 5(3/4):195–201,1960.
• [41]Detlev Stalling, Malte Westerhoff, and Hans-Christian Hege.Amira: a highly interactive system for visual data analysis.In The Visualization Handbook, pages 749–767. Elsevier, 2005.
• [42]FitalewAgimass Taye, MajaVinde Folkersen, ChristopherM. Fleming, AndrewBuckwell, Brendan Mackey, K.C. Diwakar, Dung Le, Syezlin Hasan, andChantalSaint Ange.The economic values of global forest ecosystem services: Ameta-analysis.Ecological Economics, 189:107145, 2021.
• [43]J.tit*chack, D.Baum, R.DePolHolz, M.LópezCorrea, N.Forster,S.Flögel, D.Hebbeln, and AFreiwald.Aggradation and carbonate accumulation of holocene norwegiancold-water coral reefs.Sedimentology, 62:1873–1898, 2015.
• [44]Jürgen tit*chack, Daniel Baum, Kei Matsuyama, Karin Boos, ClaudiaFärber, W-A Kahl, Karsten Ehrig, Dietmar Meinel, Carmen Soriano, andStuartR Stock.Ambient occlusion–a powerful algorithm to segment shell and skeletalintrapores in computed tomography data.Computers & Geosciences, 115:75–87, 2018.
• [45]Jürgen tit*chack, HiskeG. Fink, Daniel Baum, Claudia Wienberg, Dierk Hebbeln,and André Freiwald.Mediterranean cold-water corals – an important regional carbonatefactory?The Depositional Record, 2(1):74–96, 2016.
• [46]J.B. van Hulzen, J.van Soelen, and T.J. Bouma.Morphological variation and habitat modification are stronglycorrelated for the autogenic ecosystem engineerspartina anglica (commoncordgrass).Estuaries and Coasts, 30(1):3–11, 2007.
• [47]GrahamJ. Wills.Nicheworks — interactive visualization of very large graphs.In Giuseppe DiBattista, editor, Graph Drawing, pages 403–414,Berlin, Heidelberg, 1997. Springer Berlin Heidelberg.
• [48]Muhan Zhang and Yixin Chen.Link prediction based on graph neural networks.Advances in neural information processing systems, 31, 2018.

Supplementary Materials