Dataset

class morphonet.data.dataset.AdvancedParameters(seg_downscale: int, seg_z_downscale: int, raw_downscale: float, raw_z_downscale: float, smoothing: bool, smoothing_passband: float, smoothing_iterations: int, quadratic_clustering: bool, quadratic_clustering_divisions: int, decimation: bool, decimation_reduction: float, automatic_decimation_threshold: int)

Bases: object

Data structure of advanced parameters in a dataset. Used to keep visualization parameters, and access it using dataset.advanced_parameters.downscale

class morphonet.data.dataset.DatasetAnalysis(name, min_time, max_time, background_value, segmentation_images, raw_images, property_file, creation_date, temp_path, downscale=4, z_downscale=4, raw_downscale=4, raw_z_downscale=4, smoothing=True, smoothing_passband=0.01, smoothing_iterations=25, quadratic_clustering=True, quadratic_clustering_divisions=1, decimation=True, decimation_reduction=0.8, automatic_decimation_threshold=30)

Bases: object

The dataset is the class that represent a MorphoNet local dataset in temporary files. Loading the dataset in memory can be really long

compute_property(property_name, step=-1, ch=0, save=True)

Compute the region property for a given step, for all time points. The computed region properties will be saved to temporary folder in MorphoNet.

Parameters:

  • property_name (str) – Name of the property

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

  • save (bool) – should save the property to file? (optional, default is True)

Returns:

Property computed

Return type:

DataProperty

compute_property_all_steps(property_name, ch=0, save=True)

Compute the region property for all available steps, for all time points. The computed region properties will be saved to temporary folder in MorphoNet.

Parameters:

  • property_name (str) – Name of the property

  • ch (int) – channel index for the property computation

  • save (bool) – should save the property to file? (optional, default is True)

Returns:

Property computed

Return type:

DataProperty

compute_property_at(property_name, time, step=-1, ch=0, save=True)

Compute the region property for a given step, at given time point. The computed region properties will be saved to temporary folder in MorphoNet.

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

  • save (bool) – should save the property to file? (optional, default is True)

Returns:

segmentation image array corresponding to time point and channel, for step

Return type:

DataProperty

compute_property_at_time_step(property_name, step, time, ch=0, save=True)

Compute the property asked for segmentation at time and step. Reads a potential existing properties file, load them, compute the info and add it to the file

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

  • save (bool) – should save the property to file? (optional, default is True)

Returns:

New property created

Return type:

DataProperty

delete_property(property_name, step=-1, ch=0)

Delete the region property for a given step, for all time points.

Parameters:

  • property_name (str) – Name of the property

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

delete_property_all_steps(property_name, ch=0)

Delete the region property at all steps, for all time points.

Parameters:

  • property_name (str) – Name of the property

  • ch (int) – channel index for the property computation

delete_property_at(property_name, time, step=-1, ch=0)

Delete the region property for a given step, at given time point.

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

delete_property_at_time_step(property_name, step, time, ch=0)

Delete the property asked for segmentation at time and step. Reads a potential existing properties file, and remvoe the property

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

  • ch (int) – channel index for the property computation

export_dataset_at_step(channel, image_name_template, segmentation_folder_output, step=-1)

Export all segmentations of the dataset at step (if -1, last step found), into a folder with a specific format

Parameters:

  • channel (int) – Channel in the segmentation

  • image_name_template (str) – Template image name (with time point formated as %03d or {:03d}

  • segmentation_folder_output (str) – Folder where the segmentation images will be exported

  • step (int) – Step to export the dataset from

export_image(time, channel, image_path, step=-1)

Extract the segmentation image found at given step , channel and time, and save it to image path

Parameters:

  • channel (int) – Channel in the segmentation

  • time (int) – Time in the segmentation

  • image_path (str) – Path where the image should be stored

  • step (int) – Step to export the dataset from

export_properties_as_txt(output_folder, include_scikit=True, step=-1)

Export all properties from the dataset as text files

Parameters:

  • output_folder (str) – Folder where property texts will be generated

  • include_scikit (bool) – Whether to include scikit-image or not

  • step (int) – Step to export the dataset from, -1 for last step

export_properties_as_xml(output_file, step=-1)

Export all properties from the dataset as a XML file

Parameters:

  • output_file (str) – Property file that will be generated

  • step (int) – Step to export the dataset from, -1 for last step

find_properties_path_at_step(current_step, scikit_time=-1, include_scikit=True)

Find all properties file path for property in current_step, can include scikit properties or not. For a scikit property, can be filtered on time point using “t” parameter If not found, return empty list.

Parameters:

  • current_step (int) – step to search in

  • scikit_time (int) – time point if laoding a scikit property

  • include_scikit (bool) – should include scikit properties or not

Returns:

list of paths to properties file

Return type:

list

find_properties_path_at_step_no_name(current_step, scikit_time=-1, ch=-1)

Returns the path to the last updated property file for a given name, for a given step. (search will be done from given step to 0 returning the first found) Time can be added, if time is not specified, returns the last step found for the curation name. If specified, will find the last step for a scikit property at given time

Parameters:

  • name (str) – property name

  • scikit_time (int) – time point, for scikit properties

  • ch (int) – channel for scikit properties

  • scikit_only (bool) – if True will only find the last step for a scikit property

Current_step:

Current step

Returns:

The path to property file

Return type:

str

find_property_path_at_step(name, current_step, scikit_time=-1, ch=-1, scikit_only=False)

Returns the path to the last updated property file for a given name, for a given step. (search will be done from given step to 0 returning the first found) Time can be added, if time is not specified, returns the last step found for the curation name. If specified, will find the last step for a scikit property at given time

Parameters:

  • name (str) – property name

  • scikit_time (int) – time point, for scikit properties

  • ch (int) – channel for scikit properties

  • scikit_only (bool) – if True will only find the last step for a scikit property

Current_step:

Current step

Returns:

The path to property file

Return type:

str

find_segmentation_path_at_step(current_step, t, ch=0)

Returns the path to the last updated segmentation temporary file for a given time , channel (search will be done from given step to 0 , returning the first found)

Parameters:

  • t (int) – Time point

  • ch (int) – Channel

  • current_step (int) – Current step

Returns:

Path to the last updated segmentation temporary file

Return type:

str

get_action_at(step=-1)

Read the content of the action file for the step given in parameter. If step is -1, returns the action for last step

Parameters:

step (int) – number of the step of curation, if -1 , use the last step

Returns:

action file content or None

Return type:

str

get_all_properties(step=-1, scikit_time=-1, include_scikit=True)

Returns all properties found at step parameter, including all steps before. If step is -1, returns all properties at last step. Can include scikit properties or not. For a scikit property, can be filtered on time point using “t” parameter If not found, return empty list.

Parameters:

  • step (int) – step to search. If -1 or not specified, search for last step

  • scikit_time (int) – time point if laoding a scikit property

  • include_scikit (bool) – should include scikit properties or not

Returns:

all properties found at step parameter

Return type:

dict

get_cells_updated_at(step=-1)

Read the content of the action file for the step given in parameter, and returns the cell updated. If step is -1, returns the action cell list for last step

Parameters:

step (int) – Step of curation, if -1, use the last step

Returns:

list of cell ids content or None

Return type:

list of str

get_cells_updated_by_selection_at(step=-1)

Read the content of the action file for the step given in parameter, and returns the cell updated, grouping them by selection. If step is -1, returns the action cell list for last step

Parameters:

step (int) – number of the step of curation, if -1 , use the last step

Returns:

dict of cell ids content by selection or None

Return type:

dict

get_intensity_image(t, channel, search_for_compressed=False, load_array=True)

Filter the intensitiy images list to retrieve the intensity image corresponding to time point and channel

Parameters:

  • t (int) – Time point

  • channel (int) – channel

  • search_for_compressed (bool) – Whether to search for compressed intensity images instead of normal

  • load_array (bool) – Whether to load the intensity images numpy array

Returns:

intensity image corresponding to time point and channel

Return type:

Image

get_max_time()

Returns the dataset max time point

Returns:

max time point

Return type:

int

get_min_time()

Returns the dataset min time point

Returns:

min time point

Return type:

int

get_number_of_curations()

Read the temp folders of the dataset, and count the number of steps

Returns:

Number of curation steps

Return type:

int

get_plugin_name_at(step=-1)

Read the plugin from the action file for the step given in parameter. If step is -1, returns the action for last step

Parameters:

step (int) – number of the step of curation, if -1 , use the last step

Returns:

plugin found in action file content or None

Return type:

str

get_plugin_parameters_at(step=-1)

Read the plugin from the action file for the step given in parameter. If step is -1, returns the action for last step

Parameters:

step (int) – number of the step of curation, if -1 , use the last step

Returns:

plugin found in action file content or None

Return type:

str

get_property(property_name, step=-1)

Return the property object at given step. If step is -1, find in last step The time point parameter is only used loading a scikit properties

Parameters:

  • property_name (str) – Name of the property

  • step (int) – step to find the property version, -1 for last step

Returns:

Property found

Return type:

DataProperty

get_property_at(property_name, time, step=-1)

Return the property find by name, corresponding to time point, and step if step is -1, the function finds the image in. Time point is only used for scikit properties

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

Returns:

segmentation image array corresponding to time point and channel, for step

Return type:

DataProperty

get_property_at_time_step(property_name, step, time)

Return the property find by name, corresponding to time point, and step. Time point is only used for scikit properties

Parameters:

  • property_name (str) – Name of the property

  • time (int) – time point

  • step (int) – step to find the property version, -1 for last step

Returns:

segmentation image array corresponding to time point and channel, for step

Return type:

DataProperty

get_property_last_step(name, scikit_time=-1)

Retrieve the last step containing the property given by name. Time can be added, if time is not specified, returns the last step found for the curation name. If specified, will find the last step for a scikit property at given time

Parameters:

  • name (str) – property name

  • scikit_time (int) – time point, for scikit properties

Returns:

the step value

Return type:

int

get_scikit_property_last_step(name, time=-1)

Retrieve the last step containing the property given by name. Time can be added, if time is not specified, returns the last step found for the curation name. If specified, will find the last step for a scikit property at given time

Parameters:

  • name (str) – property name

  • scikit_time (int) – time point, for scikit properties

Returns:

the step value

Return type:

int

get_segmentation(time, step=-1, channel=0, load_array=True)

Return the segmentation specified object by time,channel at a given step (if -1, use the last step found). If load_array is set to False, the image content will not be loaded into the segmentation object

Parameters:

  • time (int) – time point

  • step (int) – step ( if -1, use the last step found)

  • channel (int) – channel

  • load_array (bool) – Should the function load the image content into the segmentation object ? True by default

Returns:

segmentation object

Return type:

Segmentation

get_segmentation_at_step(time, step, channel=0, load_array=True)

Return the segmentation specified object by time,channel at a specific given step. If load_array is set to False, the image content will not be loaded into the segmentation object

Parameters:

  • time (int) – time point

  • step (int) – step

  • channel (int) – channel

  • load_array (bool) – Should the function load the image content into the segmentation object ? True by default

Returns:

segmentation object

Return type:

Segmentation

get_segmentation_last_step(t, ch=0)

Returns the last step where a segmentation (found by channel, time) has been updated

Parameters:

  • t (int) – Time point

  • ch (int) – Channel

Returns:

Last step

Return type:

int

get_updated_properties(step=-1, scikit_time=-1, include_scikit=True)

Returns all properties found at step parameter. If step is -1, returns all properties at last step. Can include scikit properties or not. For a scikit property, can be filtered on time point using “t” parameter If not found, return empty list.

Parameters:

  • step (int) – step to search. If -1 or not specified, search for last step

  • scikit_time (int) – time point if laoding a scikit property

  • include_scikit (bool) – should include scikit properties or not

Returns:

all properties found at step parameter

Return type:

dict

has_intensities_images_at(t, channel)

Determine if dataset has any intensity images for specific t and channel

Parameters:

  • t (int) – time point

  • channel (int) – channel

Returns:

True if dataset has any intensity images

Return type:

bool

initialize_external_step(filename)

Initialize a new step to the dataset TEMP folder, as an external step.

Parameters:

filename (str) – Name of the file that generated a step

Returns:

step identifier

Return type:

int

list_all_properties()

List all properties available in the dataset

Returns:

all properties found at step parameter

Return type:

dict

print_all_intensity_images(channel=0)

Print all segmentation images found in the dataset for a given channel, and print their path to file

Parameters:

channel (int) – channel

print_all_segmentations(channel=0)

Print all segmentation images found in the dataset for a given channel, and print their path to file

Parameters:

channel (int) – channel

print_computable_properties()

Print all properties that can be computed using compute_property commands

write_property_to_step(step, name_file, property_object)

Save a given dictionary to a morphonet text format property. Giving a type to the information is important

Parameters:

  • step (int) – Step to write in, it has to exist

  • name_file (str) – Name of the information and file

  • property_object (MorphNetDataProperty) – MorphoNet Data property object to save

write_segmentation_to_step(step, segmentation_array, time, channel, voxel_size=(1, 1, 1))

Save a given segmentation to the step folder, and update step action file

Parameters:

  • step (int) – The step to write in

  • segmentation_array (ndarray) – Array of the segmentation image

  • time (int) – Time point of the segmentation

  • channel (int) – Channel of the segmentation

  • voxel_size (tuple) – Voxel size metadata of the segmentation , optional , default = (1,1,1)

class morphonet.data.dataset.DatasetStep(dataset, step, step_folder, action_file)

Bases: object

The dataset step is the snapshot of the dataset at a specific round of curation Values that are specific to a step : a list of property and a list of segmentation images, the action applied at this step

add_property(name, path, property_type)

Add a property to the step, only if not found. If found, return it

Parameters:

  • name (str) – Property name

  • path (str) – Path of the property in the temp folder

  • property_type (str) – Type of the property

Returns:

The created or found property

Return type:

DataProperty

add_segmentation(segmentation_image)

Add a segmentation image to the dataset step, if not found (search by time + channel)

Parameters:

segmentation_image (Image) –

clean_property_at(time, property_name)

Remove all values for a specific time point from a property

Parameters:

  • time (int) – Time point

  • property_name (str) – Property name

contains_property_at(time, property_name)

Test if the current step contains values for a property by name for a given time

Parameters:

  • time (int) – Time point

  • property_name (str) – Property name

Returns:

if contains the property at time given

Return type:

bool

contains_segmentation_image_at(time, channel)

Test if the step contains specific segmentation for time point and channel

Parameters:

  • time (int) – Time point to find

  • channel (int) – Channel to find

Returns:

True if step contains a specific segmentation image for time point and channel

Return type:

bool

get_action()

Read the action file of the step, and returns its content (an action file path is given when creating the step)

Returns:

Content of the action file (the step action)

Return type:

str

get_plugin_name()

Return the name of the plugin used during the step, found in the action file

Returns:

Name of the plugin

Return type:

str

get_plugin_parameters()

Return the parameters of the plugin used during the step, found in the action file

Returns:

Parameters of the plugin

Return type:

str

get_property(name)

Retrieve a property object by name

Parameters:

name (str) – Property name

Returns:

Property object or None

Return type:

DataProperty

get_segmentation_at(time, channel)

Return the image at time point and channel given in parameter

Parameters:

  • time (int) – Time point

  • channel (int) – Channel

Returns:

Image at time point and channel

Return type:

Image

list_cells_in_action()

List the cells found that were updated at the step, found in the action file

Returns:

Cells identifier : time, identifier, channel, label

Return type:

list

list_cells_in_action_by_label()

List the cells found that were updated at the step, found in the action file , and group them by their label

Returns:

Cells identifier by label: {label: [t,id,ch, …]}

Return type:

dict

class morphonet.data.dataset.Image(channel: int, time_point: int, name: str, original_channel: int, original_time_point: int, path: str, voxel_size: list, size: list, encoding: list, image_temp_path: str, ndarray: ndarray)

Bases: object

Data structure to hold image metadata and paths for MorphoNet segmentations or intensity images