Contributing and extending ZarrSmith

Package layout

Package	What it contains
zarrsmith.microscope_manager	microscope managers: one reader class per acquisition system, each exposing one image (MicroscopeManager subclasses), plus ZarrManager for reading, editing and correcting OME-Zarr stores
zarrsmith.mosaic_manager	mosaic managers: readers for acquisitions saved as separate positioned tiles (MosaicManager subclasses), the Tile model and the fusion engine (fuse_tiles, plan_fusion)
zarrsmith.utils	helpers shared by both: metadata, axes, units, colours, pyramids, OME-Zarr writing, chromatic shift correction (see Layout of zarrsmith.utils)
zarrsmith.cli	the zarrsmith convert, zarrsmith fuse and zarrsmith chromatic-shift commands
zarrsmith.napari	the Converter and Mosaic Fusion widgets

Two kinds of readers

ZarrSmith has two extension points, and a new acquisition system needs one or both:

• A microscope manager reads a dataset that already is one image (one array per pyramid level, possibly with several timepoints and channels). It is what zarrsmith convert and the Converter widget use.

• A mosaic manager reads an acquisition saved as many tiles at different stage positions. It only becomes an image once the tiles are fused, which zarrsmith fuse and the Mosaic Fusion widget do. The fused result is an ordinary OME-Zarr image, read back with ZarrManager.

Some systems need both: Luxendo data fused by the vendor software is read by LuxendoManager, the raw tiles by LuxendoMosaicManager (likewise OperaManager and OperaMosaicManager).

Adding a microscope manager

Implement the reader

Subclass MicroscopeManager (exported from zarrsmith.microscope_manager) and implement two hooks. The base class provides read(), which calls them and stores the results in self.metadata and self.data:

from zarrsmith.microscope_manager import MicroscopeManager
from zarrsmith.utils.metadata import ChannelInfo, build_metadata

class MyManager(MicroscopeManager):
    DEFAULT_CHUNKS = (1, 1, 8, 2048, 2048)  # used when chunks=None

    def __init__(self, path, chunks=None):
        super().__init__(path, chunks)  # sets self.path (Path) and self.chunks
        self.read()

    def _parse_metadata(self) -> dict:
        # Parse the vendor metadata and return the normalized schema.
        return build_metadata(
            size=[(t, c, z, y, x)],           # one shape per pyramid level
            scales=[(sz, sy, sx)],            # one voxel size per pyramid level (z, y, x)
            units=("micrometer",) * 3,
            channels=[ChannelInfo("DAPI", "0000FF"), ChannelInfo("GFP", "#00ff00")],
            dtype="uint16",
            time_increment=60.0,
            time_increment_unit="s",
        )

    def _build_dask_array(self) -> list:
        # Return one lazy dask array per pyramid level, in TCZYX order.
        ...

build_metadata() normalizes the units and colours for you, and zarrsmith.utils has helpers that the built-in readers share: parse_channel_color (vendor colour formats to RRGGBB), normalize_unit / to_micrometers, to_tczyx (reorder or add axes), parse_ome_xml (OME-XML <Pixels> and channels) and scale_for_level.

The metadata must follow the ZarrSmith schema used across the repository:

{
  "size": [... per pyramid level ...],
  "chunksize": [... per pyramid level ...],
  "scales": [... per pyramid level ...],
  "units": (...),
  "axes": "tczyx",
  "channel_names": [...],
  "channel_colors": [...],
  "time_increment": ...,
  "time_increment_unit": ...,
  "dtype": ...,
}

If a manager needs the data before it can describe it (like ArrayManager) or reads several datasets at once (like ZarrManager), it can override read() instead; it should still return None and fill self.data and self.metadata. File handles appended to self._open_files are closed by close() and when the manager is used as a context manager.

Once that contract is respected, the new manager gets the common ZarrSmith tooling: build_pyramid(), to_zarr(), visualize(), reorder_channels(), update_metadata() and subset_dataset().

Register it

1. Export the class from zarrsmith/microscope_manager/__init__.py.

2. Command line: add a key to _manager_name_map() in zarrsmith/cli/zarrsmith.py and to the --microscope choices in _add_convert_microscope() (zarrsmith/cli/__arguments.py). If the format can be recognised from the path, add a rule to _resolve_zarr_manager() so -m can be left out.

3. Napari: add the key to dataset_reader() and to the file_format choices of the Converter widget, both in zarrsmith/napari/_convert_widget.py.

4. Documentation: list it in Readers and conventions.

Test it

A reader can be tested without real data by writing a small synthetic dataset in the vendor format to tmp_path, or by subclassing with hooks that return in-memory arrays (see tests/test_manager_refactor.py). Check at least the shape, axes, scales, channel names and a round trip through to_zarr() and ZarrManager.

Adding a mosaic manager

Implement the reader

Subclass MosaicManager (exported from zarrsmith.mosaic_manager) and implement a single hook, _scan(), which returns one Tile per acquired tile. The base constructor calls it and stops with an error when it returns no tile. A mosaic with several timepoints returns the tiles of all of them.

from zarrsmith.mosaic_manager import H5Array, MosaicManager, Tile


class MyMosaicManager(MosaicManager):
    def _scan(self):
        return [
            Tile(
                data=H5Array(path, "Data"),        # any lazy array with shape/dtype/slicing
                origin=(z0, y0, x0),               # position of voxel [0, 0, 0]
                voxel_size=(dz, dy, dx),
                channel="GFP",
                group=None,                        # e.g. illumination side, for blend="sigmoid"/"midline"
            )
            for path, (z0, y0, x0) in ...
        ]

A Tile describes one stack and where it sits in the sample:

Field	Meaning
data	lazy (z, y, x) array: anything with shape, dtype and numpy-style slicing (h5py.Dataset, dask, numpy, H5Array). Only read when a fusion needs it
origin, voxel_size	(z, y, x) position of the first voxel and voxel spacing, in units. Placement is translation only, and all tiles of a mosaic share units and voxel size
channel	dye name; tiles with the same name are fused into one output channel
group	optional label of tiles that are fused among themselves first, such as the L/R illumination side; blend="sigmoid" and blend="midline" then join the groups across their overlap
timepoint	timepoint index (0 for a single timepoint)
attrs	free-form reader metadata (stack name, grid position, …); stack also matches the tiles of a position across timepoints

Things to get right:

• Keep loading light. _scan() should read metadata only, never pixel data. Pass shape and dtype to H5Array when the metadata knows them, so no file has to be opened: this matters on network drives (LuxendoMosaicManager reads its JSON files in parallel and a single .h5 header; OperaMosaicManager reads its index in one pass and a single TIFF header).

• Make tile data picklable for parallel fusion. fuse_tiles(workers > 1) reads tiles in worker processes when the data allows it. H5Array opens the file on every read, holds no handle and sets picklable = True; use it, or the same pattern, for file-backed tiles. Other data falls back to threads.

• Close what you open. Append open file handles to self._open_files; close() and the context manager close them.

• Name tiles stably. tile_id() (<channel>/<stack>) is used by skip, the CSV batch tiles column and the napari tile map. Set attrs["stack"] to the vendor’s stack name, the same at every timepoint: fusion uses it to give all timepoints the geometry of the reference timepoint. Without it the id falls back to the tile’s index, which changes with the scan order, and tiles are matched across timepoints by their order, which breaks when a timepoint misses a tile.

visualize(), fuse(), plan_fusion(), zarrsmith fuse and the napari widget then work unchanged.

Register it

1. Export the class from zarrsmith/mosaic_manager/__init__.py.

2. Add it to MOSAIC_READERS in zarrsmith/mosaic_manager/readers.py: the command line and the napari widget take their readers from there. If the folder layout is recognisable, add a rule to detect_microscope() there too: zarrsmith fuse then picks the reader without --microscope, and every reader refuses a folder recognised as another microscope’s (WrongReaderError, raised by MosaicManager.__init__ before _scan()).

3. Command line: add the key to the --microscope choices in _add_fuse_microscope() (zarrsmith/cli/__arguments.py).

4. Documentation: add it to the reader table in Mosaics.

Reader options other than the path (such as flatfield) are passed by the command line and the napari widget only to readers whose constructor accepts them (reader_options()).

For data stored as one file per plane, TiffStackArray (used by OperaMosaicManager) is the counterpart of H5Array, and CziTileArray (used by ZeissMosaicManager) for planes of one CZI file: both build on PlaneStackArray, which you can subclass for other formats. The optional transform of TiffStackArray corrects every plane when it is read (e.g. a flat-field).

Test it

Fusion can be tested with in-memory tiles: a MosaicManager subclass whose _scan() returns Tiles holding small numpy arrays at known origins (see tests/test_mosaic_fusion.py). For the reader itself, write a few tiny tiles in the vendor format to tmp_path and check positions, channels, groups and tile_id()s (see tests/test_luxendo_mosaic.py).

Layout of zarrsmith.utils

Shared helpers live in one place each, so readers, writers and the napari viewer behave the same way:

Module	What it owns
colors	parse_channel_color / parse_color / ome_color_to_hex / hex_to_rgb and the default channel palette
units	normalize_unit, to_micrometers
axes	axis validation, to_tczyx, index selections
chunks	shape_tuple, get_chunks, rechunk_to_shape
downsampling	factor math and downsample_nearest (NumPy or Dask)
metadata, ome	build_metadata, ChannelInfo, scale_for_level, parse_ome_xml
dataset_ops	validation and in-place edits shared by MicroscopeManager and ZarrManager (subset_levels, reorder_channels, apply_metadata_updates)
ngff	ZarrWriteConfig, array creation (_array_create_kwargs), sharding, multiscales metadata (_build_multiscales) and the pyramid writers
chromatic_shift	Z chromatic shift estimation (estimate_chromatic_shift) and the writer of corrected stores that copies the input’s array layout (write_chromatic_shift_corrected); see Chromatic shift correction

Documentation

ZarrSmith uses Sphinx + MyST + AutoAPI, and the documentation/ folder is the Sphinx source:

• the root README.md is the landing page. documentation/index.md pulls it in with a MyST include, so it is never copied;

• every other page is a Markdown file in documentation/, readable on GitHub and rendered on the documentation site;

• API pages are generated from the Python docstrings, so documenting classes, methods and helpers in the source code is the best way to improve the API reference.

To add a page, create documentation/<page>.md, add it to the toctree in documentation/index.md and link it from the README table. Build the site locally with:

pip install -e .[docs]
sphinx-build documentation _build

The API entry points to document and keep stable are:

• MicroscopeManager, the vendor reader classes and ZarrManager in zarrsmith.microscope_manager

• MosaicManager, Tile, H5Array, fuse_tiles and plan_fusion in zarrsmith.mosaic_manager

• the zarr-writing helpers in zarrsmith.utils

• the CLI entry points in zarrsmith.cli

• the napari widgets in zarrsmith.napari