# Contributing and extending ZarrSmith

## Package layout

| Package | What it contains |
|---|---|
| `zarrsmith.microscope_manager` | **microscope managers**: one reader class per acquisition system, each exposing *one image* (`MicroscopeManager` subclasses), plus `ZarrManager` for reading, editing and correcting OME-Zarr stores |
| `zarrsmith.mosaic_manager` | **mosaic managers**: readers for acquisitions saved as separate positioned *tiles* (`MosaicManager` subclasses), the `Tile` model and the fusion engine (`fuse_tiles`, `plan_fusion`) |
| `zarrsmith.utils` | helpers shared by both: metadata, axes, units, colours, pyramids, OME-Zarr writing, chromatic shift correction (see [Layout of `zarrsmith.utils`](#layout-of-zarrsmithutils)) |
| `zarrsmith.cli` | the `zarrsmith convert`, `zarrsmith fuse` and `zarrsmith chromatic-shift` commands |
| `zarrsmith.napari` | the Converter and Mosaic Fusion widgets |

## Two kinds of readers

ZarrSmith has two extension points, and a new acquisition system needs one or both:

- A **microscope manager** reads a dataset that already is **one image** (one array per pyramid level, possibly with several timepoints and channels). It is what `zarrsmith convert` and the Converter widget use.
- A **mosaic manager** reads an acquisition saved as **many tiles** at different stage positions. It only becomes an image once the tiles are fused, which `zarrsmith fuse` and the Mosaic Fusion widget do. The fused result is an ordinary OME-Zarr image, read back with `ZarrManager`.

Some systems need both: Luxendo data fused by the vendor software is read by `LuxendoManager`, the raw tiles by `LuxendoMosaicManager` (likewise `OperaManager` and `OperaMosaicManager`).

## Adding a microscope manager

### Implement the reader

Subclass `MicroscopeManager` (exported from `zarrsmith.microscope_manager`) and implement two hooks. The base class provides `read()`, which calls them and stores the results in `self.metadata` and `self.data`:

```python
from zarrsmith.microscope_manager import MicroscopeManager
from zarrsmith.utils.metadata import ChannelInfo, build_metadata

class MyManager(MicroscopeManager):
    DEFAULT_CHUNKS = (1, 1, 8, 2048, 2048)  # used when chunks=None

    def __init__(self, path, chunks=None):
        super().__init__(path, chunks)  # sets self.path (Path) and self.chunks
        self.read()

    def _parse_metadata(self) -> dict:
        # Parse the vendor metadata and return the normalized schema.
        return build_metadata(
            size=[(t, c, z, y, x)],           # one shape per pyramid level
            scales=[(sz, sy, sx)],            # one voxel size per pyramid level (z, y, x)
            units=("micrometer",) * 3,
            channels=[ChannelInfo("DAPI", "0000FF"), ChannelInfo("GFP", "#00ff00")],
            dtype="uint16",
            time_increment=60.0,
            time_increment_unit="s",
        )

    def _build_dask_array(self) -> list:
        # Return one lazy dask array per pyramid level, in TCZYX order.
        ...
```

`build_metadata()` normalizes the units and colours for you, and `zarrsmith.utils` has helpers that the built-in readers share: `parse_channel_color` (vendor colour formats to `RRGGBB`), `normalize_unit` / `to_micrometers`, `to_tczyx` (reorder or add axes), `parse_ome_xml` (OME-XML `<Pixels>` and channels) and `scale_for_level`.

The metadata must follow the ZarrSmith schema used across the repository:

```python
{
  "size": [... per pyramid level ...],
  "chunksize": [... per pyramid level ...],
  "scales": [... per pyramid level ...],
  "units": (...),
  "axes": "tczyx",
  "channel_names": [...],
  "channel_colors": [...],
  "time_increment": ...,
  "time_increment_unit": ...,
  "dtype": ...,
}
```

If a manager needs the data before it can describe it (like `ArrayManager`) or reads several datasets at once (like `ZarrManager`), it can override `read()` instead; it should still return `None` and fill `self.data` and `self.metadata`. File handles appended to `self._open_files` are closed by `close()` and when the manager is used as a context manager.

Once that contract is respected, the new manager gets the common ZarrSmith tooling: `build_pyramid()`, `to_zarr()`, `visualize()`, `reorder_channels()`, `update_metadata()` and `subset_dataset()`.

### Register it

1. Export the class from `zarrsmith/microscope_manager/__init__.py`.
2. Command line: add a key to `_manager_name_map()` in `zarrsmith/cli/zarrsmith.py` and to the `--microscope` choices in `_add_convert_microscope()` (`zarrsmith/cli/__arguments.py`). If the format can be recognised from the path, add a rule to `_resolve_zarr_manager()` so `-m` can be left out.
3. Napari: add the key to `dataset_reader()` and to the `file_format` choices of the Converter widget, both in `zarrsmith/napari/_convert_widget.py`.
4. Documentation: list it in [Readers and conventions](readers.md).

### Test it

A reader can be tested without real data by writing a small synthetic dataset in the vendor format to `tmp_path`, or by subclassing with hooks that return in-memory arrays (see `tests/test_manager_refactor.py`). Check at least the shape, axes, scales, channel names and a round trip through `to_zarr()` and `ZarrManager`.

## Adding a mosaic manager

### Implement the reader

Subclass `MosaicManager` (exported from `zarrsmith.mosaic_manager`) and implement a single hook, `_scan()`, which returns one `Tile` per acquired tile. The base constructor calls it and stops with an error when it returns no tile. A mosaic with several timepoints returns the tiles of all of them.

```python
from zarrsmith.mosaic_manager import H5Array, MosaicManager, Tile


class MyMosaicManager(MosaicManager):
    def _scan(self):
        return [
            Tile(
                data=H5Array(path, "Data"),        # any lazy array with shape/dtype/slicing
                origin=(z0, y0, x0),               # position of voxel [0, 0, 0]
                voxel_size=(dz, dy, dx),
                channel="GFP",
                group=None,                        # e.g. illumination side, for blend="sigmoid"/"midline"
            )
            for path, (z0, y0, x0) in ...
        ]
```

A `Tile` describes one stack and where it sits in the sample:

| Field | Meaning |
|---|---|
| `data` | lazy `(z, y, x)` array: anything with `shape`, `dtype` and numpy-style slicing (`h5py.Dataset`, dask, numpy, `H5Array`). Only read when a fusion needs it |
| `origin`, `voxel_size` | `(z, y, x)` position of the first voxel and voxel spacing, in `units`. Placement is translation only, and all tiles of a mosaic share units and voxel size |
| `channel` | dye name; tiles with the same name are fused into one output channel |
| `group` | optional label of tiles that are fused among themselves first, such as the `L`/`R` illumination side; `blend="sigmoid"` and `blend="midline"` then join the groups across their overlap |
| `timepoint` | timepoint index (`0` for a single timepoint) |
| `attrs` | free-form reader metadata (stack name, grid position, ...); `stack` also matches the tiles of a position across timepoints |

Things to get right:

- **Keep loading light.** `_scan()` should read metadata only, never pixel data. Pass `shape` and `dtype` to `H5Array` when the metadata knows them, so no file has to be opened: this matters on network drives (`LuxendoMosaicManager` reads its JSON files in parallel and a single `.h5` header; `OperaMosaicManager` reads its index in one pass and a single TIFF header).
- **Make tile data picklable for parallel fusion.** `fuse_tiles(workers > 1)` reads tiles in worker processes when the data allows it. `H5Array` opens the file on every read, holds no handle and sets `picklable = True`; use it, or the same pattern, for file-backed tiles. Other data falls back to threads.
- **Close what you open.** Append open file handles to `self._open_files`; `close()` and the context manager close them.
- **Name tiles stably.** `tile_id()` (`<channel>/<stack>`) is used by `skip`, the CSV batch `tiles` column and the napari tile map. Set `attrs["stack"]` to the vendor's stack name, the same at every timepoint: fusion uses it to give all timepoints the geometry of the reference timepoint. Without it the id falls back to the tile's index, which changes with the scan order, and tiles are matched across timepoints by their order, which breaks when a timepoint misses a tile.

`visualize()`, `fuse()`, `plan_fusion()`, `zarrsmith fuse` and the napari widget then work unchanged.

### Register it

1. Export the class from `zarrsmith/mosaic_manager/__init__.py`.
2. Add it to `MOSAIC_READERS` in `zarrsmith/mosaic_manager/readers.py`: the command line and the napari widget take their readers from there. If the folder layout is recognisable, add a rule to `detect_microscope()` there too: `zarrsmith fuse` then picks the reader without `--microscope`, and every reader refuses a folder recognised as another microscope's (`WrongReaderError`, raised by `MosaicManager.__init__` before `_scan()`).
3. Command line: add the key to the `--microscope` choices in `_add_fuse_microscope()` (`zarrsmith/cli/__arguments.py`).
4. Documentation: add it to the reader table in [Mosaics](mosaic.md).

Reader options other than the path (such as `flatfield`) are passed by the command line and the napari widget only to readers whose constructor accepts them (`reader_options()`).

For data stored as one file per plane, `TiffStackArray` (used by `OperaMosaicManager`) is the counterpart of `H5Array`, and `CziTileArray` (used by `ZeissMosaicManager`) for planes of one CZI file: both build on `PlaneStackArray`, which you can subclass for other formats. The optional `transform` of `TiffStackArray` corrects every plane when it is read (e.g. a flat-field).

### Test it

Fusion can be tested with in-memory tiles: a `MosaicManager` subclass whose `_scan()` returns `Tile`s holding small numpy arrays at known origins (see `tests/test_mosaic_fusion.py`). For the reader itself, write a few tiny tiles in the vendor format to `tmp_path` and check positions, channels, groups and `tile_id()`s (see `tests/test_luxendo_mosaic.py`).

## Layout of `zarrsmith.utils`

Shared helpers live in one place each, so readers, writers and the napari viewer behave the same way:

| Module | What it owns |
| --- | --- |
| `colors` | `parse_channel_color` / `parse_color` / `ome_color_to_hex` / `hex_to_rgb` and the default channel palette |
| `units` | `normalize_unit`, `to_micrometers` |
| `axes` | axis validation, `to_tczyx`, index selections |
| `chunks` | `shape_tuple`, `get_chunks`, `rechunk_to_shape` |
| `downsampling` | factor math and `downsample_nearest` (NumPy or Dask) |
| `metadata`, `ome` | `build_metadata`, `ChannelInfo`, `scale_for_level`, `parse_ome_xml` |
| `dataset_ops` | validation and in-place edits shared by `MicroscopeManager` and `ZarrManager` (`subset_levels`, `reorder_channels`, `apply_metadata_updates`) |
| `ngff` | `ZarrWriteConfig`, array creation (`_array_create_kwargs`), sharding, multiscales metadata (`_build_multiscales`) and the pyramid writers |
| `chromatic_shift` | Z chromatic shift estimation (`estimate_chromatic_shift`) and the writer of corrected stores that copies the input's array layout (`write_chromatic_shift_corrected`); see [Chromatic shift correction](chromatic-shift.md) |

## Documentation

ZarrSmith uses **Sphinx + MyST + AutoAPI**, and the `documentation/` folder is the Sphinx source:

- the root `README.md` is the landing page. `documentation/index.md` pulls it in with a MyST `include`, so it is never copied;
- every other page is a Markdown file in `documentation/`, readable on GitHub and rendered on the documentation site;
- API pages are generated from the Python docstrings, so documenting classes, methods and helpers in the source code is the best way to improve the API reference.

To add a page, create `documentation/<page>.md`, add it to the `toctree` in `documentation/index.md` and link it from the README table. Build the site locally with:

```console
pip install -e .[docs]
sphinx-build documentation _build
```

The API entry points to document and keep stable are:

- `MicroscopeManager`, the vendor reader classes and `ZarrManager` in `zarrsmith.microscope_manager`
- `MosaicManager`, `Tile`, `H5Array`, `fuse_tiles` and `plan_fusion` in `zarrsmith.mosaic_manager`
- the zarr-writing helpers in `zarrsmith.utils`
- the CLI entry points in `zarrsmith.cli`
- the napari widgets in `zarrsmith.napari`