Advanced writing options

These options control how the OME-Zarr output is laid out and how fast it is written. They are available in the Python API (to_zarr(...) keyword arguments), the CLI (flags and batch CSV columns) and the napari converter (the Additional parameters panel).

Existing outputs

ZarrSmith does not replace existing data unless you ask for it. The same rules apply to to_zarr, fuse, the chromatic shift correction, the CLI and the napari widgets:

What is at the output path	Default	With `overwrite=True`
nothing, or an empty folder	written	written
a Zarr store	refused (`FileExistsError`)	deleted and replaced, with a warning
a file, or a non-empty folder that is not a Zarr store	refused	refused
the input of this write, or a folder containing it	refused	refused

dataset.to_zarr("out.zarr")                   # FileExistsError if out.zarr already holds data
dataset.to_zarr("out.zarr", overwrite=True)   # replaces the store, with a warning

Replacing a store deletes everything in that folder. The input can never be the output because reading is lazy: ZarrManager("a.zarr").to_zarr("a.zarr") would delete the data it is about to read. To change a store in place, use write_image_region, write_label_region or create_empty_group instead.

A warning is also given when the output name does not end in .zarr or .ome.zarr. The napari widgets refuse such a name.

Python / CSV column	CLI flag	napari control	Default
`shards`	`-sh`, `--shards`	Sharding (zarr v3 only)	`auto`
`shard_target_mb`	`-stm`, `--shard_target_mb`	Shard target (MB)	1024
`shard_exclude_axes`	`-sea`, `--shard_exclude_axes`	Never merge axes	`t c`
`drop_singleton`	`-ds`, `--drop_singleton`	Drop singleton axes	`true`
`num_workers`	`-nw`, `--num_workers`	Write threads (0 = auto)	auto

In the batch CSV, leave a cell empty to use the default (so an empty shards cell means auto; write none to disable sharding).

Sharding (zarr v3 only)

Sharding is on by default (shards="auto"). Without it every chunk is its own file; with sharding, many chunks are packed into one larger file (a shard), which greatly reduces the number of files on disk while each chunk stays individually readable. Sharding needs zarr v3 / NGFF 0.5: with zarr v2 ZarrSmith issues a warning and sets sharding to none.

# Let ZarrSmith choose shard sizes (about shard_target_mb of uncompressed data per shard)
dataset.to_zarr("out.zarr", shards="auto", shard_target_mb=256)

# Disable sharding: one file per chunk
dataset.to_zarr("out.zarr", shards=None)

# Or give an explicit shard shape (one value per axis); it is snapped to a
# multiple of the chunk shape and clipped to each level's extent
dataset.to_zarr("out.zarr", shards=(1, 1, 8, 2160, 4096))

zarrsmith convert -i INPUT -m opera -z OUT.zarr -sh auto -stm 256 -sea t c   # or: -sh none to disable

shards="auto" picks a shard shape per pyramid level and leaves small levels unsharded.
shard_exclude_axes lists axes that "auto" never merges chunks along. It defaults to t c, so each timepoint and channel stays in its own shard. Use none (CLI/CSV) or () (Python) to allow every axis to merge. An explicit shard shape is always used exactly as given.
shard_target_mb is the target uncompressed size of a shard. The default is 1024 MB (1 GB). Larger shards mean fewer files but more memory per write thread (see below).

Singleton axes

With drop_singleton=True (the default) any t, c or z axis of size 1 is removed from the written dataset, so a single-timepoint, single-channel stack is stored as a plain zyx array. The axes list, scales, units and any chunks/shards you gave for the full axis set are adjusted accordingly. y and x are never dropped. Pass drop_singleton=False (-ds false in the CLI, false in the CSV, or untick the napari checkbox) to keep every axis.

dataset.to_zarr("out.zarr", drop_singleton=False)   # keep size-1 t/c/z axes

Note that, because this is on by default, existing code that writes datasets with size-1 t, c or z axes now produces arrays with fewer dimensions.

Parallel writing

Blocks of the dataset are compressed and written by several threads at once. num_workers sets the thread count; by default ZarrSmith uses half of the available cores, capped at 8, because writing stops getting faster beyond roughly 8 threads while memory use keeps growing.

dataset.to_zarr("out.zarr", num_workers=4)

zarrsmith convert -i INPUT -m opera -z OUT.zarr -nw 4

How it works:

Sharded levels are written one whole shard per task, so threads never touch the same file and no locking is needed. Each thread holds one shard in memory, so the thread count is automatically reduced to fit in about half of the free RAM, and a warning is shown if that limits parallelism. If it does, use smaller shards (a lower shard_target_mb).
All pyramid levels are written in a single pass, so the source data is read once instead of once per level. This matters most for slow sources such as files on a network drive.
With compute=False, to_zarr returns the unevaluated dask tasks instead of writing, and num_workers does not apply: you run them with your own scheduler.

Writing speed is also bounded by how fast the source can be read and decoded, which num_workers does not change.

Progress bar

to_zarr shows a tqdm progress bar (in a terminal or a notebook) while the arrays are written, labelled with the dataset being written (raw, a group or a label). The CLI shows it too. All pyramid levels of a dataset share one bar, because they are computed in a single pass.

dataset.to_zarr("out.zarr")                    # progress bar on (default)
dataset.to_zarr("out.zarr", progress=False)    # silent, e.g. for batch jobs and tests