Skip to contents

mizu_map_prepare() stages a map (.f, the constant arguments in ..., and .x), serialized once into a shared map region, without running it, and returns a prepared-map handle. Each mizu_map_run() then costs only task submission and collection, with no serialization and no region create. Because the region (and its name) stays alive across runs, workers that ran a previous run reuse their cached map context instead of re-attaching. Repeated stochastic simulation is the main use: mizu_map_run(pm, .seed = i) varies the RNG streams per run at no extra cost, because the seed state rides the runner payloads rather than the region.

Usage

mizu_map_prepare(
  .pool,
  .x,
  .f,
  ...,
  .template = NULL,
  .n_chunks = NULL,
  .stream = FALSE
)

mizu_map_run(pm, .x = NULL, .seed = NULL, .timeout = Inf, .collect = "copy")

Arguments

.pool

a pool handle from mizu_pool() or mizu_pool_attach(); inside a task, the evaluating worker's own handle from mizu_current_pool().

.x

a vector (atomic or list) to map over. Anything else is coerced with as.list(), as lapply() does.

.f

a function (or, as match.fun() accepts, its name) applied as .f(.x[[i]], ...). Serialized once with its enclosing environment. Keep that environment small, as with any cross-process map. A mizu_call() specification maps over a pool of any worker language; see the Cross-language maps section.

...

further constant arguments to .f, staged once. As in mizu_submit(), the formals ahead of ... are dot-prefixed, so an argument name passed here never collides with them.

.template

NULL for a list result, or a vapply()-style FUN.VALUE: an atomic vector template that each result must match.

.n_chunks

the morsel count of the map (its scheduling granularity), or NULL for the default. See the Granularity section.

.stream

TRUE streams slices of .x to workers under a sliding window instead of staging .x into shared memory wholesale, bounding shared-memory use by the window times the slice size. See the Streaming maps section. A streaming map handle collects with mizu_collect() only (it cannot be promise-armed).

pm

a prepared-map handle from mizu_map_prepare().

.seed

NULL (default: no RNG guarantees, no cost), or a numeric vector of length 1 or 2 that derives reproducible per-element RNG streams: seed, or c(seed, offset) to shift every element's stream by offset positions. See the Reproducible RNG section.

.timeout

seconds after which submission gives up, cancels the tasks already in, and returns the mizu_timeout sentinel (class c("mizu_timeout", "mizu_sentinel")). Inf (the default) waits indefinitely. Submission blocks only while an injection ring is full; the collection deadline is mizu_collect()'s timeout.

.collect

"copy" (the default) gathers template results into an owning R vector at mizu_collect(). "view" instead returns an ALTREP view over the map's shared output area, with no gather copy, for pipelines that reduce it immediately. Requires an atomic .template. The view is copy-on-write: the first write materializes a private copy. It keeps the map region alive until released, and re-sending it through a channel or pool crosses as a full copy, not by reference. The choice is stored on the handle.

Value

mizu_map_prepare(): a prepared-map handle. mizu_map_run(): what mizu_map() returns for the staged map, a map handle whose mizu_collect() returns a list, a templated atomic vector, or the mizu_timeout sentinel.

Details

Between runs, the region's shared scheduling state resets in O(1). The cursor and cancel word clear, and the run generation embedded in every claim word advances, so a leftover task from a previous run can never interfere with the new run. After an unclean run (a .timeout expiry, an error in .f, a worker death) the handle is marked stale, and the next mizu_map_run() restages into a fresh region automatically (the old one is deleted at garbage collection, once any leftover tasks are done with it). A run collected with .collect = "view" restages likewise: the returned view keeps its region alive, so the next run stages fresh rather than reusing pages a held view still reads. A map small enough to ride entirely inline keeps its staged blob on the handle instead, and runs resubmit it, still skipping the serialization.

The prepared handle keeps the staged .x (for automatic restaging) and the map region alive for its lifetime. Both release at garbage collection when the handle is dropped. The number of chunks is fixed at prepare time. The runner count adapts to the live workers at each run.

Replacing x between runs

mizu_map_run(pm, .x = x2) runs over a replacement .x. When both the staged and the replacement .x are bare-byte eligible (atomic, non-ALTREP, no attributes beyond names) with identical type and length, the swap is in place: the new bytes are copied over the .x section of the region. This runs a loop over data of the same shape (optimizer steps, simulation sweeps) at the cost of one copy, skipping the region create and the re-attach of every worker. Any other change of .x (a different shape or type, a list, a map staged inline) restages automatically on the next run. A streaming prepared map keeps .x in the submitting process, so a replacement .x of any shape re-slices on the next run. Only a length change under .template restages, because the output area is sized for the staged length.

Examples

p <- mizu_pool()
pm <- mizu_map_prepare(p, 1:1000, function(i, draws) {
  mean(rnorm(draws)) * i
}, draws = 100L)
runs <- lapply(1:50, function(s) {
  m <- mizu_map_run(pm, .seed = s)
  mizu_collect(m)
})
mizu_pool_stop(p)