mizu_map_prepare() stages a map (.f, the constant arguments in ..., and
.x), serialized once into a shared map region, without running it, and
returns a prepared-map handle. Each mizu_map_run() then costs only task
submission and collection, with no serialization and no region create.
Because the region (and its name) stays alive across runs, workers that ran a
previous run reuse their cached map context instead of re-attaching. Repeated
stochastic simulation is the main use: mizu_map_run(pm, .seed = i) varies
the RNG streams per run at no extra cost, because the seed state rides the
runner payloads rather than the region.
Usage
mizu_map_prepare(
.pool,
.x,
.f,
...,
.template = NULL,
.n_chunks = NULL,
.stream = FALSE
)
mizu_map_run(pm, .x = NULL, .seed = NULL, .timeout = Inf, .collect = "copy")Arguments
- .pool
a pool handle from
mizu_pool()ormizu_pool_attach(); inside a task, the evaluating worker's own handle frommizu_current_pool().- .x
a vector (atomic or list) to map over. Anything else is coerced with
as.list(), aslapply()does.- .f
a function (or, as
match.fun()accepts, its name) applied as.f(.x[[i]], ...). Serialized once with its enclosing environment. Keep that environment small, as with any cross-process map. Amizu_call()specification maps over a pool of any worker language; see the Cross-language maps section.- ...
further constant arguments to
.f, staged once. As inmizu_submit(), the formals ahead of...are dot-prefixed, so an argument name passed here never collides with them.- .template
NULLfor a list result, or avapply()-styleFUN.VALUE: an atomic vector template that each result must match.- .n_chunks
the morsel count of the map (its scheduling granularity), or
NULLfor the default. See the Granularity section.- .stream
TRUEstreams slices of.xto workers under a sliding window instead of staging.xinto shared memory wholesale, bounding shared-memory use by the window times the slice size. See the Streaming maps section. A streaming map handle collects withmizu_collect()only (it cannot be promise-armed).- pm
a prepared-map handle from
mizu_map_prepare().- .seed
NULL(default: no RNG guarantees, no cost), or a numeric vector of length 1 or 2 that derives reproducible per-element RNG streams:seed, orc(seed, offset)to shift every element's stream byoffsetpositions. See the Reproducible RNG section.- .timeout
seconds after which submission gives up, cancels the tasks already in, and returns the
mizu_timeoutsentinel (classc("mizu_timeout", "mizu_sentinel")).Inf(the default) waits indefinitely. Submission blocks only while an injection ring is full; the collection deadline ismizu_collect()'stimeout.- .collect
"copy"(the default) gathers template results into an owning R vector atmizu_collect()."view"instead returns an ALTREP view over the map's shared output area, with no gather copy, for pipelines that reduce it immediately. Requires an atomic.template. The view is copy-on-write: the first write materializes a private copy. It keeps the map region alive until released, and re-sending it through a channel or pool crosses as a full copy, not by reference. The choice is stored on the handle.
Value
mizu_map_prepare(): a prepared-map handle. mizu_map_run():
what mizu_map() returns for the staged map, a map handle whose
mizu_collect() returns a list, a templated atomic vector, or the
mizu_timeout sentinel.
Details
Between runs, the region's shared scheduling state resets in O(1). The cursor
and cancel word clear, and the run generation embedded in every claim word
advances, so a leftover task from a previous run can never interfere with the
new run. After an unclean run (a .timeout expiry, an error in .f, a
worker death) the handle is marked stale, and the next mizu_map_run()
restages into a fresh region automatically (the old one is deleted at garbage
collection, once any leftover tasks are done with it). A run collected with
.collect = "view" restages likewise: the returned view keeps its region
alive, so the next run stages fresh rather than reusing pages a held view
still reads. A map small enough to ride entirely inline keeps its staged blob
on the handle instead, and runs resubmit it, still skipping the
serialization.
The prepared handle keeps the staged .x (for automatic restaging)
and the map region alive for its lifetime. Both release at garbage
collection when the handle is dropped. The number of chunks is fixed
at prepare time. The runner count adapts to the live workers at each
run.
Replacing x between runs
mizu_map_run(pm, .x = x2) runs over a replacement .x. When both the
staged and the replacement .x are bare-byte eligible (atomic, non-ALTREP,
no attributes beyond names) with identical type and length, the swap is in
place: the new bytes are copied over the .x section of the region. This
runs a loop over data of the same shape (optimizer steps, simulation sweeps)
at the cost of one copy, skipping the region create and the re-attach of
every worker. Any other change of .x (a different shape or type, a list, a
map staged inline) restages automatically on the next run. A streaming
prepared map keeps .x in the submitting process, so a replacement .x of
any shape re-slices on the next run. Only a length change under .template
restages, because the output area is sized for the staged length.
Examples
p <- mizu_pool()
pm <- mizu_map_prepare(p, 1:1000, function(i, draws) {
mean(rnorm(draws)) * i
}, draws = 100L)
runs <- lapply(1:50, function(s) {
m <- mizu_map_run(pm, .seed = s)
mizu_collect(m)
})
mizu_pool_stop(p)