Version: | 1.58.0 |
Title: | Unified Parallel and Distributed Processing in R for Everyone |
Depends: | R (≥ 3.2.0) |
Imports: | digest, globals (≥ 0.18.0), listenv (≥ 0.8.0), parallel, parallelly (≥ 1.44.0), utils |
Suggests: | methods, RhpcBLASctl, R.rsp, markdown |
VignetteBuilder: | R.rsp |
Description: | The purpose of this package is to provide a lightweight and unified Future API for sequential and parallel processing of R expression via futures. The simplest way to evaluate an expression in parallel is to use 'x %<-% { expression }' with 'plan(multisession)'. This package implements sequential, multicore, multisession, and cluster futures. With these, R expressions can be evaluated on the local machine, in parallel a set of local machines, or distributed on a mix of local and remote machines. Extensions to this package implement additional backends for processing futures via compute cluster schedulers, etc. Because of its unified API, there is no need to modify any code in order switch from sequential on the local machine to, say, distributed processing on a remote compute cluster. Another strength of this package is that global variables and functions are automatically identified and exported as needed, making it straightforward to tweak existing code to make use of futures. |
License: | LGPL-2.1 | LGPL-3 [expanded from: LGPL (≥ 2.1)] |
LazyLoad: | TRUE |
ByteCompile: | TRUE |
URL: | https://future.futureverse.org, https://github.com/futureverse/future |
BugReports: | https://github.com/futureverse/future/issues |
Language: | en-US |
Encoding: | UTF-8 |
RoxygenNote: | 7.3.2 |
NeedsCompilation: | no |
Packaged: | 2025-06-05 01:03:35 UTC; henrik |
Author: | Henrik Bengtsson |
Maintainer: | Henrik Bengtsson <henrikb@braju.com> |
Repository: | CRAN |
Date/Publication: | 2025-06-05 05:30:06 UTC |
Gets the length of an object without dispatching
Description
Gets the length of an object without dispatching
Usage
.length(x)
Arguments
x |
Any R object. |
Details
This function returns length(unclass(x))
, but tries to avoid
calling unclass(x)
unless necessary.
Value
A non-negative integer.
See Also
A future represents a value that will be available at some point in the future
Description
A future is an abstraction for a value that may
available at some point in the future. A future can either be
unresolved
or resolved
, a state which can be checked
with resolved()
. As long as it is unresolved, the
value is not available. As soon as it is resolved, the value
is available via value()
.
Usage
Future(
expr = NULL,
envir = parent.frame(),
substitute = TRUE,
stdout = TRUE,
conditions = "condition",
globals = list(),
packages = NULL,
seed = FALSE,
lazy = FALSE,
gc = FALSE,
earlySignal = FALSE,
label = NULL,
...
)
Arguments
expr |
An R expression. |
envir |
The environment from where global objects should be identified. |
substitute |
If TRUE, argument |
stdout |
If TRUE (default), then the standard output is captured,
and re-outputted when |
conditions |
A character string of conditions classes to be captured
and relayed. The default is to relay all conditions, including messages
and warnings. To drop all conditions, use |
globals |
(optional) a logical, a character vector, or a named list
to control how globals are handled.
For details, see section 'Globals used by future expressions'
in the help for |
packages |
(optional) a character vector specifying packages to be attached in the R environment evaluating the future. |
seed |
(optional) If TRUE, the random seed, that is, the state of the
random number generator (RNG) will be set such that statistically sound
random numbers are produced (also during parallelization).
If FALSE (default), it is assumed that the future expression does neither
need nor use random numbers generation.
To use a fixed random seed, specify a L'Ecuyer-CMRG seed (seven integer)
or a regular RNG seed (a single integer). If the latter, then a
L'Ecuyer-CMRG seed will be automatically created based on the given seed.
Furthermore, if FALSE, then the future will be monitored to make sure it
does not use random numbers. If it does and depending on the value of
option future.rng.onMisuse, the check is
ignored, an informative warning, or error will be produced.
If |
lazy |
If FALSE (default), the future is resolved eagerly (starting immediately), otherwise not. |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
label |
A character string label attached to the future. |
... |
Additional named elements of the future. |
Details
A Future object is itself an environment.
Value
Future()
returns an object of class Future
.
See Also
One function that creates a Future is future()
.
It returns a Future that evaluates an R expression in the future.
An alternative approach is to use the %<-%
infix
assignment operator, which creates a future from the
right-hand-side (RHS) R expression and assigns its future value
to a variable as a promise.
Configure a backend that controls how and where futures are evaluated
Description
This functionality is only for developers who wish to implement their own future backend. End-users and package developers use futureverse, does not need to know about these functions.
If you are looking for available future backends to choose from, please see the 'A Future for R: Available Future Backends' vignette and https://www.futureverse.org/backends.html.
Usage
FutureBackend(
...,
earlySignal = FALSE,
gc = FALSE,
maxSizeOfObjects = getOption("future.globals.maxSize", +Inf),
interrupts = TRUE,
hooks = FALSE
)
launchFuture(backend, future, ...)
listFutures(backend, ...)
interruptFuture(backend, future, ...)
validateFutureGlobals(backend, future, ...)
stopWorkers(backend, ...)
MultiprocessFutureBackend(
...,
wait.timeout = getOption("future.wait.timeout", 24 * 60 * 60),
wait.interval = getOption("future.wait.interval", 0.01),
wait.alpha = getOption("future.wait.alpha", 1.01)
)
ClusterFutureBackend(
workers = availableWorkers(constraints = "connections"),
gc = TRUE,
earlySignal = TRUE,
interrupts = FALSE,
persistent = FALSE,
...
)
MulticoreFutureBackend(
workers = availableCores(constraints = "multicore"),
maxSizeOfObjects = +Inf,
...
)
SequentialFutureBackend(..., maxSizeOfObjects = +Inf)
MultisessionFutureBackend(
workers = availableCores(),
rscript_libs = .libPaths(),
interrupts = TRUE,
gc = FALSE,
earlySignal = FALSE,
...
)
Arguments
earlySignal |
Overrides the default behavior on whether futures
should resignal ("relay") conditions captured as soon as possible, or
delayed, for instance, until |
gc |
Overrides the default behavior of whether futures should trigger
garbage collection via |
maxSizeOfObjects |
The maximum allowed total size, in bytes, of all
objects to and from the parallel worker allows.
This can help to protect against unexpectedly large data transfers between
the parent process and the parallel workers - data that is often transferred
over the network, which sometimes also includes the internet. For instance,
if you sit at home and have set up a future backend with workers running
remotely at your university or company, then you might want to use this
protection to avoid transferring giga- or terabytes of data without noticing.
(Default: |
interrupts |
If FALSE, attempts to interrupt futures will not take place on this backend, even if the backend supports it. This is useful when, for instance, it takes a long time to interrupt a future. |
backend |
|
future |
a Future to be started. |
wait.timeout |
Number of seconds before timing out. |
wait.interval |
Baseline number of seconds between retries. |
wait.alpha |
Scale factor increasing waiting interval after each attempt. |
workers |
... |
persistent |
(deprecated) ... |
... |
(optional) not used. |
Details
The ClusterFutureBackend
is selected by
plan(cluster, workers = workers)
.
The MulticoreFutureBackend
backend is selected by
plan(multicore, workers = workers)
.
The SequentialFutureBackend
is selected by plan(sequential)
.
The MultisessionFutureBackend
backend is selected by
plan(multisession, workers = workers)
.
Value
FutureBackend()
returns a FutureBackend object, which inherits an
environment. Specific future backends are defined by subclasses
implementing the FutureBackend API.
launchFuture()
returns the launched Future
object.
interruptFuture()
returns the interrupted Future
object,
if supported, other the unmodified future.
stopWorkers()
returns TRUE if the workers were shut down,
otherwise FALSE.
The FutureBackend API
The FutureBackend
class specifies FutureBackend API,
that all backends must implement and comply to. Specifically,
See Also
For alternative future backends, see the 'A Future for R: Available Future Backends' vignette and https://www.futureverse.org/backends.html.
For alternative future backends, see the 'A Future for R: Available Future Backends' vignette and https://www.futureverse.org/backends.html.
A condition (message, warning, or error) that occurred while orchestrating a future
Description
While orchestrating (creating, launching, querying, collection) futures, unexpected run-time errors (and other types of conditions) may occur. Such conditions are coerced to a corresponding FutureCondition class to help distinguish them from conditions that occur due to the evaluation of the future.
Usage
FutureCondition(message, call = NULL, uuid = future[["uuid"]], future = NULL)
FutureMessage(message, call = NULL, uuid = future[["uuid"]], future = NULL)
FutureWarning(message, call = NULL, uuid = future[["uuid"]], future = NULL)
FutureError(message, call = NULL, uuid = future[["uuid"]], future = NULL)
RngFutureCondition(
message = NULL,
call = NULL,
uuid = future[["uuid"]],
future = NULL
)
RngFutureWarning(...)
RngFutureError(...)
UnexpectedFutureResultError(future, hint = NULL)
GlobalEnvMisuseFutureCondition(
message = NULL,
call = NULL,
differences = NULL,
uuid = future[["uuid"]],
future = NULL
)
GlobalEnvMisuseFutureWarning(...)
GlobalEnvMisuseFutureError(...)
ConnectionMisuseFutureCondition(
message = NULL,
call = NULL,
differences = NULL,
uuid = future[["uuid"]],
future = NULL
)
ConnectionMisuseFutureWarning(...)
ConnectionMisuseFutureError(...)
DeviceMisuseFutureCondition(
message = NULL,
call = NULL,
differences = NULL,
uuid = future[["uuid"]],
future = NULL
)
DeviceMisuseFutureWarning(...)
DeviceMisuseFutureError(...)
DefaultDeviceMisuseFutureCondition(
message = NULL,
incidents = NULL,
call = NULL,
uuid = future[["uuid"]],
future = NULL
)
DefaultDeviceMisuseFutureWarning(...)
DefaultDeviceMisuseFutureError(...)
FutureLaunchError(..., future = NULL)
FutureInterruptError(..., future = NULL)
FutureCanceledError(..., future = NULL)
FutureDroppedError(..., future = NULL)
FutureJournalCondition(
message,
journal,
call = NULL,
uuid = future[["uuid"]],
future = NULL
)
Arguments
message |
A message condition. |
call |
The call stack that led up to the condition. |
uuid |
A universally unique identifier for the future associated with this FutureCondition. |
future |
The Future involved. |
hint |
(optional) A string with a suggestion on what might be wrong. |
Value
An object of class FutureCondition which inherits from class condition and FutureMessage, FutureWarning, and FutureError all inherits from FutureCondition. Moreover, a FutureError inherits from error, a FutureWarning from warning, and a FutureMessage from message.
A representation of a set of globals used with futures
Description
A representation of a set of globals used with futures
Usage
FutureGlobals(object = list(), resolved = FALSE, total_size = NA_real_, ...)
Arguments
object |
A named list. |
resolved |
A logical indicating whether these globals have been scanned for and resolved futures or not. |
total_size |
The total size of all globals, if known. |
... |
Not used. |
Details
This class extends the Globals class by adding
attributes resolved
and total_size
.
Value
An object of class FutureGlobals
.
Results from resolving a future
Description
Results from resolving a future
Usage
FutureResult(
value = NULL,
visible = TRUE,
stdout = NULL,
conditions = NULL,
rng = FALSE,
...,
uuid = NULL,
started = .POSIXct(NA_real_),
finished = Sys.time(),
version = "1.8"
)
## S3 method for class 'FutureResult'
as.character(x, ...)
## S3 method for class 'FutureResult'
print(x, ...)
Arguments
value |
The value of the future expression.
If the expression was not fully resolved (e.g. an error) occurred,
the the value is |
visible |
If TRUE, the value was visible, otherwise invisible. |
conditions |
A list of zero or more list elements each containing a captured condition and possibly more meta data such as the call stack and a timestamp. |
rng |
If TRUE, the |
started , finished |
POSIXct timestamps when the evaluation of the future expression was started and finished. |
version |
The version format of the results. |
... |
(optional) Additional named results to be returned. |
Details
This function is only part of the backend Future API. This function is not part of the frontend Future API.
Value
An object of class FutureResult.
Note to developers
The FutureResult structure is under development and may change at anytime, e.g. elements may be renamed or removed. Because of this, please avoid accessing the elements directly in code. Feel free to reach out if you need to do so in your code.
A multicore future is a future whose value will be resolved asynchronously in a parallel process
Description
A multicore future is a future whose value will be resolved asynchronously in a parallel process
Usage
## S3 method for class 'MulticoreFuture'
resolved(x, run = TRUE, timeout = NULL, ...)
Value
MulticoreFuture()
returns an object of class MulticoreFuture
.
Usage
To use 'multicore' futures, use plan(multicore, ...)
, cf. multicore.
A multiprocess future is a future whose value will be resolved asynchronously in a parallel process
Description
A multiprocess future is a future whose value will be resolved asynchronously in a parallel process
Usage
MultiprocessFuture(expr = NULL, substitute = TRUE, envir = parent.frame(), ...)
Arguments
expr |
An R expression. |
substitute |
If TRUE, argument |
envir |
The environment from where global objects should be identified. |
... |
Additional named elements passed to |
Value
MultiprocessFuture()
returns an object of class MultiprocessFuture
.
An uniprocess future is a future whose value will be resolved synchronously in the current process
Description
An uniprocess future is a future whose value will be resolved synchronously in the current process
Usage
UniprocessFuture(expr = NULL, substitute = TRUE, envir = parent.frame(), ...)
Arguments
expr |
An R expression. |
substitute |
If TRUE, argument |
envir |
The environment from where global objects should be identified. |
... |
Additional named elements passed to |
Value
UniprocessFuture()
returns an object of class UniprocessFuture
.
Back trace the expressions evaluated when an error was caught
Description
Back trace the expressions evaluated when an error was caught
Usage
backtrace(future, envir = parent.frame(), ...)
Arguments
future |
A future with a caught error. |
envir |
the environment where to locate the future. |
... |
Not used. |
Value
A list with the future's call stack that led up to the error.
Examples
my_log <- function(x) log(x)
foo <- function(...) my_log(...)
f <- future({ foo("a") })
res <- tryCatch({
v <- value(f)
}, error = function(ex) {
t <- backtrace(f)
print(t)
})
Cancel a future
Description
Cancels futures, with the option to interrupt running ones.
Usage
cancel(x, interrupt = TRUE, ...)
Arguments
x |
A Future. |
interrupt |
If TRUE, running futures are interrupted, if the future backend supports it. |
... |
All arguments used by the S3 methods. |
Value
cancel()
returns (invisibly) the canceled Futures after
flagging them as "canceled" and possibly interrupting them as well.
Canceling a lazy or a finished future has no effect.
See Also
A canceled future can be reset()
to a lazy, vanilla future
such that it can be relaunched, possible on another future backend.
Examples
## Set up two parallel workers
plan(multisession, workers = 2)
## Launch two long running future
fs <- lapply(c(1, 2), function(duration) {
future({
Sys.sleep(duration)
42
})
})
## Wait until at least one of the futures is resolved
while (!any(resolved(fs))) Sys.sleep(0.1)
## Cancel the future that is not yet resolved
r <- resolved(fs)
cancel(fs[!r])
## Get the value of the resolved future
f <- fs[r]
v <- value(f)
message("Result: ", v)
## The value of the canceled future is an error
try(v <- value(fs[!r]))
## Shut down parallel workers
plan(sequential)
Create a cluster future whose value will be resolved asynchronously in a parallel process
Description
WARNING: This function must never be called.
It may only be used with plan()
Usage
cluster(
...,
workers = availableWorkers(constraints = "connections"),
gc = FALSE,
earlySignal = FALSE,
persistent = FALSE,
envir = parent.frame()
)
Arguments
workers |
A |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
persistent |
If FALSE, the evaluation environment is cleared from objects prior to the evaluation of the future. |
envir |
The environment from where global objects should be identified. |
... |
Additional named elements passed to |
Details
A cluster future is a future that uses cluster evaluation, which means that its value is computed and resolved in parallel in another process.
This function is must not be called directly. Instead, the typical usages are:
# Evaluate futures via a single background R process on the local machine plan(cluster, workers = I(1)) # Evaluate futures via two background R processes on the local machine plan(cluster, workers = 2) # Evaluate futures via a single R process on another machine on on the # local area network (LAN) plan(cluster, workers = "raspberry-pi") # Evaluate futures via a single R process running on a remote machine plan(cluster, workers = "pi.example.org") # Evaluate futures via four R processes, one running on the local machine, # two running on LAN machine 'n1' and one on a remote machine plan(cluster, workers = c("localhost", "n1", "n1", "pi.example.org"))
Value
A ClusterFuture.
See Also
For alternative future backends, see the 'A Future for R: Available Future Backends' vignette and https://www.futureverse.org/backends.html.
Examples
## Use cluster futures
cl <- parallel::makeCluster(2, timeout = 60)
plan(cluster, workers = cl)
## A global variable
a <- 0
## Create future (explicitly)
f <- future({
b <- 3
c <- 2
a * b * c
})
## A cluster future is evaluated in a separate process.
## Regardless, changing the value of a global variable will
## not affect the result of the future.
a <- 7
print(a)
v <- value(f)
print(v)
stopifnot(v == 0)
## CLEANUP
parallel::stopCluster(cl)
Export globals to the sticky-globals environment of the cluster nodes
Description
Export globals to the sticky-globals environment of the cluster nodes
Usage
clusterExportSticky(cl, globals)
Arguments
cl |
(cluster) A cluster object as returned by
|
globals |
(list) A named list of sticky globals to be exported. |
Details
This requires that the future package is installed on the cluster nodes.
Value
(invisible; cluster) The cluster object.
Get the first or all references of an R object
Description
Get the first or all references of an R object
Assert that there are no references among the identified globals
Usage
find_references(x, first_only = FALSE)
assert_no_references(
x,
action = c("error", "warning", "message", "string"),
source = c("globals", "value")
)
Arguments
x |
The R object to be checked. |
first_only |
If |
action |
Type of action to take if a reference is found. |
source |
Is the source of |
Value
find_references()
returns a list of zero or more references
identified.
If a reference is detected, an informative error, warning, message,
or a character string is produced, otherwise NULL
is returned
invisibly.
Create a future
Description
Creates a future that evaluates an R expression or
a future that calls an R function with a set of arguments.
How, when, and where these futures are evaluated can be configured
using
plan()
such that it is evaluated in parallel on,
for instance, the current machine, on a remote machine, or via a
job queue on a compute cluster.
Importantly, any R code using futures remains the same regardless
on these settings and there is no need to modify the code when
switching from, say, sequential to parallel processing.
Usage
future(
expr,
envir = parent.frame(),
substitute = TRUE,
lazy = FALSE,
seed = FALSE,
globals = TRUE,
packages = NULL,
stdout = TRUE,
conditions = "condition",
label = NULL,
gc = FALSE,
earlySignal = FALSE,
...
)
futureCall(
FUN,
args = list(),
envir = parent.frame(),
lazy = FALSE,
seed = FALSE,
globals = TRUE,
packages = NULL,
stdout = TRUE,
conditions = "condition",
earlySignal = FALSE,
label = NULL,
gc = FALSE,
...
)
minifuture(
expr,
substitute = TRUE,
globals = NULL,
packages = NULL,
stdout = NA,
conditions = NULL,
seed = NULL,
...,
envir = parent.frame()
)
Arguments
expr |
An R expression. |
envir |
The environment from where global objects should be identified. |
substitute |
If TRUE, argument |
lazy |
If FALSE (default), the future is resolved eagerly (starting immediately), otherwise not. |
seed |
(optional) If TRUE, the random seed, that is, the state of the
random number generator (RNG) will be set such that statistically sound
random numbers are produced (also during parallelization).
If FALSE (default), it is assumed that the future expression does neither
need nor use random numbers generation.
To use a fixed random seed, specify a L'Ecuyer-CMRG seed (seven integer)
or a regular RNG seed (a single integer). If the latter, then a
L'Ecuyer-CMRG seed will be automatically created based on the given seed.
Furthermore, if FALSE, then the future will be monitored to make sure it
does not use random numbers. If it does and depending on the value of
option future.rng.onMisuse, the check is
ignored, an informative warning, or error will be produced.
If |
globals |
(optional) a logical, a character vector, or a named list
to control how globals are handled.
For details, see section 'Globals used by future expressions'
in the help for |
packages |
(optional) a character vector specifying packages to be attached in the R environment evaluating the future. |
stdout |
If TRUE (default), then the standard output is captured,
and re-outputted when |
conditions |
A character string of conditions classes to be captured
and relayed. The default is to relay all conditions, including messages
and warnings. To drop all conditions, use |
label |
A character string label attached to the future. |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
FUN |
A function to be evaluated. |
args |
A list of arguments passed to function |
... |
Additional arguments passed to |
Details
The state of a future is either unresolved or resolved.
The value of a future can be retrieved using v <- value(f)
.
Querying the value of a non-resolved future will block the call
until the future is resolved.
It is possible to check whether a future is resolved or not
without blocking by using resolved(f)
.
It is possible to cancel()
a future that is being resolved.
Failed, canceled, and interrupted futures can be reset()
to a
lazy, vanilla future that can be relaunched.
The futureCall()
function works analogously to
do.call()
, which calls a function with a set of
arguments. The difference is that do.call()
returns the value of
the call whereas futureCall()
returns a future.
Value
future()
returns Future that evaluates expression expr
.
futureCall()
returns a Future that calls function FUN
with
arguments args
.
minifuture(expr)
creates a future with minimal overhead, by disabling
user-friendly behaviors, e.g. automatic identification of global
variables and packages needed, and relaying of output. Unless you have
good reasons for using this function, please use future()
instead.
This function exists mainly for the purpose of profiling and identifying
which automatic features of future()
introduce extra overhead.
Eager or lazy evaluation
By default, a future is resolved using eager evaluation
(lazy = FALSE
). This means that the expression starts to
be evaluated as soon as the future is created.
As an alternative, the future can be resolved using lazy
evaluation (lazy = TRUE
). This means that the expression
will only be evaluated when the value of the future is requested.
Note that this means that the expression may not be evaluated
at all - it is guaranteed to be evaluated if the value is requested.
Globals used by future expressions
Global objects (short globals) are objects (e.g. variables and functions) that are needed in order for the future expression to be evaluated while not being local objects that are defined by the future expression. For example, in
a <- 42 f <- future({ b <- 2; a * b })
variable a
is a global of future assignment f
whereas
b
is a local variable.
In order for the future to be resolved successfully (and correctly),
all globals need to be gathered when the future is created such that
they are available whenever and wherever the future is resolved.
The default behavior (globals = TRUE
),
is that globals are automatically identified and gathered.
More precisely, globals are identified via code inspection of the
future expression expr
and their values are retrieved with
environment envir
as the starting point (basically via
get(global, envir = envir, inherits = TRUE)
).
In most cases, such automatic collection of globals is sufficient
and less tedious and error prone than if they are manually specified.
However, for full control, it is also possible to explicitly specify exactly which the globals are by providing their names as a character vector. In the above example, we could use
a <- 42 f <- future({ b <- 2; a * b }, globals = "a")
Yet another alternative is to explicitly specify also their values using a named list as in
a <- 42 f <- future({ b <- 2; a * b }, globals = list(a = a))
or
f <- future({ b <- 2; a * b }, globals = list(a = 42))
Specifying globals explicitly avoids the overhead added from automatically identifying the globals and gathering their values. Furthermore, if we know that the future expression does not make use of any global variables, we can disable the automatic search for globals by using
f <- future({ a <- 42; b <- 2; a * b }, globals = FALSE)
Future expressions often make use of functions from one or more packages. As long as these functions are part of the set of globals, the future package will make sure that those packages are attached when the future is resolved. Because there is no need for such globals to be frozen or exported, the future package will not export them, which reduces the amount of transferred objects. For example, in
x <- rnorm(1000) f <- future({ median(x) })
variable x
and median()
are globals, but only x
is exported whereas median()
, which is part of the stats
package, is not exported. Instead it is made sure that the stats
package is on the search path when the future expression is evaluated.
Effectively, the above becomes
x <- rnorm(1000) f <- future({ library(stats) median(x) })
To manually specify this, one can either do
x <- rnorm(1000) f <- future({ median(x) }, globals = list(x = x, median = stats::median)
or
x <- rnorm(1000) f <- future({ library(stats) median(x) }, globals = list(x = x))
Both are effectively the same.
Although rarely needed, a combination of automatic identification and manual
specification of globals is supported via attributes add
(to add
false negatives) and ignore
(to ignore false positives) on value
TRUE
. For example, with
globals = structure(TRUE, ignore = "b", add = "a")
any globals
automatically identified, except b
, will be used, in addition to
global a
.
Author(s)
The future logo was designed by Dan LaBar and tweaked by Henrik Bengtsson.
See Also
How, when and where futures are resolved is given by the
future backend, which can be set by the end user using the
plan()
function.
Examples
## Evaluate futures in parallel
plan(multisession)
## Data
x <- rnorm(100)
y <- 2 * x + 0.2 + rnorm(100)
w <- 1 + x ^ 2
## EXAMPLE: Regular assignments (evaluated sequentially)
fitA <- lm(y ~ x, weights = w) ## with offset
fitB <- lm(y ~ x - 1, weights = w) ## without offset
fitC <- {
w <- 1 + abs(x) ## Different weights
lm(y ~ x, weights = w)
}
print(fitA)
print(fitB)
print(fitC)
## EXAMPLE: Future assignments (evaluated in parallel)
fitA %<-% lm(y ~ x, weights = w) ## with offset
fitB %<-% lm(y ~ x - 1, weights = w) ## without offset
fitC %<-% {
w <- 1 + abs(x)
lm(y ~ x, weights = w)
}
print(fitA)
print(fitB)
print(fitC)
## EXAMPLE: Explicitly create futures (evaluated in parallel)
## and retrieve their values
fA <- future( lm(y ~ x, weights = w) )
fB <- future( lm(y ~ x - 1, weights = w) )
fC <- future({
w <- 1 + abs(x)
lm(y ~ x, weights = w)
})
fitA <- value(fA)
fitB <- value(fB)
fitC <- value(fC)
print(fitA)
print(fitB)
print(fitC)
## EXAMPLE: futureCall() and do.call()
x <- 1:100
y0 <- do.call(sum, args = list(x))
print(y0)
f1 <- futureCall(sum, args = list(x))
y1 <- value(f1)
print(y1)
Create a future assignment
Description
x %<-% value
(also known as a "future assignment") and
futureAssign("x", value)
create a Future that evaluates the expression
(value
) and binds it to variable x
(as a
promise). The expression is evaluated in parallel
in the background. Later on, when x
is first queried, the value of future
is automatically retrieved as it were a regular variable and x
is
materialized as a regular value.
Usage
futureAssign(
x,
value,
envir = parent.frame(),
substitute = TRUE,
lazy = FALSE,
seed = FALSE,
globals = TRUE,
packages = NULL,
stdout = TRUE,
conditions = "condition",
earlySignal = FALSE,
label = NULL,
gc = FALSE,
...,
assign.env = envir
)
x %<-% value
fassignment %globals% globals
fassignment %packages% packages
fassignment %seed% seed
fassignment %stdout% capture
fassignment %conditions% capture
fassignment %lazy% lazy
fassignment %label% label
fassignment %plan% strategy
fassignment %tweak% tweaks
Arguments
x |
the name of a future variable, which will hold the value of the future expression (as a promise). |
value |
An R expression. |
envir |
The environment from where global objects should be identified. |
substitute |
If TRUE, argument |
lazy |
If FALSE (default), the future is resolved eagerly (starting immediately), otherwise not. |
seed |
(optional) If TRUE, the random seed, that is, the state of the
random number generator (RNG) will be set such that statistically sound
random numbers are produced (also during parallelization).
If FALSE (default), it is assumed that the future expression does neither
need nor use random numbers generation.
To use a fixed random seed, specify a L'Ecuyer-CMRG seed (seven integer)
or a regular RNG seed (a single integer). If the latter, then a
L'Ecuyer-CMRG seed will be automatically created based on the given seed.
Furthermore, if FALSE, then the future will be monitored to make sure it
does not use random numbers. If it does and depending on the value of
option future.rng.onMisuse, the check is
ignored, an informative warning, or error will be produced.
If |
globals |
(optional) a logical, a character vector, or a named list
to control how globals are handled.
For details, see section 'Globals used by future expressions'
in the help for |
packages |
(optional) a character vector specifying packages to be attached in the R environment evaluating the future. |
stdout |
If TRUE (default), then the standard output is captured,
and re-outputted when |
conditions |
A character string of conditions classes to be captured
and relayed. The default is to relay all conditions, including messages
and warnings. To drop all conditions, use |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
label |
A character string label attached to the future. |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
assign.env |
The environment to which the variable should be assigned. |
fassignment |
The future assignment, e.g.
|
capture |
If TRUE, the standard output will be captured, otherwise not. |
strategy |
The backend controlling how the future is
resolved. See |
tweaks |
A named list (or vector) with arguments that should be changed relative to the current backend. |
... |
Additional arguments passed to |
Details
For a future created via a future assignment, x %<-% value
or
futureAssign("x", value)
, the value is bound to a promise, which when
queried will internally call value()
on the future and which will then
be resolved into a regular variable bound to that value. For example, with
future assignment x %<-% value
, the first time variable x
is queried
the call blocks if, and only if, the future is not yet resolved. As soon
as it is resolved, and any succeeding queries, querying x
will
immediately give the value.
The future assignment construct x %<-% value
is not a formal assignment
per se, but a binary infix operator on objects x
and expression value
.
However, by using non-standard evaluation, this constructs can emulate an
assignment operator similar to x <- value
. Due to R's precedence rules
of operators, future expressions often need to be explicitly bracketed,
e.g. x %<-% { a + b }
.
Value
futureAssign()
and x %<-% expr
returns the Future invisibly,
e.g. f <- futureAssign("x", expr)
and f <- (x %<-% expr)
.
Adjust future arguments of a future assignment
future()
and futureAssign()
take several arguments that can be used
to explicitly specify what global variables and packages the future should
use. They can also be used to override default behaviors of the future,
e.g. whether output should be relayed or not. When using a future
assignment, these arguments can be specified via corresponding
assignment expression. For example, x %<-% { rnorm(10) } %seed% TRUE
corresponds to futureAssign("x", { rnorm(10) }, seed = TRUE)
. Here are
a several examples.
To explicitly specify variables and functions that a future assignment
should use, use %globals%
. To explicitly specify which packages need
to be attached for the evaluate to success, use %packages%
. For
example,
> x <- rnorm(1000) > y %<-% { median(x) } %globals% list(x = x) %packages% "stats" > y [1] -0.03956372
The median()
function is part of the 'stats' package.
To declare that you will generate random numbers, use %seed%
, e.g.
> x %<-% { rnorm(3) } %seed% TRUE > x [1] -0.2590562 -1.2262495 0.8858702
To disable relaying of standard output (e.g. print()
, cat()
, and
str()
), while keeping relaying of conditions (e.g. message()
and
> x %<-% { cat("Hello\n"); message("Hi there"); 42 } %stdout% FALSE > y <- 13 > z <- x + y Hi there > z [1] 55
To disable relaying of conditions, use %conditions%
, e.g.
> x %<-% { cat("Hello\n"); message("Hi there"); 42 } %conditions% character(0) > y <- 13 > z <- x + y Hello > z [1] 55
> x %<-% { print(1:10); message("Hello"); 42 } %stdout% FALSE > y <- 13 > z <- x + y Hello > z [1] 55
To create a future without launching in such that it will only be
processed if the value is really needed, use %lazy%
, e.g.
> x %<-% { Sys.sleep(5); 42 } %lazy% TRUE > y <- sum(1:10) > system.time(z <- x + y) user system elapsed 0.004 0.000 5.008 > z [1] 97
Error handling
Because future assignments are promises, errors produced by the the future expression will not be signaled until the value of the future is requested. For example, if you create a future assignment that produce an error, you will not be affected by the error until you "touch" the future-assignment variable. For example,
> x %<-% { stop("boom") } > y <- sum(1:10) > z <- x + y Error in eval(quote({ : boom
Use alternative future backend for future assignment
Futures are evaluated on the future backend that the user has specified
by plan()
. With regular futures, we can temporarily use another future
backend by wrapping our code in with(plan(...), { ... }]
, or temporarily
inside a function using with(plan(...), local = TRUE)
. To achieve the
same for a specific future assignment, use %plan%
, e.g.
> plan(multisession) > x %<-% { 42 } > y %<-% { 13 } %plan% sequential > z <- x + y > z [1] 55
Here x
is resolved in the background via the multisession backend,
whereas y
is resolved sequentially in the main R session.
Getting the future object of a future assignment
The underlying Future of a future variable x
can be retrieved without
blocking using f <- futureOf(x)
, e.g.
> x %<-% { stop("boom") } > f_x <- futureOf(x) > resolved(f_x) [1] TRUE > x Error in eval(quote({ : boom > value(f_x) Error in eval(quote({ : boom
Technically, both the future and the variable (promise) are assigned at
the same time to environment assign.env
where the name of the future is
.future_<name>
.
Get the future of a future variable
Description
Get the future of a future variable that has been created directly
or indirectly via future()
.
Usage
futureOf(
var = NULL,
envir = parent.frame(),
mustExist = TRUE,
default = NA,
drop = FALSE
)
Arguments
var |
the variable. If NULL, all futures in the environment are returned. |
envir |
the environment where to search from. |
mustExist |
If TRUE and the variable does not exists, then an informative error is thrown, otherwise NA is returned. |
default |
the default value if future was not found. |
drop |
if TRUE and |
Value
A Future (or default
).
If var
is NULL, then a named list of Future:s are returned.
Examples
a %<-% { 1 }
f <- futureOf(a)
print(f)
b %<-% { 2 }
f <- futureOf(b)
print(f)
## All futures
fs <- futureOf()
print(fs)
## Futures part of environment
env <- new.env()
env$c %<-% { 3 }
f <- futureOf(env$c)
print(f)
f2 <- futureOf(c, envir = env)
print(f2)
f3 <- futureOf("c", envir = env)
print(f3)
fs <- futureOf(envir = env)
print(fs)
Get future-specific session information and validate current backend
Description
Get future-specific session information and validate current backend
Usage
futureSessionInfo(test = TRUE, anonymize = TRUE)
Arguments
test |
If TRUE, one or more futures are created to query workers and validate their information. |
anonymize |
If TRUE, user names and host names are anonymized. |
Value
Nothing.
Examples
plan(multisession, workers = 2)
futureSessionInfo()
plan(sequential)
Get all futures in a container
Description
Gets all futures in an environment, a list, or a list environment and returns an object of the same class (and dimensions). Non-future elements are returned as is.
Usage
futures(x, ...)
Arguments
x |
An environment, a list, or a list environment. |
... |
Not used. |
Details
This function is useful for retrieve futures that were created via
future assignments (%<-%
) and therefore stored as promises.
This function turns such promises into standard Future
objects.
Value
An object of same type as x
and with the same names
and/or dimensions, if set.
Inject code for the next type of future to use for nested futures
Description
Inject code for the next type of future to use for nested futures
Usage
getExpression(future, ...)
Arguments
future |
Current future. |
... |
Not used. |
Details
If there is no future backend specified after this one, the default
is to use sequential futures. This conservative approach protects
against spawning off recursive futures by mistake, especially
multicore and multisession ones.
The default will also set options(mc.cores = 1L)
(*) so that
no parallel R processes are spawned off by functions such as
parallel::mclapply()
and friends.
Currently it is not possible to specify what type of nested futures to be used, meaning the above default will always be used. See Issue #37 for plans on adding support for custom nested future types.
(*) Ideally we would set mc.cores = 0
but that will unfortunately
cause mclapply()
and friends to generate an error saying
"'mc.cores' must be >= 1". Ideally those functions should
fall back to using the non-multicore alternative in this
case, e.g. mclapply(...)
=> lapply(...)
.
See https://github.com/HenrikBengtsson/Wishlist-for-R/issues/7
for a discussion on this.
Value
A future expression with code injected to set what type of future to use for nested futures, iff any.
Retrieves global variables of an expression and their associated packages
Description
Retrieves global variables of an expression and their associated packages
Usage
getGlobalsAndPackages(
expr,
envir = parent.frame(),
tweak = tweakExpression,
globals = TRUE,
locals = getOption("future.globals.globalsOf.locals", TRUE),
resolve = getOption("future.globals.resolve"),
persistent = FALSE,
maxSize = getOption("future.globals.maxSize", 500 * 1024^2),
onReference = getOption("future.globals.onReference", "ignore"),
...
)
Arguments
expr |
An R expression whose globals should be found. |
envir |
The environment from which globals should be searched. |
tweak |
(optional) A function that takes an expression and returned a modified one. |
globals |
(optional) a logical, a character vector, a named list, or a Globals object. If TRUE, globals are identified by code inspection based on |
locals |
Should globals part of any "local" environment of a function be included or not? |
resolve |
If TRUE, any future that is a global variables (or part of one) is resolved and replaced by a "constant" future. |
persistent |
If TRUE, non-existing globals (= identified in expression but not found in memory) are always silently ignored and assumed to be existing in the evaluation environment. If FALSE, non-existing globals are by default ignored, but may also trigger an informative error if option future.globals.onMissing in |
maxSize |
The maximum allowed total size (in bytes) of globals—for
the purpose of preventing too large exports / transfers happening by
mistake. If the total size of the global objects are greater than this
limit, an informative error message is produced. If
|
... |
Not used. |
Value
A named list with elements expr
(the tweaked expression), globals
(a named list of class FutureGlobals) and packages
(a character string).
See Also
Internally, globalsOf()
is used to identify globals and associated packages from the expression.
Create a Future Cluster of Stateless Workers for Parallel Processing
Description
WARNING: Please note that this sets up a stateless set of cluster nodes,
which means that clusterEvalQ(cl, { a <- 3.14 })
will have no effect.
Consider this a first beta version and use it with great care,
particularly because of the stateless nature of the cluster.
For now, I recommend to manually validate that you can get identical
results using this cluster type with what you get from using the
classical parallel::makeCluster()
cluster type.
Usage
makeClusterFuture(specs = nbrOfWorkers(), ...)
Arguments
specs |
Ignored.
If specified, the value should equal |
... |
Named arguments passed to |
Value
Returns a parallel cluster
object of class FutureCluster
.
Future Clusters are Stateless
Traditionally, a cluster nodes has a one-to-one mapping to a cluster
worker process. For example, cl <- makeCluster(2, type = "PSOCK")
launches two parallel worker processes in the background, where
cluster node cl[[1]]
maps to worker #1 and node cl[[2]]
to
worker #2, and that never changes through the lifespan of these
workers. This one-to-one mapping allows for deterministic
configuration of workers. For examples, some code may assign globals
with values specific to each worker, e.g.
clusterEvalQ(cl[1], { a <- 3.14 })
and
clusterEvalQ(cl[2], { a <- 2.71 })
.
In contrast, there is no one-to-one mapping between cluster nodes and the parallel workers when using a future cluster. This is because we cannot make assumptions on where are parallel task will be processed. Where a parallel task is processes is up to the future backend to decide - some backends do this deterministically, whereas others other resolves task at the first available worker. Also, the worker processes might be transient for some future backends, i.e. the only exist for the life-span of the parallel task and then terminates.
Because of this, one must not rely in node-specific behaviors,
because that concept does not make sense with a future cluster.
To protect against this, any attempt to address a subset of future
cluster nodes, results in an error, e.g. clusterEvalQ(cl[1], ...)
,
clusterEvalQ(cl[1:2], ...)
, and clusterEvalQ(cl[2:1], ...)
in
the above example will all give an error.
Exceptions to the latter limitation are clusterSetRNGStream()
and clusterExport()
, which can be safely used with future clusters.
See below for more details.
If clusterEvalQ()
is called, the call is ignored, and a warning
is produced.
clusterSetRNGStream
parallel::clusterSetRNGStream()
distributes "L'Ecuyer-CMRG" RNG
streams to the cluster nodes, which record them such that the next
round of futures will use them. When used, the RNG state after the
futures are resolved are recorded accordingly, such that the next
round again of future will use those, and so on. This strategy
makes sure clusterSetRNGStream()
has the expected effect although
futures are stateless.
clusterExport
parallel::clusterExport()
assign values to the cluster nodes.
Specifically, these values are recorded and are used as globals
for all futures created there on.
Examples
plan(multisession)
cl <- makeClusterFuture()
parallel::clusterSetRNGStream(cl)
y <- parallel::parLapply(cl, 11:13, function(x) {
message("Process ID: ", Sys.getpid())
mean(rnorm(n = x))
})
str(y)
plan(sequential)
Mandelbrot convergence counts
Description
Mandelbrot convergence counts
Usage
mandelbrot(...)
## S3 method for class 'matrix'
mandelbrot(Z, maxIter = 200L, tau = 2, ...)
## S3 method for class 'numeric'
mandelbrot(
xmid = -0.75,
ymid = 0,
side = 3,
resolution = 400L,
maxIter = 200L,
tau = 2,
...
)
Arguments
Z |
A complex matrix for which convergence counts should be calculated. |
maxIter |
Maximum number of iterations per bin. |
tau |
A threshold; the radius when calling divergence (Mod(z) > tau). |
xmid , ymid , side , resolution |
Alternative specification of
the complex plane |
Value
Returns an integer matrix (of class Mandelbrot) with non-negative counts.
Author(s)
The internal Mandelbrot algorithm was inspired by and adopted from similar GPL code of Martin Maechler available from ftp://stat.ethz.ch/U/maechler/R/ on 2005-02-18 (sic!).
Examples
counts <- mandelbrot(xmid = -0.75, ymid = 0, side = 3)
str(counts)
## Not run:
plot(counts)
## End(Not run)
## Not run:
demo("mandelbrot", package = "future", ask = FALSE)
## End(Not run)
Create a multicore future whose value will be resolved asynchronously in a forked parallel process
Description
WARNING: This function must never be called.
It may only be used with plan()
Usage
multicore(
...,
workers = availableCores(constraints = "multicore"),
gc = FALSE,
earlySignal = FALSE,
envir = parent.frame()
)
Arguments
workers |
The number of parallel processes to use. If a function, it is called without arguments when the future is created and its value is used to configure the workers. |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
envir |
The environment from where global objects should be identified. |
... |
Additional named elements to |
Details
A multicore future is a future that uses multicore evaluation, which means that its value is computed and resolved in parallel in another process.
This function is must not be called directly. Instead, the typical usages are:
# Evaluate futures in parallel on the local machine via as many forked # processes as available to the current R process plan(multicore) # Evaluate futures in parallel on the local machine via two forked processes plan(multicore, workers = 2)
Value
A Future.
If workers == 1
, then all processing using done in the
current/main R session and we therefore fall back to using a
sequential future. To override this fallback, use workers = I(1)
.
This is also the case whenever multicore processing is not supported,
e.g. on Windows.
Support for forked ("multicore") processing
Not all operating systems support process forking and thereby not multicore
futures. For instance, forking is not supported on Microsoft Windows.
Moreover, process forking may break some R environments such as RStudio.
Because of this, the future package disables process forking also in
such cases. See parallelly::supportsMulticore()
for details.
Trying to create multicore futures on non-supported systems or when
forking is disabled will result in multicore futures falling back to
becoming sequential futures. If used in RStudio, there will be an
informative warning:
> plan(multicore) Warning message: In supportsMulticoreAndRStudio(...) : [ONE-TIME WARNING] Forked processing ('multicore') is not supported when running R from RStudio because it is considered unstable. For more details, how to control forked processing or not, and how to silence this warning in future R sessions, see ?parallelly::supportsMulticore
See Also
For processing in multiple background R sessions, see multisession futures.
For alternative future backends, see the 'A Future for R: Available Future Backends' vignette and https://www.futureverse.org/backends.html.
Use parallelly::availableCores()
to see the total number of
cores that are available for the current R session.
Use availableCores("multicore") > 1L
to check
whether multicore futures are supported or not on the current
system.
Examples
## Use multicore futures
plan(multicore)
## A global variable
a <- 0
## Create future (explicitly)
f <- future({
b <- 3
c <- 2
a * b * c
})
## A multicore future is evaluated in a separate forked
## process. Changing the value of a global variable
## will not affect the result of the future.
a <- 7
print(a)
v <- value(f)
print(v)
stopifnot(v == 0)
Create a multisession future whose value will be resolved asynchronously in a parallel R session
Description
WARNING: This function must never be called.
It may only be used with plan()
Usage
multisession(
...,
workers = availableCores(),
lazy = FALSE,
rscript_libs = .libPaths(),
gc = FALSE,
earlySignal = FALSE,
envir = parent.frame()
)
Arguments
workers |
The number of parallel processes to use. If a function, it is called without arguments when the future is created and its value is used to configure the workers. |
lazy |
If FALSE (default), the future is resolved eagerly (starting immediately), otherwise not. |
rscript_libs |
A character vector of R package library folders that
the workers should use. The default is |
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
envir |
The environment from where global objects should be identified. |
... |
Additional arguments passed to |
Details
A multisession future is a future that uses multisession evaluation, which means that its value is computed and resolved in parallel in another R session.
This function is must not be called directly. Instead, the typical usages are:
# Evaluate futures in parallel on the local machine via as many background # processes as available to the current R process plan(multisession) # Evaluate futures in parallel on the local machine via two background # processes plan(multisession, workers = 2)
The background R sessions (the "workers") are created using
makeClusterPSOCK()
.
For the total number of
R sessions available including the current/main R process, see
parallelly::availableCores()
.
A multisession future is a special type of cluster future.
Value
A MultisessionFuture.
If workers == 1
, then all processing is done in the
current/main R session and we therefore fall back to using a
lazy future. To override this fallback, use workers = I(1)
.
See Also
For processing in multiple forked R sessions, see multicore futures.
Use parallelly::availableCores()
to see the total number of
cores that are available for the current R session.
Examples
## Use multisession futures
plan(multisession)
## A global variable
a <- 0
## Create future (explicitly)
f <- future({
b <- 3
c <- 2
a * b * c
})
## A multisession future is evaluated in a separate R session.
## Changing the value of a global variable will not affect
## the result of the future.
a <- 7
print(a)
v <- value(f)
print(v)
stopifnot(v == 0)
## Explicitly close multisession workers by switching plan
plan(sequential)
Get the number of workers available
Description
Get the number of workers available
Usage
nbrOfWorkers(evaluator = NULL)
nbrOfFreeWorkers(evaluator = NULL, background = FALSE, ...)
Arguments
evaluator |
A future evaluator function.
If NULL (default), the current evaluator as returned
by |
background |
If TRUE, only workers that can process a future in the background are considered. If FALSE, also workers running in the main R process are considered, e.g. when using the 'sequential' backend. |
... |
Not used; reserved for future use. |
Value
nbrOfWorkers()
returns a positive number in {1, 2, 3, ...}
, which
for some future backends may also be +Inf
.
nbrOfFreeWorkers()
returns a non-negative number in
{0, 1, 2, 3, ...}
which is less than or equal to nbrOfWorkers()
.
Examples
plan(multisession)
nbrOfWorkers() ## == availableCores()
plan(sequential)
nbrOfWorkers() ## == 1
Creates a connection to the system null device
Description
Creates a connection to the system null device
Usage
nullcon()
Value
Returns a open, binary base::connection()
.
Functions Moved to 'parallelly'
Description
The following function used to be part of future but has since been migrated to parallelly. The migration started with future 1.20.0 (November 2020). They were moved because they are also useful outside of the future framework.
Details
If you are using any of these from the future package, please switch to use the ones from the parallelly package. Thank you!
-
parallelly::autoStopCluster()
(no longer re-exported) -
parallelly::makeClusterMPI()
(no longer re-exported) -
parallelly::makeNodePSOCK()
(no longer re-exported)
For backward-compatible reasons, some of these functions remain available as exact copies also from this package (as re-exports), e.g.
cl <- parallelly::makeClusterPSOCK(2)
can still be accessed as:
cl <- future::makeClusterPSOCK(2)
Note that it is the goal to remove all of the above from this package.
Writes and Reads 'immediateCondition' RDS Files
Description
Writes and Reads 'immediateCondition' RDS Files
Usage
readImmediateConditions(
path = immediateConditionsPath(rootPath = rootPath),
rootPath = tempdir(),
pattern = "[.]rds$",
include = getOption("future.relay.immediate", "immediateCondition"),
signal = FALSE,
remove = TRUE
)
saveImmediateCondition(
cond,
path = immediateConditionsPath(rootPath = rootPath),
rootPath = tempdir()
)
Arguments
path |
(character string) The folder where the RDS files are. |
pattern |
(character string) A regular expression selecting the RDS files to be read. |
include |
(character vector) The class or classes of the objects to be kept. |
signal |
(logical) If TRUE, the condition read are signaled. |
remove |
(logical) If TRUE, the RDS files used are removed on exit. |
cond |
A condition of class |
Value
readImmediateConditions()
returns an unnamed base::list of
named lists with elements condition
and signaled
, where
the condition
elements hold immediateCondition
objects.
saveImmediateCondition()
returns, invisibly, the pathname of
the RDS written.
Request a core for multicore processing
Description
If no cores are available, the current process blocks until a core is available.
Usage
requestCore(
await,
workers = availableCores(constraints = "multicore"),
timeout,
delta,
alpha
)
Arguments
await |
A function used to try to "collect" finished multicore subprocesses. |
workers |
Total number of workers available. |
timeout |
Maximum waiting time (in seconds) allowed before a timeout error is generated. |
delta |
Then base interval (in seconds) to wait between each try. |
alpha |
A multiplicative factor used to increase the wait interval after each try. |
Value
Invisible TRUE. If no cores are available after extensive waiting, then a timeout error is thrown.
Reset a finished, failed, canceled, or interrupted future to a lazy future
Description
A future that has successfully completed, canceled, interrupted, or has failed due to an error, can be relaunched after resetting it.
Usage
reset(x, ...)
Arguments
x |
A Future. |
... |
Not used. |
Details
A lazy, vanilla Future can be reused in another R session. For instance, if we do:
library(future) a <- 2 f <- future(42 * a, lazy = TRUE) saveRDS(f, "myfuture.rds")
Then we can read and evaluate the future in another R session using:
library(future) f <- readRDS("myfuture.rds") v <- value(f) print(v) #> [1] 84
Value
reset()
returns a lazy, vanilla Future that can be relaunched.
Resetting a running future results in a FutureError.
Examples
## Like mean(), but fails 90% of the time
shaky_mean <- function(x) {
if (as.double(Sys.time()) %% 1 < 0.90) stop("boom")
mean(x)
}
x <- rnorm(100)
## Calculate the mean of 'x' with a risk of failing randomly
f <- future({ shaky_mean(x) })
## Relaunch until success
repeat({
v <- tryCatch(value(f), error = identity)
if (!inherits(v, "error")) break
message("Resetting failed future, and retry in 0.1 seconds")
f <- reset(f)
Sys.sleep(0.1)
})
cat("mean:", v, "\n")
Free up active background workers
Description
Free up active background workers
Usage
resetWorkers(x, ...)
Arguments
x |
A FutureStrategy. |
... |
Not used. |
Details
This function will resolve any active futures that is currently being evaluated on background workers.
Examples
resetWorkers(plan())
Resolve one or more futures synchronously
Description
This function provides an efficient mechanism for waiting for multiple futures in a container (e.g. list or environment) to be resolved while in the meanwhile retrieving values of already resolved futures.
Usage
resolve(
x,
idxs = NULL,
recursive = 0,
result = FALSE,
stdout = FALSE,
signal = FALSE,
force = FALSE,
sleep = getOption("future.wait.interval", 0.01),
...
)
Arguments
x |
A Future to be resolved, or a list, an environment, or a list environment of futures to be resolved. |
idxs |
(optional) integer or logical index specifying the subset of elements to check. |
recursive |
A non-negative number specifying how deep of a recursion should be done. If TRUE, an infinite recursion is used. If FALSE or zero, no recursion is performed. |
result |
(internal) If TRUE, the results are retrieved, otherwise not. Note that this only collects the results from the parallel worker, which can help lower the overall latency if there are multiple concurrent futures. This does not return the collected results. |
stdout |
(internal) If TRUE, captured standard output is relayed, otherwise not. |
signal |
(internal) If TRUE, captured conditions are relayed, otherwise not. |
force |
(internal) If TRUE, captured standard output and captured conditions already relayed is relayed again, otherwise not. |
sleep |
Number of seconds to wait before checking if futures have been resolved since last time. |
... |
Not used. |
Details
This function is resolves synchronously, i.e. it blocks until x
and
any containing futures are resolved.
Value
Returns x
(regardless of subsetting or not).
If signal
is TRUE and one of the futures produces an error, then
that error is produced.
See Also
To resolve a future variable, first retrieve its
Future object using futureOf()
, e.g.
resolve(futureOf(x))
.
Check whether a future is resolved or not
Description
Check whether a future is resolved or not
Usage
resolved(x, ...)
Arguments
x |
A Future, a list, or an environment (which also includes list environment). |
... |
Not used. |
Details
This method needs to be implemented by the class that implement
the Future API. The implementation should return either TRUE or FALSE
and must never throw an error (except for FutureError:s which indicate
significant, often unrecoverable infrastructure problems).
It should also be possible to use the method for polling the
future until it is resolved (without having to wait infinitely long),
e.g. while (!resolved(future)) Sys.sleep(5)
.
Value
A logical of the same length and dimensions as x
.
Each element is TRUE unless the corresponding element is a
non-resolved future in case it is FALSE.
Get the results of a resolved future
Description
Get the results of a resolved future
Usage
## S3 method for class 'Future'
result(future, ...)
Arguments
future |
A Future. |
... |
Not used. |
Details
This function is only part of the backend Future API. This function is not part of the frontend Future API.
Value
The FutureResult object.
Run a future
Description
Run a future
Usage
## S3 method for class 'Future'
run(future, ...)
Arguments
future |
A Future. |
... |
Not used. |
Details
This function can only be called once per future. Further calls will result in an informative error. If a future is not run when its value is queried, then it is run at that point.
Value
The Future object.
Robustly Saves an Object to RDS File Atomically
Description
Robustly Saves an Object to RDS File Atomically
Usage
save_rds(object, pathname, ...)
Arguments
object |
The R object to be save. |
pathname |
RDS file to written. |
... |
(optional) Additional arguments passed to |
Details
Uses base::saveRDS internally but writes the object atomically by first writing to a temporary file which is then renamed.
Value
The pathname of the RDS written.
Create a sequential future whose value will be in the current R session
Description
WARNING: This function must never be called.
It may only be used with plan()
Usage
sequential(..., gc = FALSE, earlySignal = FALSE, envir = parent.frame())
Arguments
gc |
If TRUE, the garbage collector run (in the process that
evaluated the future) only after the value of the future is collected.
Exactly when the values are collected may depend on various factors such
as number of free workers and whether |
earlySignal |
Specified whether conditions should be signaled as soon as possible or not. |
envir |
The environment from where global objects should be identified. |
... |
Additional named elements to |
Details
A sequential future is a future that is evaluated sequentially in the current R session similarly to how R expressions are evaluated in R. The only difference to R itself is that globals are validated by default just as for all other types of futures in this package.
This function is must not be called directly. Instead, the typical usages are:
# Evaluate futures sequentially in the current R process plan(sequential)
Value
A Future.
Examples
## Use sequential futures
plan(sequential)
## A global variable
a <- 0
## Create a sequential future
f <- future({
b <- 3
c <- 2
a * b * c
})
## Since 'a' is a global variable in future 'f' which
## is eagerly resolved (default), this global has already
## been resolved / incorporated, and any changes to 'a'
## at this point will _not_ affect the value of 'f'.
a <- 7
print(a)
v <- value(f)
print(v)
stopifnot(v == 0)
Outputs details on the current R session
Description
Outputs details on the current R session
Usage
sessionDetails(env = FALSE)
Arguments
env |
If TRUE, |
Value
Invisibly a list of all details.
Signals Captured Conditions
Description
Captured conditions that meet the include
and exclude
requirements are signaled in the order as they were captured.
Usage
signalConditions(
future,
include = "condition",
exclude = NULL,
resignal = TRUE,
...
)
Arguments
future |
A resolved Future. |
include |
A character string of condition classes to signal. |
exclude |
A character string of condition classes not to signal. |
resignal |
If TRUE, then already signaled conditions are signaled again, otherwise not. |
... |
Not used. |
Value
Returns the Future where conditioned that were signaled have been flagged to have been signaled.
See Also
Conditions are signaled by
signalCondition()
.
Place a sticky-globals environment immediately after the global environment
Description
Place a sticky-globals environment immediately after the global environment
Usage
sticky_globals(erase = FALSE, name = "future:sticky_globals", pos = 2L)
Arguments
erase |
(logical) If TRUE, the environment is erased, otherwise not. |
name |
(character) The name of the environment on the base::search path. |
pos |
(integer) The position on the search path where the
environment should be positioned. If |
Value
(invisible; environment) The environment.
Get number of cores currently used
Description
Get number of children (and don't count the current process) used by the current R session. The number of children is the total number of subprocesses launched by this process that are still running and whose values have yet not been collected.
Usage
usedCores()
Value
A non-negative integer.
The value of a future or the values of all elements in a container
Description
Gets the value of a future or the values of all elements (including futures) in a container such as a list, an environment, or a list environment. If one or more futures is unresolved, then this function blocks until all queried futures are resolved.
Usage
value(...)
## S3 method for class 'Future'
value(future, stdout = TRUE, signal = TRUE, drop = FALSE, ...)
## S3 method for class 'list'
value(
x,
idxs = NULL,
recursive = 0,
reduce = NULL,
stdout = TRUE,
signal = TRUE,
cancel = TRUE,
interrupt = cancel,
inorder = TRUE,
drop = FALSE,
force = TRUE,
sleep = getOption("future.wait.interval", 0.01),
...
)
## S3 method for class 'listenv'
value(
x,
idxs = NULL,
recursive = 0,
reduce = NULL,
stdout = TRUE,
signal = TRUE,
cancel = TRUE,
interrupt = cancel,
inorder = TRUE,
drop = FALSE,
force = TRUE,
sleep = getOption("future.wait.interval", 0.01),
...
)
## S3 method for class 'environment'
value(x, ...)
Arguments
future , x |
A Future, an environment, a list, or a list environment. |
stdout |
If TRUE, standard output captured while resolving futures is relayed, otherwise not. |
signal |
If TRUE, conditions captured while resolving futures are relayed, otherwise not. |
drop |
If TRUE, resolved futures are minimized in size and invalidated
as soon the as their values have been collected and any output and
conditions have been relayed.
Combining |
idxs |
(optional) integer or logical index specifying the subset of elements to check. |
recursive |
A non-negative number specifying how deep of a recursion should be done. If TRUE, an infinite recursion is used. If FALSE or zero, no recursion is performed. |
reduce |
An optional function for reducing all the values.
Optional attribute |
cancel , interrupt |
If TRUE and |
inorder |
If TRUE, then standard output and conditions are relayed,
and value reduction, is done in the order the futures occur in |
force |
(internal) If TRUE, captured standard output and captured conditions already relayed is relayed again, otherwise not. |
sleep |
Number of seconds to wait before checking if futures have been resolved since last time. |
... |
All arguments used by the S3 methods. |
Value
value()
of a Future object returns the value of the future, which can
be any type of R object.
value()
of a list, an environment, or a list environment returns an
object with the same number of elements and of the same class.
Names and dimension attributes are preserved, if available.
All future elements are replaced by their corresponding value()
values.
For all other elements, the existing object is kept as-is.
If signal
is TRUE and one of the futures produces an error, then
that error is relayed. Any remaining, non-resolved futures in x
are
canceled, prior to signaling such an error.
Examples
## ------------------------------------------------------
## A single future
## ------------------------------------------------------
x <- sample(100, size = 50)
f <- future(mean(x))
v <- value(f)
message("The average of 50 random numbers in [1,100] is: ", v)
## ------------------------------------------------------
## Ten futures
## ------------------------------------------------------
xs <- replicate(10, { list(sample(100, size = 50)) })
fs <- lapply(xs, function(x) { future(mean(x)) })
## The 10 values as a list (because 'fs' is a list)
vs <- value(fs)
message("The ten averages are:")
str(vs)
## The 10 values as a vector (by manually unlisting)
vs <- value(fs)
vs <- unlist(vs)
message("The ten averages are: ", paste(vs, collapse = ", "))
## The values as a vector (by reducing)
vs <- value(fs, reduce = `c`)
message("The ten averages are: ", paste(vs, collapse = ", "))
## Calculate the sum of the averages (by reducing)
total <- value(fs, reduce = `sum`)
message("The sum of the ten averages is: ", total)
Evaluate an expression using a temporarily set future plan
Description
This function allows the user to plan the future, more specifically,
it specifies how future()
:s are resolved,
e.g. sequentially or in parallel.
Usage
## S3 method for class 'FutureStrategyList'
with(data, expr, ..., local = FALSE, envir = parent.frame(), .cleanup = NA)
plan(
strategy = NULL,
...,
substitute = TRUE,
.skip = FALSE,
.call = TRUE,
.cleanup = NA,
.init = TRUE
)
tweak(strategy, ..., penvir = parent.frame())
Arguments
data |
The future plan to use temporarily. |
expr |
The R expression to be evaluated. |
local |
If TRUE, then the future plan specified by |
envir |
The environment where the future plan should be set and the expression evaluated. |
.cleanup |
(internal) Used to stop implicitly started clusters. |
strategy |
An existing future function or the name of one. |
substitute |
If |
.skip |
(internal) If |
.call |
(internal) Used for recording the call to this function. |
.init |
(internal) Used to initiate workers. |
penvir |
The environment used when searching for a future function by its name. |
... |
Named arguments to replace the defaults of existing arguments. |
Details
The default backend is sequential
, but another one can be set
using plan()
, e.g. plan(multisession)
will launch parallel workers
running in the background, which then will be used to resolve future.
To shut down background workers launched this way, call plan(sequential)
.
Value
The value of the expression evaluated (invisibly).
plan()
returns a the previous plan invisibly if a new future backend
is chosen, otherwise it returns the current one visibly.
a future function.
Built-in evaluation strategies
The future package provides the following built-in backends:
sequential
:-
Resolves futures sequentially in the current R process, e.g.
plan(sequential)
. multisession
:-
Resolves futures asynchronously (in parallel) in separate R sessions running in the background on the same machine, e.g.
plan(multisession)
andplan(multisession, workers = 2)
. multicore
:-
Resolves futures asynchronously (in parallel) in separate forked R processes running in the background on the same machine, e.g.
plan(multicore)
andplan(multicore, workers = 2)
. This backend is not supported on Windows. cluster
:-
Resolves futures asynchronously (in parallel) in separate R sessions running typically on one or more machines, e.g.
plan(cluster)
,plan(cluster, workers = 2)
, andplan(cluster, workers = c("n1", "n1", "n2", "server.remote.org"))
.
Other evaluation strategies available
In addition to the built-in ones, additional parallel backends are implemented in future-backend packages future.callr and future.mirai that leverage R package callr and mirai:
callr
:-
Similar to
multisession
, this resolved futures in parallel in background R sessions on the local machine via the callr package, e.g.plan(future.callr::callr)
andplan(future.callr::callr, workers = 2)
. The difference is that each future is processed in a fresh parallel R worker, which is automatically shut down as soon as the future is resolved. This can help decrease the overall memory. Moreover, contrary tomultisession
,callr
does not rely on socket connections, which means it is not limited by the number of connections that R can have open at any time. mirai_multisession
:-
Similar to
multisession
, this resolved futures in parallel in background R sessions on the local machine via the mirai package, e.g.plan(future.mirai::mirai_multisession)
andplan(future.mirai::mirai_multisession, workers = 2)
. mirai_cluster
:-
Similar to
cluster
, this resolved futures in parallel via pre-configured R mirai daemon processes, e.g.plan(future.mirai::mirai_cluster)
.
Another example is the future.batchtools package, which leverages batchtools package, to resolve futures via high-performance compute (HPC) job schedulers, e.g. LSF, Slurm, TORQUE/PBS, Grid Engine, and OpenLava;
batchtools_slurm
:-
The backend resolved futures via the Slurm scheduler, e.g.
plan(future.batchtools::batchtools_slurm)
. batchtools_torque
:-
The backend resolved futures via the TORQUE/PBS scheduler, e.g.
plan(future.batchtools::batchtools_torque)
. batchtools_sge
:-
The backend resolved futures via the Grid Engine (SGE, AGE) scheduler, e.g.
plan(future.batchtools::batchtools_sge)
. batchtools_lsf
:-
The backend resolved futures via the Load Sharing Facility (LSF) scheduler, e.g.
plan(future.batchtools::batchtools_lsf)
. batchtools_openlava
:-
The backend resolved futures via the OpenLava scheduler, e.g.
plan(future.batchtools::batchtools_openlava)
.
For package developers
Please refrain from modifying the future backend inside your packages /
functions, i.e. do not call plan()
in your code. Instead, leave
the control on what backend to use to the end user. This idea is part of
the core philosophy of the future framework—as a developer you can never
know what future backends the user have access to. Moreover, by not making
any assumptions about what backends are available, your code will also work
automatically with any new backends developed after you wrote your code.
If you think it is necessary to modify the future backend within a
function, then make sure to undo the changes when exiting the function.
This can be achieved by using with(plan(...), local = TRUE)
, e.g.
my_fcn <- function(x) { with(plan(multisession), local = TRUE) y <- analyze(x) summarize(y) }
This is important because the end-user might have already set the future strategy elsewhere for other purposes and will most likely not known that calling your function will break their setup. Remember, your package and its functions might be used in a greater context where multiple packages and functions are involved and those might also rely on the future framework, so it is important to avoid stepping on others' toes.
Using plan() in scripts and vignettes
When writing scripts or vignettes that use futures, try to place any
call to plan()
as far up (i.e. as early on) in the code as possible.
This will help users to quickly identify where the future plan is set up
and allow them to modify it to their computational resources.
Even better is to leave it to the user to set the plan()
prior to
source()
:ing the script or running the vignette.
If a ‘.future.R’ exists in the current directory and / or in
the user's home directory, it is sourced when the future package is
loaded. Because of this, the ‘.future.R’ file provides a
convenient place for users to set the plan()
.
This behavior can be controlled via an R option—see
future options for more details.
See Also
Use plan()
to set a future to become the
new default strategy.
Examples
# Evaluate a future using the 'multisession' plan
with(plan(multisession, workers = 2), {
f <- future(Sys.getpid())
w_pid <- value(f)
})
print(c(main = Sys.getpid(), worker = w_pid))
# Evaluate a future locally using the 'multisession' plan
local({
with(plan(multisession, workers = 2), local = TRUE)
f <- future(Sys.getpid())
w_pid <- value(f)
print(c(main = Sys.getpid(), worker = w_pid))
})
a <- b <- c <- NA_real_
# An sequential future
plan(sequential)
f <- future({
a <- 7
b <- 3
c <- 2
a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs
# A sequential future with lazy evaluation
plan(sequential)
f <- future({
a <- 7
b <- 3
c <- 2
a * b * c
}, lazy = TRUE)
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs
# A multicore future (specified as a string)
plan("multicore")
f <- future({
a <- 7
b <- 3
c <- 2
a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs
## Multisession futures gives an error on R CMD check on
## Windows (but not Linux or macOS) for unknown reasons.
## The same code works in package tests.
# A multisession future (specified via a string variable)
plan("future::multisession")
f <- future({
a <- 7
b <- 3
c <- 2
a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs
## Explicitly specifying number of workers
## (default is parallelly::availableCores())
plan(multicore, workers = 2)
message("Number of parallel workers: ", nbrOfWorkers())
## Explicitly close multisession workers by switching plan
plan(sequential)
Options used for futures
Description
Below are the R options and environment variables that are used by the
future package and packages enhancing it.
WARNING: Note that the names and the default values of these options may
change in future versions of the package. Please use with care until
further notice.
Packages must not change future options
Just like for other R options, as a package developer you must not change
any of the below future.*
options. Only the end-user should set these.
If you find yourself having to tweak one of the options, make sure to
undo your changes immediately afterward. For example, if you want to
bump up the future.globals.maxSize
limit when creating a future,
use something like the following inside your function:
oopts <- options(future.globals.maxSize = 1.0 * 1e9) ## 1.0 GB on.exit(options(oopts)) f <- future({ expr }) ## Launch a future with large objects
Options for controlling futures
- future.plan:
(character string or future function) Default future backend used unless otherwise specified via
plan()
. This will also be the future plan set when callingplan("default")
. If not specified, this option may be set when the future package is loaded if command-line option--parallel=ncores
(short-p ncores
) is specified; ifncores > 1
, then option future.plan is set tomultisession
otherwisesequential
(in addition to option mc.cores being set toncores
, ifncores >= 1
). (Default:sequential
)- future.globals.maxSize:
(numeric) Maximum allowed total size (in bytes) of global variables identified. This is used to protect against exporting too large objects to parallel workers by mistake. Transferring large objects over a network, or over the internet, can be slow and therefore introduce a large bottleneck that increases the overall processing time. It can also result in large egress or ingress costs, which may exist on some systems. If set of
+Inf
, then the check for large globals is skipped. (Default:500 * 1024 ^ 2
= 500 MiB)- future.globals.onReference: (beta feature - may change)
(character string) Controls whether the identified globals should be scanned for so called references (e.g. external pointers and connections) or not. It is unlikely that another R process ("worker") can use a global that uses a internal reference of the master R process—we call such objects non-exportable globals. If this option is
"error"
, an informative error message is produced if a non-exportable global is detected. If"warning"
, a warning is produced, but the processing will continue; it is likely that the future will be resolved with a run-time error unless processed in the master R process (e.g.plan(sequential)
andplan(multicore)
). If"ignore"
, no scan is performed. (Default:"ignore"
but may change)- future.resolve.recursive:
(integer) An integer specifying the maximum recursive depth to which futures should be resolved. If negative, nothing is resolved. If
0
, only the future itself is resolved. If1
, the future and any of its elements that are futures are resolved, and so on. If+Inf
, infinite search depth is used. (Default:0
)- future.onFutureCondition.keepFuture:
(logical) If
TRUE
, aFutureCondition
keeps a copy of theFuture
object that triggered the condition. IfFALSE
, it is dropped. (Default:TRUE
)- future.wait.timeout:
(numeric) Maximum waiting time (in seconds) for a future to resolve or for a free worker to become available before a timeout error is generated. (Default:
30 * 24 * 60 * 60
(= 30 days))- future.wait.interval:
(numeric) Initial interval (in seconds) between polls. This controls the polling frequency for finding an available worker when all workers are currently busy. It also controls the polling frequency of
resolve()
. (Default:0.01
= 1 ms)- future.wait.alpha:
(numeric) Positive scale factor used to increase the interval after each poll. (Default:
1.01
)
Options for built-in sanity checks
Ideally, the evaluation of a future should have no side effects. To protect against unexpected side effects, the future framework comes with a set of built-in tools for checking against this. Below R options control these built-in checks and what should happen if they fail. You may modify them for troubleshooting purposes, but please refrain from disabling these checks when there is an underlying problem that should be fixed.
Beta features: Please consider these checks to be "under construction".
- future.connections.onMisuse:
(character string) A future must close any connections it opens and must not close connections it did not open itself. If such misuse is detected and this option is set to
"error"
, then an informative error is produced. If it is set to"warning"
, a warning is produced. If"ignore"
, no check is performed. (Default:"warning"
)- future.defaultDevice.onMisuse:
(character string) A future must open graphics devices explicitly, if it creates new plots. It should not rely on the default graphics device that is given by R option
"default"
, because that rarely does what is intended. If such misuse is detected and this option is set to"error"
, then an informative error is produced. If it is set to"warning"
, a warning is produced. If"ignore"
, no check is performed. (Default:"warning"
)- future.devices.onMisuse:
(character string) A future must close any graphics devices it opens and must not close devices it did not open itself. If such misuse is detected and this option is set to
"error"
, then an informative error is produced. If it is set to"warning"
, a warning is produced. If"ignore"
, no check is performed. (Default:"warning"
)- future.globalenv.onMisuse:
(character string) Assigning variables to the global environment for the purpose of using the variable at a later time makes no sense with futures, because the next the future may be evaluated in different R process. To protect against mistakes, the future framework attempts to detect when variables are added to the global environment. If this is detected, and this option is set to
"error"
, then an informative error is produced. If"warning"
, then a warning is produced. If"ignore"
, no check is performed. (Default:"ignore"
)- future.rng.onMisuse:
(character string) If random numbers are used in futures, then parallel RNG should be declared in order to get statistical sound RNGs. You can declare this by specifying future argument
seed = TRUE
. The defaults in the future framework assume that no random number generation (RNG) is taken place in the future expression because L'Ecuyer-CMRG RNGs come with an unnecessary overhead if not needed. To protect against mistakes of not declaring use of the RNG, the future framework detects when random numbers were used despite not declaring such use. If this is detected, and this options is set"error"
, then an informative error is produced. If"warning"
, then a warning is produced. If"ignore"
, no check is performed. (Default:"warning"
)
Options for debugging futures
- future.debug:
(logical) If
TRUE
, extensive debug messages are generated. (Default:FALSE
)
Options for controlling package startup
- future.startup.script:
(character vector or a logical) Specifies zero of more future startup scripts to be sourced when the future package is attached. It is only the first existing script that is sourced. If none of the specified files exist, nothing is sourced—there will be neither a warning nor an error. If this option is not specified, environment variable R_FUTURE_STARTUP_SCRIPT is considered, where multiple scripts may be separated by either a colon (
:
) or a semicolon (;
). If neither is set, or either is set toTRUE
, the default is to look for a ‘.future.R’ script in the current directory and then in the user's home directory. To disable future startup scripts, set the option or the environment variable toFALSE
. Importantly, this option is always set toFALSE
if the future package is loaded as part of a future expression being evaluated, e.g. in a background process. In order words, they are sourced in the main R process but not in future processes. (Default:TRUE
in main R process andFALSE
in future processes / during future evaluation)- future.cmdargs:
(character vector) Overrides
commandArgs()
when the future package is loaded.
Options for configuring low-level system behaviors
- future.fork.multithreading.enable (beta feature - may change):
(logical) Enable or disable multi-threading while using forked parallel processing. If
FALSE
, different multi-thread library settings are overridden such that they run in single-thread mode. Specifically, multi-threading will be disabled for OpenMP (which requires the RhpcBLASctl package) and for RcppParallel. IfTRUE
, or not set (the default), multi-threading is allowed. Parallelization via multi-threaded processing (done in native code by some packages and external libraries) while at the same time using forked (aka "multicore") parallel processing is known to unstable. Note that this is not only true when usingplan(multicore)
but also when using, for instance,mclapply()
of the parallel package. (Default: not set)- future.output.windows.reencode:
(logical) Enable or disable re-encoding of UTF-8 symbols that were incorrectly encoded while captured. In R (< 4.2.0) and on older versions of MS Windows, R cannot capture UTF-8 symbols as-is when they are captured from the standard output. For examples, a UTF-8 check mark symbol (
"\u2713"
) would be relayed as"<U+2713>"
(a string with eight ASCII characters). Setting this option toTRUE
will causevalue()
to attempt to recover the intended UTF-8 symbols from<U+nnnn>
string components, if, and only if, the string was captured by a future resolved on MS Windows. (Default:TRUE
)
Options for demos
- future.demo.mandelbrot.region:
(integer) Either a named list of
mandelbrot()
arguments or an integer in {1, 2, 3} specifying a predefined Mandelbrot region. (Default:1L
)- future.demo.mandelbrot.nrow:
(integer) Number of rows and columns of tiles. (Default:
3L
)
Deprecated or for internal prototyping
The following options exists only for troubleshooting purposes and must not be used in production. If used, there is a risk that the results are non-reproducible if processed elsewhere. To lower the risk of them being used by mistake, they are marked as deprecated and will produce warnings if set.
- future.globals.onMissing:
(character string) Action to take when non-existing global variables ("globals" or "unknowns") are identified when the future is created. If
"error"
, an error is generated immediately. If"ignore"
, no action is taken and an attempt to evaluate the future expression will be made. The latter is useful when there is a risk for false-positive globals being identified, e.g. when future expression contains non-standard evaluation (NSE). (Default:"ignore"
)- future.globals.method:
(character string) Method used to identify globals. For details, see
globalsOf()
. (Default:"ordered"
)- future.globals.resolve:
(logical) If
TRUE
, globals that areFuture
objects (typically created as explicit futures) will be resolved and have their values (usingvalue()
) collected. Because searching for unresolved futures among globals (including their content) can be expensive, the default is not to do it and instead leave it to the run-time checks that assert proper ownership when resolving futures and collecting their values. (Default:FALSE
)
Environment variables that set R options
All of the above R future.* options can be set by corresponding
environment variable R_FUTURE_* when the future package is
loaded. This means that those environment variables must be set before
the future package is loaded in order to have an effect.
For example, if R_FUTURE_RNG_ONMISUSE="ignore"
, then option
future.rng.onMisuse is set to "ignore"
(character string).
Similarly, if R_FUTURE_GLOBALS_MAXSIZE="50000000"
, then option
future.globals.maxSize is set to 50000000
(numeric).
Options moved to the 'parallelly' package
Several functions have been moved to the parallelly package:
The options and environment variables controlling those have been adjusted accordingly to have different prefixes. For example, option future.fork.enable has been renamed to parallelly.fork.enable and the corresponding environment variable R_FUTURE_FORK_ENABLE has been renamed to R_PARALLELLY_FORK_ENABLE. For backward compatibility reasons, the parallelly package will support both versions for a long foreseeable time. See the parallelly::parallelly.options page for the settings.
See Also
To set R options or environment variables when R starts (even before the future package is loaded), see the Startup help page. The startup package provides a friendly mechanism for configurating R's startup process.
Examples
# Allow at most 5 MB globals per futures
options(future.globals.maxSize = 5e6)
# Be strict; catch all RNG mistakes
options(future.rng.onMisuse = "error")