Title: A Framework for Parallelizing Dependent Tasks
Version: 0.5.0
Description: Mechanisms to parallelize dependent tasks in a manner that optimizes the compute resources available. It provides access to "delayed" computations, which may be parallelized using futures. It is, to an extent, a facsimile of the 'Dask' library (https://www.dask.org/), for the 'Python' language.
Depends: R (≥ 3.2.0)
Imports: R6, igraph, future, rstackdeque, rlang, data.table, visNetwork, uuid, BBmisc, progress, R.utils, R.oo
Suggests: testthat, knitr, rmarkdown, shiny
License: GPL-3
URL: https://tlverse.org/delayed/
BugReports: https://github.com/tlverse/delayed/issues
Encoding: UTF-8
VignetteBuilder: knitr
RoxygenNote: 7.2.0
NeedsCompilation: no
Packaged: 2024-04-29 17:26:53 UTC; jrcoyle
Author: Jeremy Coyle ORCID iD [aut, cre, cph], Nima Hejazi ORCID iD [ctb]
Maintainer: Jeremy Coyle <jeremyrcoyle@gmail.com>
Repository: CRAN
Date/Publication: 2024-04-29 17:40:02 UTC

Delayed class that manages dependencies and computes when necessary

Description

Delayed class that manages dependencies and computes when necessary

Examples

d <- delayed(3 + 4)
methods::is(d, "Delayed")
d$compute()

Future Delayed Jobs

Description

A Job that leverages the future framework to evaluate asynchronously.

Examples

library(future)
plan(multicore, workers = 1)
d <- delayed(3 + 4)
sched <- Scheduler$new(d, FutureJob, nworkers = 1)

Evaluation of Delayed Objects

Description

A Job encapsulates the act of evaluating a given delayed object in a particular way. SequentialJobs evaluate immediately, blocking the current process until they are complete. FutureJobs leverages future to evaluate according to the specified plan.

Scheduler class that orders compute tasks and dispatches tasks to workers

Description

Scheduler class that orders compute tasks and dispatches tasks to workers

Examples

d <- delayed(3 + 4)
sched <- Scheduler$new(d, SequentialJob)
sched$compute()

Sequential Delayed Jobs

Description

A Job that will evaluate immediately (i.e., in a sequential fashion), blocking the current process until it completes.

Examples

d <- delayed(3 + 4)
sched <- Scheduler$new(d, SequentialJob)

Bundle Delayed Objects

Description

Bundling Delayed objects builds a single Delayed object out of an arbitrary number of input Delayed objects.

Usage

bundle_delayed(delayed_list)

bundle_args(...)

Arguments

delayed_list

A list of Delayed objects to bundle into a single Delayed object

...

Ignore (this is a convenience function)

Examples

ident_fun <- function(x) {
  Sys.sleep(0.01)
  x
}
delayed_ident <- delayed_fun(ident_fun)
d_list <- lapply(1:10, delayed_ident)
d_bundle <- bundle_delayed(d_list)
d_bundle$compute(progress = FALSE)

Generates Delayed Version of an Expression

Description

A Delayed version of a function may be called to generate Delayed objects

Usage

delayed(expr, sequential = FALSE, expect_error = FALSE, timeout = NULL)

delayed_fun(fun, sequential = FALSE, expect_error = FALSE)

Arguments

expr

expression to delay

sequential

if TRUE, never parallelize this task

expect_error

if TRUE, pass error to downstream tasks instead of

timeout

specify a time limit for computation halting computation

fun

function to delay

Examples

d <- delayed(3 + 4)
d$compute()
adder <- function(x, y) {
  x + y
}
delayed_adder <- delayed_fun(adder)
z <- delayed_adder(3, 4)
z$compute()

Helper Function to Evaluate Delayed

Description

Helper Function to Evaluate Delayed

Usage

eval_delayed(to_eval, timeout = Inf)

Arguments

to_eval

a list as generated from Delayed$prepare_eval()

timeout

a timeout indicating when to terminate the job

Find error in delayed chain

Description

Searches through a network of delayed objects for the first object with state "error"

Usage

find_delayed_error(delayed_object)

Arguments

delayed_object

the object in which an error occured

Examples

delayed_error <- delayed_fun(stop)
error_message <- "this is an error"
broken_delayed <- delayed_error(error_message)
broken_delayed$expect_error <- TRUE
result <- broken_delayed$compute()

Graphical Representation of a Task Dependency Structure

Description

Graphical Representation of a Task Dependency Structure

Usage

make_graph(delayed_object, graph = NULL, level = 1)

Arguments

delayed_object

the Delayed object to graph

graph

the current graph, usually NULL

level

the level of the node to be graphed, usually NULL

Plot Method for Delayed Objects

Description

Plot Method for Delayed Objects

Usage

## S3 method for class 'Delayed'
plot(x, color = TRUE, height = "500px", width = "100%", ...)

Arguments

x

An object of class Delayed for which a task dependency graph will be generated.

color

If TRUE, color-code nodes according to status, and display legend

height

passed to visNetwork

width

passed to visNetwork

...

Additional arugments (passed to visNetwork).

Examples

adder <- function(x, y) {
  x + y
}
delayed_adder <- delayed_fun(adder)
z <- delayed_adder(3, 4)
z2 <- delayed_adder(z, 4)
z2$sequential <- TRUE
z3 <- delayed_adder(z2, z)
plot(z3)

Animated Representation a Task Dependency Structure

Description

uses shiny

Usage

plot_delayed_shiny(scheduler)

Arguments

scheduler

the scheduler to animate

Examples

## Not run: 
adder <- function(x, y) {
  x + y
}
delayed_adder <- delayed_fun(adder)
z <- delayed_adder(3, 4)
z2 <- delayed_adder(z, 4)
z2$sequential <- TRUE
z3 <- delayed_adder(z2, z)
plot_delayed_shiny(z3)

## End(Not run)