Saving a simList
may not work using the standard approaches (e.g.,
save
, saveRDS
, and qs::qsave
). There are 2 primary reasons why this doesn't
work as expected: the activeBindings
that are in place within modules (these
allow the mod
and Par
to exist), and file-backed rasters. Because of these,
a user should use saveSimList
and loadSimList
(and the zipSimList
/unzipSimList
alternatives).
The most robust way if there are file-backed Raster* objects seems to be to
set fileBackend = 2
, though this may not be desirable if there are many
large Raster*
objects. When using fileBackend = 0
or fileBackend = 1
, and
when errors are noticed, please file a bug report on GitHub.
zipSimList
will save the simList
and file-backed Raster*
objects, plus,
optionally, files identified in outputs(sim)
and inputs(sim)
.
This uses Copy
under the hood, to not affect the original
simList
.
VERY experimental.
saveSimList(sim, filename, fileBackend = 0, filebackedDir = NULL, envir, ...)
zipSimList(sim, zipfile, ..., outputs = TRUE, inputs = TRUE, cache = FALSE)
Either a simList
or a character string of the name
of a simList
that can be found in envir
. Using
a character string will assign that object name to the saved
simList
, so when it is recovered it will be given that
name.
Character string with the path for saving simList
to or
reading the simList
from. Currently, only .rds
and .qs
file types are supported.
Numeric. 0
means don't do anything with
file backed rasters. Leave their file intact as is, in its place.
1
means save a copy of the file backed rasters in fileBackedDir
.
2
means move all data in file-backed rasters to memory. This
means that the objects will be part of the main qs
file
of the simList
. Default is 0
.
Only used if fileBackend
is 1.
NULL
, the default, or Character string. If NULL
, then then the
files will be copied to the directory:
file.path(dirname(filename), "rasters")
. A character string
will be interpreted as a path to copy all rasters to.
If sim
is a character string, then this must be provided.
It is the environment where the object named sim
can be found.
passed to saveSimList()
, including non-optional ones
such as filename
. Also see fileBackend
and filebackedDir
arguments in that function.
A character string indicating the filename for the zip file. Passed to zip
.
Logical. If TRUE
, all files identified in
outputs(sim)
will be included in the zip.
Logical. If TRUE
, all files identified in
inputs(sim)
will be included in the zip.
Logical. Not yet implemented. If TRUE
, all files in cachePath(sim)
will be included in the
zip archive. Defaults to FALSE
as this could be large, and may include many
out of date elements. See Details.
saveSimList()
: A saved .qs
file in filename
location.
zipSimList()
: A saved .zip
file in zipfile
location.
There is a family of 4 functions that are mutually useful for saving and
loading simList
objects and their associated files (e.g., file-backed
Raster*
, inputs
, outputs
, cache
) saveSimList()
, loadSimList()
,
zipSimList()
, unzipSimList()
There are 3 different workflows for "save - move files - load" that work in our tests:
filebackend = 0
: No renaming of file-backed rasters, on recovery attempts to rebuild
This approach is attempting to emulate a "relative filenames" approach,
i.e., attempt to treat the file-backed raster file names as if they were
relative (which they are not -- raster package forces absolute file
paths). To do this, all the renaming occurs within loadSimList
or
unzipSimList
. These function will use the paths
argument to rewrite
the paths of the files that are identified with Filenames(sim)
so that
they are in the equivalent (relative) position as they were. This will
only work if all files were in one of the paths
of the original
simList
, so that they can be matched up with the new paths
passed in
loadSimList
. This is not guaranteed to work correctly, though it works
in a wide array of testing.
zipSimList(sim, zipfile = tmpZip, filename = "sim.qs")
<- getPaths(mySim)
pths <- unzipSimList(tmpZip, paths = pths) out
filebackend = 1
: On-the-fly renaming of file-backed rasters;
Save the sim object with a filename, e.g., file
,
make a copy of all file-backed rasters to fileBackedDir
,
update all the pointers to those files so that they are correct in the raster metadata
saveSimList(sim, file = "sim.qs", fileBackend = 1, fileBackedDir = "here")
<- loadSimList(file = "sim.qs") simNew
filebackend = 2
: On-the-fly bringing to memory of all rasters
All rasters are brought to memory, and then saved into sim.qs
saveSimList(sim, file = "sim.qs", fileBackend = 2)
<- loadSimList(file = "sim.qs") simNew
If cache
is used, it is likely that it should be trimmed before
zipping, to include only cache elements that are relevant.