RuntimeContext

class tolteca.utils.RuntimeContext(config)

Bases: object

A class to manage configurations.

This class manages configurations in a coherent way for different scenarios. The constructor could take a file path that points to a YAML config file, or a tolteca working directory path containing multiple config files (most recommended, see below), or an python dict.

Depending on the type of config source (file/dir/dict), the respective ConfigBackend object (config_backend) is created and the handling of the config is done through it.

Among the three, the DirConf backend provides the most complete features and capabilities, via the notion of “tolteca workdir”. Using the workdir allows saving a snapshot of the full config in the setup_info into the setup.yaml file in the workdir. This setup info config is then used to check any incompatibility in the software or pipeline versions to ensure repeatability of the work. The factory method from_dir() can be used to create a tolteca workdir from an empty directory. The directory will be populated with pre-defined content as outlined in DirConf._contents.

From a runtime context object, the full config dict can be accessed using the config property. The properties runtime_info, config_info, and seup_info can be used to access the information that are related to the current config and setup.

The class provides a registry that maps itself to subclasses of RuntimeBase, a consumer class of this class. Objects of RuntimeBase subclasses can be constructed conveniently using the __get_item__ interface:

>>> from tolteca.simu import SimulatorRuntime
>>> from tolteca.reduce import PipelineRuntime
>>> rc = RuntimeContext('/path/to/workdir')
>>> simrt, plrt = rc[SimulatorRuntime, PipelineRuntime]

Parameters

configstr, pathlib.Path, dict

The config source, can be a file path, a directory path or a python dict.

Attributes Summary

bindir	The bin directory.
caldir	The cal directory.
config	The config dict.
config_backend	The config backend of this runtime context.
config_info	The config info.
docdir	The doc directory, available only for DirConf backend.
is_persistent	True if the runtime context is created from a file system path.
logdir	The log directory.
logger
rootpath	The path from which the runtime context is created.
runtime_info	The runtime info of the context.
setup_info	The setup info.

Methods Summary

from_dir(dirpath[, init_config])	Create RuntimeContext instance from dirpath.
get_setup_rc()	Return the runtime context constructed from the setup info.
setup([overwrite, runtime_info_only, ...])	Save the config dict to the setup info dict.
yaml_dump(config[, output])	Dump config as YAML to output
yaml_load(stream)

Attributes Documentation

bindir

The bin directory.

caldir

The cal directory.

config

The config dict.

config_backend

The config backend of this runtime context.

config_info

The config info.

docdir

The doc directory, available only for DirConf backend.

is_persistent

True if the runtime context is created from a file system path.

logdir

The log directory.

logger = <Logger RuntimeContext (DEBUG)>

rootpath

The path from which the runtime context is created.

runtime_info

The runtime info of the context.

setup_info

The setup info.

For runtime context created from a tolteca workdir, it makes use of the setup_file (the setup.yaml file) to store a copy of the config dict.

This config dict is consulted in the check() function to detect any changes in the runtime context which may affect the correctness of the program.

Methods Documentation

classmethod from_dir(dirpath, init_config=None, **kwargs)

Create RuntimeContext instance from dirpath.

This factory method is preferred when the given directory may not be already setup as tolteca workdir, in which case the setup can be done by passing create=True to this method.

For paths that have already been setup as a tolteca workdir previously, use the constructor with config=dirpath is more convenient.

Parameters

dirpathpathlib.Path, str

The path to the work directory.

init_configdict, optional

The dict to add to the setup_file.

**kwargsdict

Additional arguments passed to the underlying DirConfMixin.populate_dir().

get_setup_rc()

Return the runtime context constructed from the setup info.

setup(overwrite=False, runtime_info_only=False, setup_filepath=None, overwrite_setup_file=False, backup_setup_file=True)

Save the config dict to the setup info dict.

This will behave differently for different type of config backend.

For DirConf, the config dict is saved to the setup.yaml file, so subsequent creation of the RuntimeContext from the workdir will be able to access the config dict via the setup_info.

For FileConf and DirConf, this will only update the in-memory setup info dict and it won’t get saved persistently, unless the setup_filepath is specified, in which case the dict is saved in the specified path. The setup file can be subsequently used together with other config files using -c option for the tolteca CLI so the full check() can be performed.

Parameters

overwritebool

Set to True to force overwrite if previous setup config exists. Otherwise a RuntimeContextError is raised. Note that this is not related to whether to overwrite the setup_filepath, which can be specified by overwrite_setup_file.

runtime_info_onlybool

If True, only dump the runtime info dict to the setup config.

setup_filepathstr, pathlib.Path, optional

If set, and use this filepath to save the config dict. For DirConf config backend, this will be used instead of the default setup.yaml file in the workdir if specified. For FileConf and DictConf, this has to be a valid path in order to save the config dict.

overwrite_setup_filebool

If True, the setup file is overwritten if exists. Otherwise RuntimeContextError is raised. This is not to be confused with the overwrite option, which determines whether allowing to overwrite the setup config dict itself.

backup_setup_filebool

If True, a copy of the setup file is created. This applies to all three types of the config backends.

classmethod yaml_dump(config, output=None)

Dump config as YAML to output

Parameters

configdict

The config to write.

outputio.StringIO, optional

The object to write to. If None, return the YAML as string.

classmethod yaml_load(stream)