## Registry.Registry Registry for defining and executing computational jobs. Usage ``` python Registry.Registry( path, overwrite=False, readonly=False, backend=None, ) ``` Jobs can be registered with arbitrary parameters and subsequently executed using a configurable execution backend. The registry keeps track of completed and failed jobs and provides methods for inspecting their results. ## Parameters `path: str` Path to registry folder in the file system. `overwrite: bool = ``False` Shall the registry folder be overwriten if it already exists? Default is `False`. `readonly: bool = ``False` Is the registry in read-only mode? In read-only mode jobs cannot be added anymore. Default is `False`. `backend: RunnerBackend | None = None` Instance of a runner backend. Default is an instance of the JoblibRunnerBackend. ## Attributes `path: str` Path to registry folder in the file system. `overwrite: bool` Shall the registry folder be overwriten if it already exists? `job_collection: list[Job]` List of Jobs. Note that the first element is a dummy (always `None`) since jobs are numbered with natural numbers. `njobs: int` Number of jobs. `max_job_id: int` Highest sequential job number assigned so far. `readonly: bool` Is the registry in read-only mode? In read-only mode jobs cannot be added anymore. `design_path: str` path to experimental design file in the registry. `backend: RunnerBackend` Instance of a runner backend. Default is an instance of the JoblibRunnerBackend. ## Methods | Name | Description | |----|----| | [__repr__()](#__repr__) | Return official string representation, which is aimed at programmers as they develop and maintain a program. | | [__str__()](#__str__) | Return informal readable string representation. | | [add_jobs()](#add_jobs) | Adds jobs via a full factorial design by default. | | [clear_logs()](#clear_logs) | Clears job logs. | | [get_done()](#get_done) | Calculate job IDs of done/finished jobs. | | [get_failed()](#get_failed) | Calculate job IDs of failed jobs. | | [get_initialised()](#get_initialised) | Calculate job IDs of initialised jobs. | | [get_job()](#get_job) | Return the job with the respective job ID. | | [get_jobs()](#get_jobs) | Return jobs. | | [get_results()](#get_results) | Return job results. | | [get_running()](#get_running) | Calculate job IDs of running jobs. | | [load()](#load) | Load registry from file system. | | [load_design()](#load_design) | Read setup parameters from a CSV file. | | [pyrallex()](#pyrallex) | Run jobs in parallel. | | [reset_jobs()](#reset_jobs) | Resets jobs. | | [set_backend()](#set_backend) | Set runner backend. | | [show_logs()](#show_logs) | Display job logs to stdout. | | [size()](#size) | Return the number of jobs. | | [tag()](#tag) | Append tags to jobs. | | [touch()](#touch) | Update job statuses. | ------------------------------------------------------------------------ #### \_\_repr\_\_() Return official string representation, which is aimed at programmers as they develop and maintain a program. Usage ``` python __repr__() ``` ##### Returns `str` Technical string representation. ------------------------------------------------------------------------ #### \_\_str\_\_() Return informal readable string representation. Usage ``` python __str__() ``` ##### Returns `str` Human-readbale string representation. ------------------------------------------------------------------------ #### add_jobs() Adds jobs via a full factorial design by default. Usage ``` python add_jobs(predicate=lambda _: True, **kwargs) ``` Optionally, arbitrary configurations can be excluded via a filter function. ##### Parameters `**kwargs: dict[str, any]` dictionary of keyword arguments. `predicate: Callable[[dict[str, any]], bool] = lambda _: True` Optional function that allows to ignore certain `and to return a Boolean` `True` if the parameter combination shall be kept and `False` if it is meant to be discarded. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### clear_logs() Clears job logs. Usage ``` python clear_logs(jobids=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### get_done() Calculate job IDs of done/finished jobs. Usage ``` python get_done(jobids=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. ##### Returns `list[int]` An integer list of job IDs. ------------------------------------------------------------------------ #### get_failed() Calculate job IDs of failed jobs. Usage ``` python get_failed(jobids=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. ##### Returns `list[int]` An integer list of job IDs. ------------------------------------------------------------------------ #### get_initialised() Calculate job IDs of initialised jobs. Usage ``` python get_initialised(jobids=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. ##### Returns `list[int]` An integer list of job IDs. ------------------------------------------------------------------------ #### get_job() Return the job with the respective job ID. Usage ``` python get_job(jobid) ``` ##### Parameters `jobid: int` The ID of the job starting at 1 (not 0). ##### Returns `Job` The respective 'Job' object. ------------------------------------------------------------------------ #### get_jobs() Return jobs. Usage ``` python get_jobs(jobids=None, tags=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. `tags: list[str] = None` An optional list of tags. Defaults to no tags at all. ##### Returns `list[Job]` A list of Job objects. ------------------------------------------------------------------------ #### get_results() Return job results. Usage ``` python get_results(jobids, filter_none=True, simplify=False) ``` ##### Parameters `jobids: list[int]` List of integer job IDs, e.g., of finished jobs. `filter_none: bool = ``True` If `True` jobs that did not return anything are skipped. Default is `True`. `simplify: bool = ``False` Should the result returned per job be simplified to a single dictionary? Default is `False`. ##### Returns `list[tuple[int, dict[str, any], dict[str, any]]] | list[dict[str, any]]` A list of 3-tuples (jobid, dictionary of parameters, dictionary of results) or a list of "merged" dictionaries if the argument simplify is set to `True`. ------------------------------------------------------------------------ #### get_running() Calculate job IDs of running jobs. Usage ``` python get_running(jobids=None) ``` ##### Parameters `jobids: list[int] = None` An optional list of job IDs. If `None` (the default) all job IDs are used. ##### Returns `list[int]` An integer list of job IDs. ------------------------------------------------------------------------ #### load() Load registry from file system. Usage ``` python load(path, readonly=True) ``` ##### Parameters `path: str` Path to registry folder in the file system. `readonly: bool = ``True` Load in read-only mode? Default is `True`. ##### Returns A registry object. ------------------------------------------------------------------------ #### load_design() Read setup parameters from a CSV file. Usage ``` python load_design(path, sep=",") ``` The file-format expected is a Comma Separated Values (CSV) file. The first line is interpreted as the header line with the experimental parameter names. Each following line contains the parameter values for one experiment. ##### Parameters `path: str` Path to the design file. `sep: str = ``","` Seperator used in the CSV-file to separate columns. Default is ','. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### pyrallex() Run jobs in parallel. Usage ``` python pyrallex(runner, jobids=None, batchsize=None, shuffle=True, rerun=False) ``` ##### Parameters `runner: Callable[[int, dict[any]], bool]` The actual runner function. It expects exactly two parameters: `jobids: list[int] | None = None` Optional list of job IDs. If `None` (the default), all job IDs are used. `batchsize: int | None = None` Size of job batches. If `None` (the default) the value is set via a heuristic as follows. The `shuffle: bool = ``True` Shall jobs be shuffled randomly prior to execution? Default is `True`. Setting this value to `True` is `rerun: bool = ``False` Shall already finished jobs be re-run? Default is `False`, i.e., finished jobs are skipped. ##### Returns `tuple[int, dict[str, any], dict[str, any]]` A list of 3-tuples (jobid, dictionary of parameters, dictionary of results). ------------------------------------------------------------------------ #### reset_jobs() Resets jobs. Usage ``` python reset_jobs(jobids) ``` I.e., all results will be deleted and the job status will be reset to 'initialised'. ##### Parameters `jobids: list[int]` A list of integer job IDs. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### set_backend() Set runner backend. Usage ``` python set_backend(backend) ``` ##### Parameters `backend: RunnerBackend` Instance of a sub-class of 'RunnerBackend'. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### show_logs() Display job logs to stdout. Usage ``` python show_logs(jobids=None) ``` The function goes through all log files and prints the content to stdout as a side effect. Args: jobids: An optional list of job IDs. If ``None`` (the default) all job IDs are used. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### size() Return the number of jobs. Usage ``` python size() ``` ------------------------------------------------------------------------ #### tag() Append tags to jobs. Usage ``` python tag(jobids, tags) ``` ##### Parameters `jobids: list[int]` list of job IDs. `tags: list[str]` list of tags, i.e., simple strings. ##### Returns `Self` The callee itself (enables chaining). ------------------------------------------------------------------------ #### touch() Update job statuses. Usage ``` python touch() ``` The function goes through re-loads all jobs from their pickle files. In particular the job status is loaded. ##### Returns `Self` The callee itself (enables chaining).