Callbacks

Callbacks are a way to extend the functionality of an experiment. Enabale them by passing them to the @experiment decorator.

Avaliable callbacks

class digital_experiments.callbacks.SaveLogs(name: str = 'logs.txt')

Responsible for saving logs to file

Parameters:

name (str, optional) – The name of the file to save logs to. Defaults to logs.txt.

Example

from digital_experiments import experiment, SaveLogs

@experiment(callbacks=[SaveLogs("my-logs")])
def example():
    print("hello world")

example()
id = example.observations()[-1].id
artefacts = example.artefacts(id)
# returns [Path("<some>/<path>/<id>/my-logs")]
artefacts[0].read_text()
# returns 'hello world\n'

Implementing your own callbacks

class digital_experiments.callbacks.Callback

Abstract base class for callbacks

Subclass this class and override any of the hook methods to implement relevant, custom behaviour.

setup(function: Callable) → None

Called when an experiment is first created.

start(id: str, config: dict[str, Any]) → None

Called at the start of each run of an experiment.

end(observation: Observation) → None

Called at the end of each run of an experiment. Callbacks typically modify observation.metadata to record additional information.

The callback lifecycle

We can learn about the lifecycle of a callback by implementing a simple callback of our own:

from typing import Callable, Any
from digital_experiments import Callback, Observation


class DummyCallback(Callback):
    def __init__(self):
        super().__init__()
        print("I'm being initialized")

    def setup(self, function: Callable) -> None:
        print("I'm being setup")

    def start(self, id: str, config: dict[str, Any]) -> None:
        print(f"Experiment {id} is starting")

    def end(self, observation: Observation) -> None:
        print(f"Experiment {observation.id} has ended")
        observation.metadata["dummy"] = "hello there"

Each callback to be used by an experiment is instantiated by the user and passed to the @experiment decorator:

>>> callback = DummyCallback()
I'm being initialized

When an experiment is first imported/defined, the setup method is called:

>>> @experiment(callbacks=[callback])
... def my_experiment():
...     print("my_experiment is running")
I'm being setup

Hint

The __init__ and setup steps typically occur one after the other when the experiment is decorated using compact syntax: @experiment(callbacks=[DummyCallback()])

Everytime the experiment is subsequently run, the start and end methods are called:

>>> my_experiment()
Experiment 1 is starting
my_experiment is running
Experiment 1 has ended

>>> my_experiment()
Experiment 2 is starting
my_experiment is running
Experiment 2 has ended

We can see that the end method has worked:

>>> my_experiment.observations[0].metadata["dummy"]
hello there