Skip to content

sillonlab — analysis API

Load projects and explore their runs from a script or a notebook. See Querying and analysis for worked examples.

import sillonlab as sl

Finding and loading projects

sillonlab.projects

Machine-wide project discovery — the notebook counterpart of sillon projects.

project.py is about one loaded project; this module is about finding which projects exist on this machine in the first place, so you can open one without remembering where you put it.

Both this and the CLI read the same records from silloncore.projects, so they never disagree.

list_projects(with_stats=True, include_missing=True)

Every sillon project registered on this machine.

Example
import sillonlab as sl

for p in sl.list_projects():
    print(p["project_name"], p["run_count"], p["project_path"])

Parameters:

Name Type Description Default
with_stats bool

Include run_count and last_activity, which means opening each project's database. False for a fast listing.

True
include_missing bool

Keep projects whose directory is gone, flagged exists=False, rather than hiding them — an unmounted drive should look different from a project you never had.

True

Returns:

Type Description
list

list[dict]: project_name, project_path, project_storage, exists, and when with_stats, run_count, last_activity, readable.

sillonlab

sillonlab: the analysis library of the sillon toolchain.

Load a sillon project from a python script or a jupyter notebook and explore its logged runs, the same way silloncli does from the shell.

Example
import sillonlab as sl

project = sl.load_project("path/to/project")
print(project.runs().list())

run = project.get("my_run")
print(run.parameters)
data = run.load_result("coef")

load_project(name_or_path=None)

Loads a sillon project by registered name, or by path.

With no argument, the project you are inside — searching upward, so a notebook in a subdirectory opens the project it belongs to.

A name is matched against this machine's registry, case-insensitively and by unique prefix, so the output of list_projects() can be used directly. A path wins over a name when both would match.

Example
sl.load_project()                    # the one I am in
sl.load_project("Shaking Lattice")   # by name, from anywhere
sl.load_project("~/Code/phd/shaking")

Parameters:

Name Type Description Default
name_or_path str | Path

A registered project name, a path inside a project, or a project root.

None

Raises:

Type Description
FileNotFoundError

If a path is given and no project contains it.

LookupError

If a name matches nothing, or matches several projects.

Returns:

Name Type Description
Project Project

The loaded project.

Project

sillonlab

sillonlab: the analysis library of the sillon toolchain.

Load a sillon project from a python script or a jupyter notebook and explore its logged runs, the same way silloncli does from the shell.

Example
import sillonlab as sl

project = sl.load_project("path/to/project")
print(project.runs().list())

run = project.get("my_run")
print(run.parameters)
data = run.load_result("coef")

Project

The entry point of sillonlab: a loaded sillon project.

Wraps the silloncore engine so logged runs can be explored from a python script or a jupyter notebook, the same way silloncli does from the shell.

Example
import sillonlab as sl

project = sl.load_project("path/to/project")
runs = project.runs()                # RunCollection of all runs
run = project.get("integration_alpha")
run.parameters                       # {"learning_rate": 0.01, ...}
run.load_result("final_loss")        # read back from the HDF5 glob

Attributes:

Name Type Description
path Path

The root path of the project.

storage_root Path

The folder holding the database, glob and artifact storage (the .sillon folder by default).

engine Engine

The SQLAlchemy engine connected to the project database.

show(*run_names)

Pretty-prints the project's runs, like sillon list does.

With no argument, an overview table of every run. With run names, a detail card per run. Works in a terminal and in a jupyter notebook.

Parameters:

Name Type Description Default
*run_names str

Optional run names to target.

()

runs()

Loads all the runs of the project.

Returns:

Name Type Description
RunCollection RunCollection

Lazy Run handles, sorted by timestamp.

query(has_parameter=None, has_metadata=None, has_result=None, has_analysis=None, has_artifact=None, has_tag=None, tags=None, parameters=None, metadata=None, results=None, analyses=None, fields=None, before=None, after=None, **param_conditions)

Finds the runs matching value/predicate and presence criteria.

Value conditions map a name to a plain value (equality) or a callable predicate, and can target parameters, metadata, results, or analyses. Bare keyword arguments are a shorthand for parameter conditions. fields filters on top-level columns (status, author, git hash, ...). before/after filter on the run date. Presence filters (has_*, tags) keep only runs that have the named item.

Cheap criteria (parameters, metadata, tags, date, fields, presence) are resolved entirely from the database; result/analysis value conditions read the glob, but only for runs that already passed the cheap filters.

Example
project.query(optimizer="adam")                       # param equality
project.query(learning_rate=lambda lr: lr < 0.1)      # param predicate
project.query(metadata={"sillon.language": "python"})
project.query(tags="baseline", after="2026-06-01")
project.query(fields={"status": "SUCCESS"})
project.query(results={"final_loss": lambda v: v < 0.05})
project.query(tags="prod", results={"loss": lambda v: v < 0.1})

Parameters:

Name Type Description Default
has_parameter str | list

Parameter name(s) that must exist.

None
has_metadata str | list

Metadata name(s) that must exist.

None
has_result str | list

Result/artifact name(s) that must exist.

None
has_analysis str | list

Analysis name(s) that must exist.

None
has_artifact str | list

Artifact name(s) that must exist.

None
has_tag str | list

Tag(s) the run must have.

None
tags str | list

Alias for has_tag.

None
parameters dict

Parameter value/predicate conditions.

None
metadata dict

Metadata value/predicate conditions.

None
results dict

Result value/predicate conditions.

None
analyses dict

Analysis value/predicate conditions.

None
fields dict

Conditions on top-level columns (e.g. {"status": "SUCCESS", "author": "doph"}).

None
before datetime | str

Keep runs created before this date.

None
after datetime | str

Keep runs created after this date.

None
**param_conditions

Shorthand parameter value/predicate conditions.

{}

Returns:

Name Type Description
RunCollection RunCollection

The matching runs.

get(run_name)

Loads a single run by name or uuid.

Parameters:

Name Type Description Default
run_name str

The name or uuid of the run.

required

Raises:

Type Description
LookupError

If the run does not exist in the project database.

Returns:

Name Type Description
Run Run

The loaded run handle.

add(run_names, notes=None, tags=None)

Appends notes or tags to existing runs, like sillon add.

Parameters:

Name Type Description Default
run_names str | list

The run name(s) to annotate.

required
notes str | list

Note(s) to append.

None
tags str | list

Tag(s) to append.

None

Returns:

Name Type Description
dict dict

The engine status payload of the operation.

delete_run(run)

Permanently deletes a run (its stored data and database row).

Parameters:

Name Type Description Default
run Run | str

A Run handle, or the run name/uuid to delete.

required

Returns:

Name Type Description
dict dict

{"status": "success", "deleted": str, "freed_bytes": int}, or an error status if the run does not exist.

rename(run, new_name)

Renames a run, rejecting the rename if new_name is already taken.

Parameters:

Name Type Description Default
run Run | str

A Run handle, or the run name/uuid to rename.

required
new_name str

The new run name.

required

Returns:

Name Type Description
dict dict

{"status": "success", "old": ..., "new": ...} or an error status.

trace(file_or_hash)

Finds which run(s) own a file, by its content hash.

Pass a file path (it gets hashed) or a hash string. Useful to trace a stray figure/artifact back to the run that produced it.

Parameters:

Name Type Description Default
file_or_hash str | Path

A file path or a SHA-256 hash.

required

Returns:

Type Description
list

list[dict]: One {run_name, run_uuid, kind, name} per match.

diff(run_name1, run_name2, max_bytes=None)

Compares two runs: parameters, code, results and context.

Richer than compare, which only reaches parameters, status, runtime and a source-text diff. This also reports whether the code differed in logic or only in a tuned constant, and whether the results moved — arrays by shape, dtype and hash rather than element by element.

Example
d = project.diff("baseline", "refined")
d["code"]["same_logic"]          # was this a fair comparison?
d["parameters"]["changed"]       # {key: {old, new, delta_pct}}
d["results"]["rmse"]["delta_pct"]

Parameters:

Name Type Description Default
run_name1 str

Baseline run name, uuid, or uuid prefix.

required
run_name2 str

Target run.

required
max_bytes int

Largest result to load for comparison. Bigger ones are compared by shape and dtype only.

None

Raises:

Type Description
LookupError

If either run cannot be found.

Returns:

Name Type Description
dict dict

{"runs", "parameters", "code", "results", "context"}.

diff_across(run_names=None, **query)

What varies, and what is held fixed, across a set of runs.

The question a sweep leaves behind months later: which knobs were actually turned?

Example
project.diff_across(has_tag="sweep")
# {"varying": {"degree": [1,2,3,4,5], "ridge": [0.0, 0.1]},
#  "constant": {"seed": 7, "solver": "lstsq"}, ...}

Parameters:

Name Type Description Default
run_names list[str]

Explicit runs. Omit to use every run matching the filters.

None
**query

Cheap filters (has_tag, fields, parameters, ...).

{}

Returns:

Name Type Description
dict dict

{"run_count", "varying", "constant", "logic_versions", "runs"}.

like(run, limit=10)

The runs most similar to this one, by shared configuration.

"Did I do something close to this?" — more useful than an exact-match check, which only fires on a perfect repeat and tells you nothing about the near misses.

Example
for match in project.like("happy_perlman"):
    print(f"{match['score']:.0%}", match["name"], match["changes"])

Parameters:

Name Type Description Default
run str | Run

The run to compare against.

required
limit int

How many matches to return.

10

Raises:

Type Description
LookupError

If the run cannot be found.

Returns:

Type Description
list

list[dict]: {"name", "uuid", "score", "differing", "changes", "same_code"}, best match first.

versions()

The project's runs grouped by the code version that produced them.

The notebook counterpart of sillon versions. A version covers the entry script and your own modules, hashed so a tuned constant does not read as a change — see the Code versions guide.

Example
for group in project.versions():
    print(group["logic_version"][:8], group["run_count"])

current = project.versions()[0]["logic_version"]
clean = project.query(fields={"logic_version": current})

Returns:

Type Description
list

list[dict]: {"logic_version", "run_count", "runs", "first_seen", "last_seen", "constant_variants", "files", "partial"}, most recent first.

prune(run_names=None, before=None, keep_metadata=True)

Frees disk space by deleting the stored data of selected runs.

With keep_metadata the database rows survive, so you still know what you ran and with which parameters — only the heavy data goes. Set it False to remove the rows too, which is irreversible.

Requires a scope: pass run_names or before, never neither.

Parameters:

Name Type Description Default
run_names list[str]

Runs to prune.

None
before str

Prune runs older than this date or age.

None
keep_metadata bool

Keep the database rows. Defaults to True.

True

Returns:

Name Type Description
dict dict

{"pruned", "freed_bytes", ...}.

export(run_names=None, dest=None, with_data=True, **query)

Writes runs to a portable bundle another machine can import.

A different thing from Run.report(), which produces a zip for a person to read and is lossy by design. A bundle is complete and is meant to be read back by sillon.

Example
project.export(tags="sweep", dest="/Volumes/share/")
project.export(["deg3"], dest="deg3.sillon")

Parameters:

Name Type Description Default
run_names list[str]

Explicit runs. Omit to use every run matching query, or the whole project when there is none.

None
dest str | Path

A file to write, or a directory to write a self-describing name into.

None
with_data bool

Include the heavy data. Defaults to True; False writes an index-only bundle whose runs import as stubs.

True
**query

The same cheap filters as query() (tags, before, ...).

{}

Returns:

Name Type Description
dict dict

{"path", "runs", "bytes", "with_data", "skipped_data"}.

import_from(path, with_data=True, dry_run=False)

Adds every run a source has and this project does not.

The source is a bundle file, or another project directory on this machine — crossing a machine boundary needs a bundle, merging two projects that are both right here does not.

Union, never overwrite: a run already here is left alone, and an incoming name that is taken gets suffixed with where it came from. Safe to repeat — re-importing is how you find out whether a source has anything new in it.

Example
project.import_from("lattice~laptop.sillon", dry_run=True)
project.import_from("~/old-phd/shaking")

Parameters:

Name Type Description Default
path str | Path

A bundle file, or a project directory.

required
with_data bool

Copy the heavy data. Defaults to True.

True
dry_run bool

Report what would happen and change nothing.

False

Returns:

Name Type Description
dict dict

What was (or would be) imported — see silloncore.engine.import_from.

import_tree(root, with_data=True, dry_run=False)

Finds every project under a directory and imports each of them.

For a history of projects made wherever the script happened to be. This project is left out of its own scan.

Example
project.import_tree("~/Code/phd", dry_run=True)

Parameters:

Name Type Description Default
root str | Path

The directory to search.

required
with_data bool

Copy the heavy data. Defaults to True.

True
dry_run bool

Report what would happen and change nothing.

False

Returns:

Name Type Description
dict dict

{"root", "sources", "imported", "skipped"} — see silloncore.engine.scan_and_import.

check()

Everything in the project that no longer adds up.

Runs whose data is gone, data no run claims, lineage pointing at runs that are not here, source blobs referenced but not stored, a storage root that has moved, a registry entry that disagrees.

Example
for finding in project.check()["findings"]:
    print(finding["severity"], finding["message"])

Returns:

Name Type Description
dict dict

{"project", "run_count", "findings", "counts", "healthy"}.

repair()

Applies the fixes from check() that cannot lose anything.

Re-registers the project, and deletes data and source blobs no run refers to. Never touches a run row: anything whose fix is a judgement call is reported and left alone.

Returns:

Name Type Description
dict dict

The check result, plus {"repaired", "freed_bytes", "remaining"}.

Run and RunCollection

sillonlab

sillonlab: the analysis library of the sillon toolchain.

Load a sillon project from a python script or a jupyter notebook and explore its logged runs, the same way silloncli does from the shell.

Example
import sillonlab as sl

project = sl.load_project("path/to/project")
print(project.runs().list())

run = project.get("my_run")
print(run.parameters)
data = run.load_result("coef")

Run

A lazy handle on a single logged simulation run.

The object is cheap to create: it only holds the run name and the project pointers. All the data is fetched through the silloncore engine on first access and cached, so a notebook can hold hundreds of handles for free.

Attributes:

Name Type Description
name str

The human-readable name of the run.

uuid str

The unique identifier of the run (available after the first data access).

timestamp str

The creation date of the run.

status str

The final execution status of the run.

name property writable

The run's name in this project.

Reading it loads the snapshot when the handle was built from a uuid, because until then the only name available came from somewhere else and may belong to a different run locally.

parameters property

All logged parameters of the run as a dictionary.

results property

The names of all logged results and artifacts of the run.

metadata property

All logged metadata of the run as a dictionary.

tags property

The tags attached to the run.

notes property

The notes attached to the run.

runtime property

The execution duration of the run.

figures property

The figures logged during the run, with their provenance metadata.

Each entry maps the figure name to its metadata: the used key lists the parameters/results the figure was built from.

analyses property

The post-processed analyses attached to the run, with their context.

parents()

Runs this run derives from (lineage) — as walkable Run handles.

Inheritance copies nothing; it only records the link. Walk back to read a parent's parameters, e.g. run.parents()[0].load_parameter("lr").

The edge carries both the uuid and the name the parent had when the link was made. The handle follows the uuid: a project that two machines were merged into can hold more than one run called deg3, and the name in the edge would then find the wrong one.

The raw parent edges ([{"uuid", "name"}, ...]).

children()

Runs that derive from this run (reverse lineage), as Run handles.

load_parameter(*names)

Loads one or more parameter values by name.

Parameters:

Name Type Description Default
*names str

The parameter name(s) to load.

()

Returns:

Name Type Description
Any Any

The value if one name is given, otherwise a list of values.

load_result(*names)

Loads one or more results by name, through the engine.

Heavy results saved to the HDF5 glob are read back from disk. Results logged as artifacts return the path to the stored copy. Plain values or external paths are returned as stored in the database.

Parameters:

Name Type Description Default
*names str

The result name(s) to load.

()

Returns:

Name Type Description
Any Any

The value if one name is given, otherwise a list of values.

load_metadata(*names)

Loads one or more metadata values by name, or all of them.

Parameters:

Name Type Description Default
*names str

The metadata name(s) to load. If omitted, the whole metadata dictionary is returned.

()

Returns:

Name Type Description
Any Any

The full dictionary, a single value, or a list of values.

load_artifact(name)

Resolves the on-disk location of a saved artifact.

Parameters:

Name Type Description Default
name str

The result name the artifact was logged under.

required

Returns:

Name Type Description
Path Path

The path to the artifact file, or to its folder if the artifact holds several files.

load_source()

Loads the main script source code recorded for the run.

load_figure(name)

Resolves the on-disk location of a figure logged during the run.

Parameters:

Name Type Description Default
name str

The name the figure was logged under.

required

Returns:

Name Type Description
Path Path

The path to the figure file (e.g., to open or display in a notebook with IPython.display.Image).

fetch(name, dest=None)

Fetches a result, artifact, or figure as a file on disk.

Artifacts and figures are copied as-is; glob results are saved as .npy (or .txt for strings). Use load_result instead to get the value directly in memory.

Parameters:

Name Type Description Default
name str

The result, artifact, or figure name to fetch.

required
dest str | Path

The destination directory. Defaults to the current working directory.

None

Returns:

Name Type Description
Path Path

The path of the fetched copy.

sizes()

Measures the storage footprint of every stored item of the run.

Returns:

Name Type Description
dict dict

Mapping of item names to {"kind": str, "bytes": int}.

export(dest=None, format='npz')

Exports the stored data of the run into a portable file.

Parameters:

Name Type Description Default
dest str | Path

Output file (or folder for "npy"). Defaults to <run_name>_export.<ext> in the current directory.

None
format str

One of "npz", "npy", "hdf5". The hdf5 export also embeds the parameters and run identity. Defaults to "npz".

'npz'

Returns:

Name Type Description
dict dict

{"path": Path, "exported": [names], "skipped": [names]}.

report(dest=None, with_data=False)

Writes the run up as a zip for a person to read.

Not a bundle: this is lossy on purpose and nothing reads it back. To move runs to another project or machine, see Project.export.

The report answers "what did this run use and do": a JSON manifest, a readable report.md, the recorded entry script, and optionally the run's data as HDF5. Ideal to archive a run or drop it in a thesis appendix.

Parameters:

Name Type Description Default
dest str | Path

Output zip path. Defaults to <run_name>_report.zip in the current directory.

None
with_data bool

Also embed results/analyses as HDF5. Defaults to False.

False

Returns:

Name Type Description
Path Path

The path of the written zip.

manifest()

Returns the structured run report as a dictionary (no file written).

add_note(note)

Appends one or more notes to the run.

Parameters:

Name Type Description Default
note str | list

The note(s) to append.

required

add_tag(tag)

Appends one or more tags to the run.

Parameters:

Name Type Description Default
tag str | list

The tag(s) to append.

required

add_metadata(key_or_dict, value=None)

Merges metadata into the run.

Parameters:

Name Type Description Default
key_or_dict str | dict

A metadata key, or a dictionary of metadata key/value pairs.

required
value Any

The value when a single key is given.

None

add_analysis(name, data, used=None, **info)

Attaches post-processed data to the run for later reuse.

Use this when you derive new data from the run after the fact: for example, if the simulation fitted a function, store f(x) evaluated on your grid of interest and reload it later with load_analysis.

used records which of the run's own values the analysis was computed from — the same idea as log_figure(used=...), so a derived quantity does not lose its origin the moment you compute it:

spectrum = np.fft.rfft(run.load_result("field"))
run.add_analysis("spectrum", spectrum, used=["field"])

Omit it and sillon fills it in from what this process actually loaded from this run, which is usually exactly right.

Parameters:

Name Type Description Default
name str

The analysis name.

required
data Any

The processed data to store (array-like).

required
used str | list

The run's results/parameters this was computed from. Inferred from what you loaded when omitted.

None
**info

Free-form context saved with the analysis (e.g. comment="evaluated on fine grid").

{}

Returns:

Name Type Description
dict dict

The stored analysis row (name, hash, date...).

load_analysis(name)

Loads back a post-processed analysis attached to the run.

Parameters:

Name Type Description Default
name str

The analysis name to load.

required

Returns:

Name Type Description
Any Any

The stored analysis data.

to_dataframe(metadata=False, results=False)

Builds a single-row pandas DataFrame of the run.

Parameters:

Name Type Description Default
metadata bool

If True, metadata keys are added as columns.

False
results bool

If True, results are loaded and added as columns.

False

Returns:

Type Description

pandas.DataFrame: A one-row summary of the run.

delete()

Permanently deletes this run — its stored data and database row.

This is irreversible: the run's glob, artifacts and figures are removed from disk and its database entry (with linked artifacts/figures/ analyses) is dropped. After this call the handle is stale.

Returns:

Name Type Description
dict dict

{"status": "success", "deleted": str, "freed_bytes": int}, or an error status if the run no longer exists.

rename(new_name)

Renames the run, rejecting the rename if new_name is already taken.

Parameters:

Name Type Description Default
new_name str

The new run name.

required

Returns:

Name Type Description
dict dict

{"status": "success", "old": ..., "new": ...} or an error status (e.g. the name clashes with an existing run).

show()

Pretty-prints the run as a detail card (terminal or notebook).

RunCollection

A list-like container of Run objects with analysis helpers.

list()

Returns the names of all runs in the collection.

filter(predicate)

Returns a new collection of the runs matching the predicate.

Parameters:

Name Type Description Default
predicate Callable[[Run], bool]

A function tested on each run.

required

sort_by(key, reverse=False)

Returns a new collection ordered by key.

key is either a callable taking a Run, or the name of a parameter or result to order on -- so the common question ("my five best runs") is one call:

best = project.query(status="SUCCESS").sort_by("final_loss")[:5]
worst_first = runs.sort_by(lambda r: r.runtime, reverse=True)

Runs missing the named key sort last, whatever the direction, so a partially-logged run never displaces a real result. But a key that no run has raises, rather than quietly returning the collection in the order it already had — a typo there would otherwise hand you the first five runs you happened to log and let you call them the best.

Parameters:

Name Type Description Default
key Callable[[Run], Any] | str

A key function, or a parameter or result name.

required
reverse bool

Descending order. Defaults to False.

False

Raises:

Type Description
LookupError

If no run in the collection has key.

TypeError

If the values cannot be ordered against each other.

Returns:

Name Type Description
RunCollection RunCollection

A new collection; the original is left untouched.

where(has_parameter=None, has_metadata=None, has_result=None, has_analysis=None, has_artifact=None, has_tag=None, tags=None, parameters=None, metadata=None, results=None, analyses=None, fields=None, before=None, after=None, **param_conditions)

Filters the collection on value/predicate and presence criteria.

Same criteria as Project.query (parameters, metadata, results, analyses, fields, tags, date, and has_* presence), but applied to the runs already in this collection. Cheap criteria are checked before any glob is read.

Example
runs.where(optimizer="adam", learning_rate=lambda v: v < 0.1)
runs.where(metadata={"sillon.language": "python"})
runs.where(tags="baseline", after="2026-06-01")
runs.where(results={"final_loss": lambda v: v < 0.05})

Parameters:

Name Type Description Default
has_parameter str | list

Parameter name(s) that must exist.

None
has_metadata str | list

Metadata name(s) that must exist.

None
has_result str | list

Result/artifact name(s) that must exist.

None
has_analysis str | list

Analysis name(s) that must exist.

None
has_artifact str | list

Artifact name(s) that must exist.

None
has_tag str | list

Tag(s) the run must have.

None
tags str | list

Alias for has_tag.

None
parameters dict

Parameter value/predicate conditions.

None
metadata dict

Metadata value/predicate conditions.

None
results dict

Result value/predicate conditions.

None
analyses dict

Analysis value/predicate conditions.

None
fields dict

Conditions on top-level columns.

None
before datetime | str

Keep runs created before this date.

None
after datetime | str

Keep runs created after this date.

None
**param_conditions

Shorthand parameter value/predicate conditions.

{}

Returns:

Name Type Description
RunCollection RunCollection

The matching runs.

show()

Pretty-prints the collection as a summary table.

to_dataframe(metadata=False, results=False)

Builds a pandas DataFrame summarizing the collection.

Each row is a run with its name, timestamp, status, runtime and one column per logged parameter.

Parameters:

Name Type Description Default
metadata bool

If True, metadata keys are added as columns. Defaults to False.

False
results bool

If True, results are loaded (glob reads included) and added as columns. Defaults to False.

False

Returns:

Type Description

pandas.DataFrame: The summary table, one row per run.