API Reference¶

class libinsdb.DataFile(uuid: UUID, name: str, upload_date: datetime, metadata: dict[str, Any], data_file_local_path: Path | None, data_file_download_url: Path | None, quantity: UUID, spec_version: str, dependencies: set[UUID], plot_file_local_path: Path | None, plot_mime_type: str, comment: str, release_tags: set[str] | None = None)¶

Bases: object

A data file stored in the InstrumentDB database.

Data files belong to Quantity objects. They usually point to an actual local file, whose type is determined by a FormatSpecification object pointed by its Quantity owner. In some cases, there are no actual files connected with a DataFile: all the information is stored in the metadata field (see below).

Fields:

uuid: a sequence of bytes uniquely identifying this resource (uuid.UUID type)
name: the name (and possibly the extension) of the file
upload_date: a datetime object; data files belonging to the same quantity are sorted according to this field, which must be always set.
metadata: a dictionary associating keys (strings) to values. The actual contents of this dictionary depend on the quantity, and they should be specified by the corresponding FormatSpecification object.
data_file_local_path: a pathlib.Path object pointing to the data file, or None if no data file exists.
quantity: the UUID of the Quantity object that «owns» this data file.
spec_version: a string representing the version number of the specification document associated with the corresponding FormatSpecification object. There are no constraints on the way this string is formatted.
dependencies: a set object containing the UUIDs of other data files that have been used to create this very file.
plot_file_local_path: either a pathlib.Path object pointing to an image file that contains a plot of the quantities in the data file, or None.
plot_mime_type: the MIME type of the image file pointed by plot_file_name.
comment: a string containing free-form comments related to this data file.

open_data_file(database: Any)¶

class libinsdb.Entity(uuid: UUID, name: str, full_path: str | None = None, parent: UUID | None = None, quantities: set[UUID] | None = None)¶

Bases: object

An entity describing some part of the experiment.

Entities are a generic concept that is used by InstrumentDB to group quantities related to the same part of the instrument, e.g., detector, HWP, telescope, …

Fields:

uuid: a sequence of bytes uniquely identifying this resource (uuid.UUID type)
name: a string descriptive name of the quantity. It can contain only letters, numbers, or the characters _ (underscore) and - (hyphen).
full_path: a string containing the full path of this entity, considering also the parents in the hierarchical tree of entries. This is filled only when accessing local copies of the database, as it would be too costly to navigate the tree when using remote connections.
parent: the UUID of the parent entity, or None if this is a root entity.
quantities: a set object containing the UUID of each quantity belonging to this entity (see the Quantity class).

class libinsdb.FormatSpecification(uuid: UUID, document_ref: str, title: str, local_doc_file_path: Path | None, doc_mime_type: str, file_mime_type: str)¶

Bases: object

A format specification for a quantity in the database.

Format specifications are document that detail the file format used to encode a quantity in a file or in a dictionary (metadata).

Any Quantity object must point to a valid FormatSpecification object: in this way, the database ensures that the data in the database can be interpreted by users.

Fields of this class:

uuid: a sequence of bytes uniquely identifying this resource (uuid.UUID type)
document_ref: an unique label identifying the document; the database does not enforce any scheme for this string (apart from its uniqueness).
title: the title of the document
doc_file_name: a pathlib.Path object pointing to a local copy of the document, or None if no document is provided.
doc_mime_type: the MIME type of the document. This specifies the format of the document (e.g., PDF, Microsoft Word, etc.), and it can be useful when prompting the user to open it
file_mime_type: the MIME type of the file being described in this specification. For instance, if the document describes the format used to save Healpix maps in FITS files (columns, measure units, etc.), the value of file_mime_type must be application/fits.

exception libinsdb.InstrumentDbConnectionError(response: Response, message: str)¶

Bases: Exception

Exception raised when there are problems communicating with a remote database

The fields of this class contain details about the failed requests:

response: the requests.Response object associated with the failed
request.
message: a human-readable error message detailing what went wrong

exception libinsdb.InstrumentDbFormatError¶

Bases: Exception

Exception raised when a local InstrumentDB database has some internal error.

class libinsdb.LocalInsDb(storage_path: str | Path)¶

Bases: InstrumentDatabase

A class that interfaces with a flat-file representation of a database.

This class assumes that the storage is read-only: no change in the files is ever done!

The only argument necessary to build this class is the path to the location where the database is located. You can either specify the folder that contains the schema.json file and all the other files exported from an InstrumentDB instance, or the full path to the JSON file containing the schema.

It is not necessary that all the ancillary data are immediately available to create a LocalInsDb instance: just the JSON file containing the schema is read on the spot. Of course, if you do not have anything else than the JSON schema, you cannot call methods like query_data_file().

check_consistency() → None¶

Perform some basic checks on the structure of the flat-file database

If the structure of the folders is not compliant with the specifications, raise a InstrumentDbFormatError exception.

The checks are not comprehensive, but they should spot the most obvious errors.

merge(other: LocalInsDb) → None¶: Merge another LocalInsDb object into this one

open_data_file(data_file: DataFile) → IO¶

Open the data file for reading

This is meant to be used as a context-manager.

parse_schema(schema: dict[str, Any]) → None¶

quantity_path(uuid: UUID) → str¶

query_data_file(identifier: str | UUID, track: bool = True) → DataFile¶

Retrieve a data file

The identifier parameter can be one of the following types:

A uuid.UUID object
A string representing a UUID
A full path to an object included in a release. In this case, the path has the following form:

/relname/sequence/of/entities/…/quantity

You can check if your local copy of the database contains data files at all through the Boolean variable are_data_files_available:

db = LocalInsDb(SOME_PATH)
if not db.are_data_files_available:
    print("Error, you only have the JSON schema file!")

query_entity(identifier: UUID | str) → Entity¶: Return a Entity object from either its UUID or path

query_format_spec(identifier: UUID) → FormatSpecification¶

query_quantity(identifier: UUID | str) → Quantity¶: Return a Quantity object from either its UUID or path

query_release(tag: str) → Release¶: Retrieve a release

read_schema() → None¶

Read the JSON file containing the metadata

The schema file can be kept either in JSON or YAML format. If the schema file is compressed using GZip, this method will decompress it on the fly before parsing its contents.

class libinsdb.Quantity(uuid: UUID, name: str, format_spec: UUID | None, entity: UUID, data_files: set[UUID] | None = None)¶

Bases: object

A quantity stored in the InstrumentDB database.

Quantities are meant to gather numbers and actual information about entities (Entity) in the database. Each quantity must be connected to a FormatSpecification object, which specifies how the information has been computed and stored.

The actual information is kept in a DataFile: this further hierarchical level permits to keep several versions of the data.

uuid: a sequence of bytes uniquely identifying this resource (uuid.UUID type)
name: a string containing the name of the quantity. This must contain only letters, numbers, or the characters _ (underscore) and - (hyphen).
format_spec: either the UUID of a FormatSpecification object, or None.
entity: the UUID of a Entity object.
data_files: a set containing the UUIDs of the data files belonging to this quantity.

class libinsdb.Release(tag: str, rel_date: datetime, comment: str, data_files: set[UUID])¶

Bases: object

An official release of the database.

This class encodes a list of UUIDs for the set of DataFile objects that compose a release.

Fields:

tag: an unique string identifying the release, e.g., v1.0.
rel_date: a datetime object containing the release date.
comments: a free-form string.
data_files: a set object containing the UUIDs of the DataFile objects that make this release.

class libinsdb.RemoteInsDb(server_address: str, username: str, password: str)¶

Bases: InstrumentDatabase

A class that interfaces with a remote InstrumentDB server.

This class implements the interface of the abstract class InstrumentDatabase, which means that it is able to access the database much like the class LocalInsDb.

In addition, this class implements methods that are able to modify the content of the database, either by patching what is already saved in the database, by adding new objects, or by deleting existing ones.

create_data_file(quantity: str, parent_path: str, data_file_path: Path | None = None, data_file_name: str | None = None, plot_file_path: Path | None = None, plot_file: BufferedReader | None = None, plot_file_name: str | None = None, plot_mime_type: str | None = None, upload_date: str | None = None, spec_version: str = '1.0', metadata: Any = None, comment: str = '', dependencies: list[str] | None = None) → str¶

Add a new data file to the database.

This creates a data file for quantity (the name of the quantity holding this file, not its URL!) within the entity whose path is parent_path.

All the other parameters are the same accepted by the RESTFul API exported by InstrumentDB to create a data file.

Return the URL of the new data file.

create_entity(name: str, parent_path: str | None = None) → str¶

Add a new entity to the database

This creates an entity named name, which is a child of the entity whose path is parent_path (a full path, not just the name of the parent entity!). If the latter is None, the entity is placed at the topmost position in the tree (root).

Return the URL of the new entity.

create_format_spec(document_ref: str, document_title: str, document_file: IO, document_file_name: str, document_mime_type: str, file_mime_type: str) → str¶

Add a new format specification to the database

Each of the parameters to this method match the parameters needed by the API to create a format specification.

Return the URL of the new format specification.

create_quantity(name: str, parent_path: str, format_spec_url: str) → str¶

Add a new quantity to the database

This creates a quantity named name, which is placed within the entity whose path is parent_path (a full path, not just the name of the parent entity!). The parameter format_spec_url must be the URL of a format specification, which must already exist.

Return the URL of the new quantity.

create_release(release_tag: str, data_file_url_list: list[str] | None = None, release_date: str | None = None, release_document_path: Path | None = None, release_document_mime_type: str | None = None, comment: str = '') → str¶

Add a new release to the database.

This creates a release containing all the data files listed in data_file_url_list (list of the URLs of each data file).

All the other parameters are the same to be passed to the RESTful API provided by InstrumentDB to create a release.

Return the URL of the new release.

delete(url: str) → dict[str, Any]¶

Send a DELETE request to the server

This method should be used to remove objects (entities, quantities, data files, format specifications) from the database.

If there is an error connecting to the server, a InstrumentDBError will be raised.

get(url: str, params: Any = None) → dict[str, Any]¶

Send a GET request to the server

This method should be used to retrieve information about one or more objects in the database.

If the object requires files to be associated with it (for example, a PDF document), you must pass it through the files parameter.

If there is an error connecting to the server, a InstrumentDBError will be raised.

get_data_file_from_release(release: str, path: str)¶

open_data_file(data_file: DataFile) → IO¶: This is meant to be used as a context-manager

patch(url: str, data: dict[str, Any], files: dict[str, Any] | None = None) → dict[str, Any]¶

Send a PATCH request to the server

This method should be used to modify an existing object in the database.

If the object requires files to be associated with it (for example, a PDF document), you must pass it through the files parameter.

If there is an error connecting to the server, a InstrumentDBError will be raised.

post(url: str, data: dict[str, Any], files: dict[str, Any] | None = None) → dict[str, Any]¶

Send a POST request to the server

This method should be used to create a new object in the database, be it a format specification, an entity, a quantity, or a data file.

If the object requires files to be associated with it (for example, a PDF document), you must pass it through the files parameter.

If there is an error connecting to the server, a InstrumentDBError will be raised.

query_data_file(identifier: str | UUID, track: bool = True) → DataFile¶

Retrieve a data file

The identifier parameter can be one of the following types:

A uuid.UUID object
A string representing a UUID
A full path to an object included in a release. In this case, the path has the following form:

/relname/sequence/of/entities/…/quantity

If track is True, the UUID of the object will be saved in a set and can be retrieved using the method get_queried_data_files().

query_entity(identifier: UUID | str) → Entity¶: Return a Entity object from either its UUID or path

query_format_spec(identifier: UUID) → FormatSpecification¶

query_quantity(identifier: UUID | str) → Quantity¶: Return a Quantity object from either its UUID or path

query_release(tag: str) → Release¶: Retrieve a release

API Reference¶

Libinsdb

Navigation

Related Topics