SciDataContainer API

Container classes

class scidatacontainer.Container(items: Optional[dict] = None, file: Optional[str] = None, uuid: Optional[str] = None, server: Optional[str] = None, key: Optional[str] = None, compression: int = 8, compresslevel: int = -1)

Bases: AbstractContainer

Scientific data container.

decode(data: bytes, validate: bool = True, strict: bool = True)

Take ZIP package as binary string. Read items from the package and store them in this object.

Parameters:

  • data – Bytestring containing the ZIP DataContainer.

  • validate – If true, validate the content.

  • strict – If true, validate the hash, too.

encode()

Encode container as ZIP package. Return package as binary string.

freeze()

Calculate the hash value of this container and make it static. The container cannot be modified any more when this method was called once.

hash()

Calculate and save the hash value of this container.

items()

Return this container as a dictionary of item objects (key, value) tuples.

keys() → List[str]

Return a sorted list of the full paths of all items.

Returns:

List of paths of Container items.

Return type:

List[str]

release()

Make this container mutable. If it was immutable, this method will create a new UUID and initialize the attributes replaces, createdstorageTime and modelVersion in the item “content.json”. It will also delete an existing hash and make it a new container.

upload(data: Optional[bytes] = None, server: Optional[str] = None, key: Optional[str] = None)

Create a ZIP archive of the DataContainer and upload it to a server.

If data is passed to the function, data will be written to the file. Otherwise the byte representation of the class instance will be written to the file, which is what you typically want.

Parameters:

  • data – If given, data to write to the file.

  • server – URL of the server.

  • key – API Key from the server to identify yourself.

validate_content()

Make sure that the item “content.json” exists and contains all required attributes.

validate_meta()

Make sure that the item “meta.json” exists and contains all required attributes.

values() → List

Return a list of all item objects.

Returns:

List of item objects of the Container.

Return type:

List

write(fn: str, data: Optional[bytes] = None)

Write the container to a ZIP package file.

If data is passed to the function, data will be written to the file. Otherwise the byte representation of the class instance will be written to the file, which is what you typically want.

Parameters:

  • fn – Filename of export file.

  • data – If given, data to write to the file.

class scidatacontainer.AbstractContainer(items: Optional[dict] = None, file: Optional[str] = None, uuid: Optional[str] = None, server: Optional[str] = None, key: Optional[str] = None, compression: int = 8, compresslevel: int = -1)

Bases: ABC

Scientific data container with minimal file support.

The following file types are supported:

  • .json <-> dict

  • .txt <-> str

  • .bin <-> bytes

decode(data: bytes, validate: bool = True, strict: bool = True)

Take ZIP package as binary string. Read items from the package and store them in this object.

Parameters:

  • data – Bytestring containing the ZIP DataContainer.

  • validate – If true, validate the content.

  • strict – If true, validate the hash, too.

encode()

Encode container as ZIP package. Return package as binary string.

freeze()

Calculate the hash value of this container and make it static. The container cannot be modified any more when this method was called once.

hash()

Calculate and save the hash value of this container.

items()

Return this container as a dictionary of item objects (key, value) tuples.

keys() → List[str]

Return a sorted list of the full paths of all items.

Returns:

List of paths of Container items.

Return type:

List[str]

release()

Make this container mutable. If it was immutable, this method will create a new UUID and initialize the attributes replaces, createdstorageTime and modelVersion in the item “content.json”. It will also delete an existing hash and make it a new container.

upload(data: Optional[bytes] = None, server: Optional[str] = None, key: Optional[str] = None)

Create a ZIP archive of the DataContainer and upload it to a server.

If data is passed to the function, data will be written to the file. Otherwise the byte representation of the class instance will be written to the file, which is what you typically want.

Parameters:

  • data – If given, data to write to the file.

  • server – URL of the server.

  • key – API Key from the server to identify yourself.

validate_content()

Make sure that the item “content.json” exists and contains all required attributes.

validate_meta()

Make sure that the item “meta.json” exists and contains all required attributes.

values() → List

Return a list of all item objects.

Returns:

List of item objects of the Container.

Return type:

List

write(fn: str, data: Optional[bytes] = None)

Write the container to a ZIP package file.

If data is passed to the function, data will be written to the file. Otherwise the byte representation of the class instance will be written to the file, which is what you typically want.

Parameters:

  • fn – Filename of export file.

  • data – If given, data to write to the file.

File type support

scidatacontainer.register(suffix: str, fclass: Type[AbstractFile], pclass: Optional[Type[object]] = None)

Register a suffix to a conversion class.

If the parameter class is a string, it is interpreted as known suffix and the conversion class of this suffix is registered also for the new one.

Parameters:

  • suffix – file suffix to identify this file type.

  • fclass – Conversion class derived from AbstractFile.

  • pclass – Python class that represents this object type.

Built-in conversion classes

class scidatacontainer.filebase.AbstractFile(data)

Base class for converting datatypes to their file representation.

abstract decode(data: bytes)

Decode the Container content from bytes. This is an abstract method and it neets to be overwritten by inheriting class.

abstract encode() → bytes

Encode the Container content to bytes. This is an abstract method and it needs to be overwritten by inheriting class.

Returns:

Byte string representation of the object.

Return type:

bytes

hash() → str

Return hex digest of SHA256 hash.

Returns:

Hex digest of this object as string.

Return type:

str

class scidatacontainer.filebase.BinaryFile(data)

Bases: AbstractFile

Data conversion class for a binary file.

decode(data: bytes)

Store bytes in this class.

encode() → bytes

Return byte string stored in this class.

Returns:

Byte string representation of the object.

Return type:

bytes

hash() → str

Return hex digest of SHA256 hash.

Returns:

Hex digest of this object as string.

Return type:

str

class scidatacontainer.filebase.TextFile(data)

Bases: AbstractFile

Data conversion class for a text file.

charset = 'utf8'

Character encoding used for translation from text to bytes.

Type:

charset (str)

decode(data: bytes)

Decode text from given bytes string.

encode() → bytes

Encode text to bytes string.

Returns:

Byte string representation of the object.

Return type:

bytes

hash() → str

Return hex digest of SHA256 hash.

Returns:

Hex digest of this object as string.

Return type:

str

class scidatacontainer.filebase.JsonFile(data)

Bases: AbstractFile

Data conversion class for a JSON file represented as Python dictionary.

charset = 'utf8'

Character encoding used for translation from text to bytes.

Type:

charset (str)

decode(data: bytes)

Decode dictionary from given bytes string.

encode() → bytes

Convert dictionary to pretty string representation with indentation and return it as bytes string.

Returns:

Byte string representation of the object.

Return type:

bytes

hash() → str

Return hex digest of the SHA256 hash calculated from the sorted compact representation. This should result in the same hash for semantically equal data dictionaries.

Returns:

Hex digest of this object as string.

Return type:

str

indent = 4

Indentation of exported JSON files.

Type:

indent (int)

sortit(data: Union[dict, list, tuple]) → str

Return compact string representation with keys of all sub-dictionaries sorted.

Parameters:

data – Dictionary, list or tuple to convert to string”

Returns:

String representation of data

Return type:

str

Convenience functions

scidatacontainer.timestamp() → str

Return the current ISO 8601 compatible timestamp as string.

Returns:

timestamp as string

Return type:

str

scidatacontainer.config.load_config(config_path: Optional[str] = None) → dict

Get config data from environment variables and config file.

This functions prefers values in the scidata config file and potentially overwrites values that are present as environmental variables.

Usually, users doen’t need to call this function. However, it can be used for debugging purposes if the configuration parameters are not as expected.

Parameters:

str – Path of the config file. If this is None, the default file will be used. This filename is only required for testing.

Returns:

A dictionary containing information strings with keys “author”, “email”, “server”, “key”.

Return type:

dict