slycat.darray

Slycat makes extensive use of darray objects - dense, multi-dimension, multi-attribute arrays - as its fundamental unit of storage and organization. In the abstract, a darray can be modeled as follows:

• A set of dimensions. Each dimension has a name, index type, and a half-open range of valid index values. Currently, the only supported index type is “int64”, and indices are all zero-based (i.e. the range always begins at zero), but these may change in the future. Collectively, the dimensions define the size and shape of the array.
• A set of attributes, each with a name and type. Allowed attribute types include a full complement of signed and unsigned fixed-width integer types, plus floating-point and string types. Collectively, attributes define what will be stored in the array.
• The array data. Because darrays are dense, the data will include one value per attribute, for every location in the array.

This definition allows darrays to be flexible and efficient - for example, a “table” data structure with heterogenous column types can be stored as a 1D darray with multiple attributes, while a “matrix” would be stored as a 2D darray with a single floating-point attribute.

Note that darrays are an abstract concept with multiple concrete representations. This module defines an abstract interface for manipulating Python darrays, and a concrete implementation with in-memory storage. The slycat.hdf5 module defines functionality for manipulating darrays stored in HDF5 files on disk, and the REST API defines functionality for working with darrays using HTTP.

Note that it is rare to manipulate entire darrays in memory at once, due to their size - most applications will work with slices of a darray to keep memory use manageable.

class slycat.darray.MemArray(dimensions, attributes, data)

Bases: slycat.darray.Stub

darray implementation that holds the full array contents in memory.

get_data(attribute=0)

Return a data slice from one attribute.

get_statistics(attribute=0)

Return statistics describing one attribute.

set_data(attribute, slice, data)

Write a data slice to one attribute.

class slycat.darray.Prototype

Bases: object

Abstract interface for all darray implementations.

attributes

Return a description of the array attributes.

dimensions

Return a description of the array dimensions.

get_data(attribute=0)

Return data from one attribute.

get_statistics(attribute=0)

Return statistics describing one attribute.

ndim

Return the number of dimensions in the array.

set_data(attribute, slice, data)

Write data to one attribute.

shape

Return the shape (size along each dimension) of the array.

size

Return the size (total number of elements) of the array.

class slycat.darray.Stub(dimensions, attributes)

Bases: slycat.darray.Prototype

darray implementation that only stores array metadata (dimensions and attributes).

attributes

Return a description of the array attributes.

dimensions

Return a description of the array dimensions.

ndim

Return the number of dimensions in the array.

shape

Return the shape (size along each dimension) of the array.

size

Return the size (total number of elements) of the array.