ORM

The Object Relational Model (ORM) defines the mapping between python objects and the SQL database

The objects themselves act as abstractions over whatever underlying filesystem is being used. For example, a Space roughly corresponds to an independent fileset in GPFS.

Warning

In general, you should not use these objects directly - use the REST interface instead.

The REST interface will be (mostly) stable across release versions with only compatible changes being made, whereas the backend code may undergo breaking changes between releases.

ORM Diagram

ArcaPix REST API ORM Diagram

Fig 1. The Object Relational Model (ORM) Diagram

Comparison to GPFS/General Linux

To help existing GPFS and Linux systems administrators, we can roughly match the APMgmt objects and concepts in GPFS (Spectrum Scale) as follows:

  • Datastore: storage pool
  • Exposer: filesystem, samba or nfs share
  • ILM Step: file placement policy rule
  • Profiles: file placement policy
  • Space: independent fileset
  • Snapshot: snapshot
  • Template: directory structure, which can include dependent filesets

Attention

However, the APMgmt objects represent abstract concepts. How they are implemented at the filesystem layer does not match completely with GPFS, and is subject to change.

API Documentation

Datastores

class arcapix.management.orm.datastores.DataStore(*args, **kwargs)

A data store represents a piece of persistent storage which can contain 1 (or usually many more) bytes of data, and from which individual Spaces can be carved.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

datastore name

status

current status of the item

status_message

current status as a string

type

specific type of the datastore

class arcapix.management.orm.datastores.GPFSDataStore(*args, **kwargs)

A datastore belonging to a GPFS filesystem.

comment

user comment on the item

extras

additional metadata and settings

filesystem

name of the filesystem the datastore belongs to

id

unique identifier for the item

name

datastore name

status

current status of the item

status_message

current status as a string

type

specific type of the datastore

Exposers

class arcapix.management.orm.exposers.CIFSExposer(name, sharepath, visible=True, readonly=False, comment=None, extras=None, *args, **kwargs)

Generic CIFS (Samba) exposer for a space.

Supported extras:

extras can contain any valid smb.conf settings. Any provided values will be written to smb.conf ‘as is’.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

exposer name

readonly

indicates whether the share is readonly access

sharepath

path associated with the share

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

visible

indicates whether the share is visible/browsable

class arcapix.management.orm.exposers.GPFSCIFSExposer(name, sharepath, visible=True, readonly=False, comment=None, extras=None, *args, **kwargs)

An exposer to expose a GPFS based space via CIFS.

This requires that the space is already exposed via the Native exposer

(i.e. that the underlying fileset is linked)

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

exposer name

readonly

indicates whether the share is readonly access

sharepath

path associated with the share

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

visible

indicates whether the share is visible/browsable

class arcapix.management.orm.exposers.GPFSNFSExposer(name, sharepath, *args, **kwargs)

An exposer to expose a GPFS based space via NFS.

This requires that the space is already exposed via the Native exposer (i.e. that the underlying fileset is linked)

client

client associates with the share

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

exposer name

readonly

indicates whether the share is readonly access

sharepath

path associated with the share

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

visible

indicates whether the share is visible/browsable

class arcapix.management.orm.exposers.GPFSNativeExposer(*args, **kwargs)

The native exposer presents the way a space is exposed on the filesystem.

Whenever a space is linked into the filesystem (e.g. a GPFS Fileset being linked), then a native exposer is required.

Typically when a GPFS filesystem is to be exposed via NFS or CIFS, it will also require a native exposer as well.

comment

user comment on the item

extras

additional metadata and settings

filesystem

name of the filesystem associated with the exposer

id

unique identifier for the item

mountpoint

path at which the exposer is mounted

name

exposer name

snapshots

collection of snapshots of the exposer

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

class arcapix.management.orm.exposers.NFSExposer(name, sharepath, *args, **kwargs)

Generic NFS Exposer for a space.

NB. For GPFS, you would normally use a GPFSNFSExposer.

Supported extras:

extras can contain any valid /etc/exports settings. Any provided values will be written to /etc/exports ‘as is’.

Boolean options can be provided in positive or negative form - e.g.

async=True <=> sync=False
root_squash=False <=> no_root_squash=True
client

client associates with the share

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

exposer name

readonly

indicates whether the share is readonly access

sharepath

path associated with the share

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

visible

indicates whether the share is visible/browsable

class arcapix.management.orm.exposers.NativeExposer(*args, **kwargs)

Generic native _Exposer for a space.

NB. For GPFS, you would normally use a GPFSNativeExposer

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

mountpoint

path at which the exposer is mounted

name

exposer name

status

current status of the item

status_message

current status as a string

type

specific type of the exposer

ILM Steps

In general, the Information Life-Cycle (ILM) ORM isn’t used directly, but via the Profile that the particular Step belongs to.

class arcapix.management.orm.lifecycle.ILMStep(*args, **kwargs)

Represents a step in the lifecycle process.

age

length of time before files should be moved in seconds.

comment

user comment on the item

extras

additional metadata and settings

filetypes

types of files that should be moved

id

unique identifier for the item

low_threshold

Calculates the lower bound when using age/size thresholds

minsize

files larger than this size should be moved

profile_id

profile the ilmstep belongs to

status

current status of the item

status_message

current status as a string

targetds

datastore the ilmstep places files on

threshold

high water mark before migration should be considered

Profiles

class arcapix.management.orm.profiles.Profile(*args, **kwargs)

A profile defines the location, ILM, and other characteristics of a space.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

ilm_steps

lifecycle steps belonging to the profile

name

name of the profile typical format ‘{filesystem}-{pool}’

status

current status of the item

status_message

current status as a string

Spaces

Hint

If a fileset is created via the Nexus UI, it will only be a Space if it’s a ‘templated’ fileset. ‘Basic’ filesets created in the Nexus UI are not Spaces at present.

class arcapix.management.orm.spaces.Space(*args, **kwargs)

A space is the fundemental unit of the system.

in essence it represents an instance of some space made available matching a given profile, and exposed via one or more points, and based on a given template.

Supported extras:

  • gpfs.fileset.name (str):

    The name of the fileset corresponding to the space.

    Note - this is a readonly attribute. The fileset name is determined by the Space name and profile.

  • gpfs.fileset.dependent (bool):

    Indicates whether the corresponding fileset is dependent or independent.

    When provided with a POST request, it indicates the type of fileset to create.

  • gpfs.fileset.inodespace (int):

    The inodes space to which the corresponding fileset belongs.

    If dependent is True, a value can be provided to POST. If not provided, the inodespace will be inferred from exposers and relativepath.

    If dependent is False, any value provided to POST will be ingored. Independent filesets always belong to inodespace = 0

  • gpfs.fileset.maxinodes (int):

    The maximum inodes allocation of the corresponding fileset.

    If dependent is False, a value can be provided to POST. If not provided, a default value will be used.

    If dependent is True any provided value will be ignored

  • gpfs.fileset.allocinodes (int):

    The number of inodes allocated to the corresponding fileset.

    If dependent is False, a value can be provided to pre-allocate inodes. If not provided, a default value will be used.

    If dependent is True any provided value will be ignored

    allocinodes must be less than maxinodes

Note

Once a Space is created, changes to these extras will have no effect on the corresponding fileset

comment

user comment on the item

exposers

exposers providing access to the space

extras

additional metadata and settings

id

unique identifier for the item

name

space name

profile

profile applied to the space

relativepath

path of the space relative to its exposers

size

hard limit on the size of the space in blocks

snapshots

snapshots of the space

status

current status of the item

status_message

current status as a string

templates

templates applied to the space

Snapshots

Hint

Snapshots only refer to “one-off” snapshots. Periodic snapshots will be covered (in a future release) by profiles.

class arcapix.management.orm.snapshots.ExposerSnapshot(*args, **kwargs)

A snapshot of an exposer.

Such as of a GPFS Filesystem

comment

user comment on the item

exposer_id

id of the exposer the snapshot is associated with

extras

additional metadata and settings

id

unique identifier for the item

name

snapshot name

status

current status of the item

status_message

current status as a string

taken

time at which the snapshot was taken

type

type of the snapshot

class arcapix.management.orm.snapshots.GPFSNativeExposerSnapshot(*args, **kwargs)

A snapshot of a GPFSNativeExposer.

comment

user comment on the item

extras

additional metadata and settings

filesystem

name of the filesystem associated with the snapshot

id

unique identifier for the item

name

snapshot name

status

current status of the item

status_message

current status as a string

taken

time at which the snapshot was taken

type

type of the snapshot

class arcapix.management.orm.snapshots.GPFSSpaceSnapshot(*args, **kwargs)

Snapshot of a GPFS space.

comment

user comment on the item

extras

additional metadata and settings

filesystem

name of the filesystem associated with the snapshot

id

unique identifier for the item

name

snapshot name

status

current status of the item

status_message

current status as a string

taken

time at which the snapshot was taken

type

type of the snapshot

class arcapix.management.orm.snapshots.Snapshot(*args, **kwargs)

Represents an abstract Snapshot

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

name of the snapshot

status

current status of the item

status_message

current status as a string

taken

datetime at which the snapshot was created

type

type of the snapshot

class arcapix.management.orm.snapshots.SpaceSnapshot(*args, **kwargs)

A snapshot of a space.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

snapshot name

space_id

id of the space associated with the snapshot

status

current status of the item

status_message

current status as a string

taken

time at which the snapshot was taken

type

type of the snapshot

Templates

class arcapix.management.orm.templates.FilesystemTemplate(*args, **kwargs)

Template based on a directory structure.

Can include GPFS dependent filesets.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

template name

status

current status of the item

status_message

current status as a string

template_location

path to the template

template_version

hash of the template

type

specific type of the template

class arcapix.management.orm.templates.Template(*args, **kwargs)

Represents an abstract Template.

comment

user comment on the item

extras

additional metadata and settings

id

unique identifier for the item

name

template name

status

current status of the item

status_message

current status as a string

type

specific type of the template