Fileset¶

PixStor provides three types of Fileset:

Dependent

Independent

AFM Cache

By default a PixStor file system contains one independent fileset (‘root’).

By default dependent filesets inherit the inode space of the root Fileset.

To create separate containers of inodes, create additional independent filesets. Dependent filesets can be parented under additional independent filesets inheriting their individual inode allocations.

Cache filesets are managed by AFM, and are copies of other filesets or filesystems held on (normally) remote systems, typically for the purposes of performance or resiliency management.

Fileset Objects support Callback Functions

Description¶

class arcapix.fs.gpfs.fileset.DependentFileset(filesystemName, name, comment='', **kwargs)¶

DependentFileset class represents a GPFS Dependent Fileset and its associated attributes.

Conceptually, dependent filesets inherit the inode space of their parent fileset. They cannot be used for caching or have snapshots associated.

Creates a DependentFileset object.

Parameters:	filesystemName (str) – the name of the GPFS file system name (str) – the name of the fileset comment (str) – a comment to attach to the fileset. A UUID is recommended. inodeSpace (`IndependentFileset`) – An Independent Fileset whose inode space the Dependent Fileset should share

inodeSpace¶

Returns the Independent Fileset that owns the inode space to which the Dependent Fileset belongs

Return type:	IndependentFileset

create()¶

Create the fileset for the filesystem. The fileset is not linked at this point.

Returns:	Nothing
Raises:	GPFSError

change(**kwargs)¶

Change properties of the Dependent Fileset.

Parameters:	comment – some comment attached to the fileset name – fileset name path – link the fileset to a new path. If the fileset is already linked, that path will be unlinked first force – pass True to force path change. If False, path change might fail. (default=False)

callbacks¶

Returns a collection of the callbacks that are installed on this object

Return type:	Callbacks

comment¶

Returns the user defined optional comment that appears in the output of the mmlsfileset command.

This comment must be less than 256 characters in length. This option cannot be used on the root fileset.

Return type:	str

created¶

Returns the date and time the fileset was created (or None if not created yet or not available)

Return type:	datetime

dataInKB¶

Returns the number of blocks in use for the fileset(when the ‐d flag is included)

Return type:	long

delete(deleteNonEmpty=False, unlink=False, force=False, qosClass=None)¶

Deletes the fileset, optionally forcing the operation if the fileset contains data.

Parameters:	deleteNonEmpty (bool) – Synonym for force True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. force (bool) – Partial Synonym for deleteNonEmpty True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. In addition, setting force to True will forcibly unlink the fileset, if unlink is also specified. unlink (bool) – True: The Fileset will be unlinked prior to deletion. False (default): Fileset will proceed immediately to deletion. qosClass (str) – Quality of Service class to which the operation should be assigned. One of ‘maintenance’ (default) or ‘other’. (GPFS version 4.2+ only)
Returns:	None
Raises:	GPFSError

filesetId¶

Returns the unique ID of the fileset

Return type:	long

filesetMode¶

Returns the mode of the fileset

Also known as Integrated archive manger (IAM) mode

Return type:	str

filesystemName¶

Returns the name of the filesystem on which this fileset is located

Return type:	str

iamMode¶

Integrated archive manger (IAM) mode

Synonym for filesetMode

Return type:	str

id¶

Returns the name of the fileset

Synonym for name

Return type:	str

inodeSpaceId¶

Returns the ID of the inode space which the fileset inherits

Return type:	long

inodeSpaceMask¶

Returns the mask of the inode space

Return type:	long

inodes¶

Returns the number of inodes currently allocated to the fileset

Return type:	long

isInodeSpaceOwner¶

Returns True if the fileset is the inode space owner, false otherwise

Return type:	boolean

is_linked()¶

Check if the fileset is linked to a junction point.

Returns:

True: if the fileset is linked.

False: if the fileset is not linked

Return type:	bool

link(junction_path)¶

Links the Fileset to a ‘junction path’ on the file system.

Parameters:	junction_path (str) – A valid directory path not currently present on the file system
Returns:	None
Raises:	GPFSError

name¶

Returns the name of the fileset

Return type:	str

onAfmCmdRequeued¶

Returns a Callbacks collection, the members of which will be triggered during replication when messages are queued up again because of errors. These messages are retried after 15 minutes (Local Event)

GPFS 4.2.1+ only

onAfmFilesetChange¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is changed. If a fileset is renamed the new name is part of %reason. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetCreate¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is created. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetDelete¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is deleted. (Local Event)

GPFS 5.0.0+ only.

onAfmFilesetExpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset expire. (Global Event)

onAfmFilesetLink¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is linked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnexpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset become unexpired. (Global Event)

onAfmFilesetUnlink¶

Returns a Callbacks collection, the members of which will be triggered when a AFM fileset is unlinked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnmounted¶

Returns a Callbacks collection, the members of which will be triggered when the fileset is moved to an Unmounted state because NFS server is not reachable or remote cluster mount is not available for GPFS Native protocol. (Local Event)

GPFS 4.1.0+ only

onAfmHomeConnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node connects to the home NFS server of the fileset that it is serving. (Local Event)

onAfmHomeDisconnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node gets disconnected from the home NFS server of the fileset that it is serving. (Local Event)

onAfmManualResyncComplete¶: Returns a Callbacks collection, the members of which will be triggered when a manual resync is completed. (Local Event)

onAfmPrepopEnd¶: Returns a Callbacks collection, the members of which will be triggered when all the files specified by a prefetch operation have been cached successfully. (Local Event)

onAfmQueueDropped¶

Returns a Callbacks collection, the members of which will be triggered when replication encounters an issue that cannot be corrected. After the queue is dropped, next recovery action attempts to fix the error and continue to replicate. (Local Event)

GPFS 4.2.1+ only

onAfmRPOMiss¶

Returns a Callbacks collection, the members of which will be triggered when Recovery Point Objective (RPO) is missed on DR primary filesets, RPO Manager keeps retrying the snapshots. This event occurs when there is lot of data to replicate for the RPO snapshot to be taken or there is an error such as, deadlock and recovery keeps failing. (Local Event)

GPFS 4.2.1+ only.

onAfmRecoveryEnd¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery ends. (Local Event)

onAfmRecoveryFail¶

Returns a Callbacks collection, the members of which will be triggered when recovery fails. The recovery action is retried after 300 seconds. If recovery keeps failing, fileset is moved to a resync state if the fileset mode allows it. (Local Event)

GPFS 4.2.1+ only

onAfmRecoveryStart¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery starts. (Local Event)

onFilesetLimitExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a fileset quota has been exceeded. (Local Event)

Note

GPFS recommends using the softQuotaExceeded event instead.

onNoDiskSpace¶: Returns a Callbacks collection, the members of which will be triggered when the file system encounters a disk that ran out of space. (Local Event)

Note

This event is triggered every two minutes until the condition is solved.

onSnapshotCreated¶: Returns a Callbacks collection, the members of which will be triggered after a snapshot is created, and run before the file system is resumed. (Local Event)

onSoftQuotaExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a soft quota limit (for either files or blocks) has been exceeded. (Local Event)

onUsageUnderSoftQuota¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that quota usage has dropped below soft limits and grace time is reset. (Local Event)

parentId¶

Returns the fileset ID of the parent fileset

Return type:	long

path¶

Returns the junction path name for the fileset.

A junction is a special directory entry that connects a name in a directory of one fileset to the root directory of another fileset.

Returns None if not linked to a junction

Return type:	str

permChangeFlag¶

Returns the permission change flags

One of [chmodOnly, setAclOnly, chmodAndSetAcl, chmodAndUpdateAcl]

Return type:	str

quotas¶

Provides access to quotas for this fileset.

NB. Per User and Per Group Quotas won’t work unless you have per-fileset quota enforcement enabled for the filesystem.

Return type:	Quotas

rootInode¶

Returns the inode that the fileset is linked at

Return type:	long

snapId¶

Returns the snapId of the inode space

Return type:	long

status¶

Returns the status of the fileset

One of [Linked, Unlinked, Deleted]

Return type:	str

unlink(force=False)¶

Unlinks the Fileset from the fileset’s current ‘junction path’ on the file system.

In effect, the ‘junction path’ is removed from the file system’

Parameters:

force (bool) –

True: Forcibly unlink the fileset if the fileset contains open files.

False (default): Do not unlink the fileset if the fileset contains open files.

Returns: None

Raises: GPFSError

class arcapix.fs.gpfs.fileset.IndependentFileset(filesystemName, name, comment='', **kwargs)¶

IndependentFileset class represents a GPFS independent Fileset and its associated attributes.

Conceptually, independent filesets are allocated their own inode space. As such they can be used for caching and have snapshots associated.

Creates a IndependentFileset object.

Parameters:	filesystemName (str) – the name of the GPFS file system name (str) – the name of the fileset comment (str) – a comment to attach to the fileset. A UUID is recommended. maxInodes (long) – the maximum number of inodes to allocate to the fileset’s inode space allocInodes (long) – the number of inodes to pre-allocate in the fileset’s inode space

path¶

Returns the junction path name for the fileset.

A junction is a special directory entry that connects a name in a directory of one fileset to the root directory of another fileset.

Returns None if not linked to a junction

Return type:	str

maxInodes¶

Returns the maximum number of inodes which this fileset may have

Default:	0
Return type:	long

allocInodes¶

Returns the allocated number of inodes which this fileset may have

Default:	0
Return type:	long

freeInodes¶

Returns the number of inodes which this fileset is not using

Default:	0
Return type:	long

usedInodes¶

Returns the number of inodes allocated to this fileset which are in use

Return type:	long

snapDir¶

Returns the name of the directory into which fileset snapshots are created

Same as arcapix.fs.gpfs.filesets.Filesets.snapDir

Default:	.snapshots
Return type:	str

dependents¶

Returns a collection of Dependent Filesets, which share an inode space with this Independent Fileset

Return type:	Filesets collection

snapshots¶

Returns a collection of snapshots associated with the fileset

Return type:	Snapshots object

create()¶

Creates an Independent Fileset for the file system within GPFS.

Returns:	Nothing
Raises:	GPFSError

change(**kwargs)¶

Change properties of the Independent Fileset.

Parameters:	comment – some comment attached to the fileset name – fileset name path – link the fileset to a new path. If the fileset is already linked, that path will be unlinked first force – pass True to force path change. If False, path change might fail. (default=False)

Independent Filesets accomodate the following additional parameters:

Parameters:	allocInodes (str) – The number of inodes to preallocate on fileset creation. maxInodes (str) – The total number of inodes to allocate to the fileset on creation.

callbacks¶

Returns a collection of the callbacks that are installed on this object

Return type:	Callbacks

comment¶

Returns the user defined optional comment that appears in the output of the mmlsfileset command.

This comment must be less than 256 characters in length. This option cannot be used on the root fileset.

Return type:	str

created¶

Returns the date and time the fileset was created (or None if not created yet or not available)

Return type:	datetime

dataInKB¶

Returns the number of blocks in use for the fileset(when the ‐d flag is included)

Return type:	long

delete(deleteNonEmpty=False, unlink=False, force=False, qosClass=None)¶

Deletes the fileset, optionally forcing the operation if the fileset contains data.

Parameters:	deleteNonEmpty (bool) – Synonym for force True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. force (bool) – Partial Synonym for deleteNonEmpty True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. In addition, setting force to True will forcibly unlink the fileset, if unlink is also specified. unlink (bool) – True: The Fileset will be unlinked prior to deletion. False (default): Fileset will proceed immediately to deletion. qosClass (str) – Quality of Service class to which the operation should be assigned. One of ‘maintenance’ (default) or ‘other’. (GPFS version 4.2+ only)
Returns:	None
Raises:	GPFSError

filesetId¶

Returns the unique ID of the fileset

Return type:	long

filesetMode¶

Returns the mode of the fileset

Also known as Integrated archive manger (IAM) mode

Return type:	str

filesystemName¶

Returns the name of the filesystem on which this fileset is located

Return type:	str

iamMode¶

Integrated archive manger (IAM) mode

Synonym for filesetMode

Return type:	str

id¶

Returns the name of the fileset

Synonym for name

Return type:	str

inodeSpaceId¶

Returns the ID of the inode space which the fileset inherits

Return type:	long

inodeSpaceMask¶

Returns the mask of the inode space

Return type:	long

inodes¶

Returns the number of inodes currently allocated to the fileset

Return type:	long

isInodeSpaceOwner¶

Returns True if the fileset is the inode space owner, false otherwise

Return type:	boolean

is_linked()¶

Check if the fileset is linked to a junction point.

Returns:

True: if the fileset is linked.

False: if the fileset is not linked

Return type:	bool

link(junction_path)¶

Links the Fileset to a ‘junction path’ on the file system.

Parameters:	junction_path (str) – A valid directory path not currently present on the file system
Returns:	None
Raises:	GPFSError

name¶

Returns the name of the fileset

Return type:	str

onAfmCmdRequeued¶

Returns a Callbacks collection, the members of which will be triggered during replication when messages are queued up again because of errors. These messages are retried after 15 minutes (Local Event)

GPFS 4.2.1+ only

onAfmFilesetChange¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is changed. If a fileset is renamed the new name is part of %reason. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetCreate¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is created. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetDelete¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is deleted. (Local Event)

GPFS 5.0.0+ only.

onAfmFilesetExpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset expire. (Global Event)

onAfmFilesetLink¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is linked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnexpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset become unexpired. (Global Event)

onAfmFilesetUnlink¶

Returns a Callbacks collection, the members of which will be triggered when a AFM fileset is unlinked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnmounted¶

Returns a Callbacks collection, the members of which will be triggered when the fileset is moved to an Unmounted state because NFS server is not reachable or remote cluster mount is not available for GPFS Native protocol. (Local Event)

GPFS 4.1.0+ only

onAfmHomeConnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node connects to the home NFS server of the fileset that it is serving. (Local Event)

onAfmHomeDisconnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node gets disconnected from the home NFS server of the fileset that it is serving. (Local Event)

onAfmManualResyncComplete¶: Returns a Callbacks collection, the members of which will be triggered when a manual resync is completed. (Local Event)

onAfmPrepopEnd¶: Returns a Callbacks collection, the members of which will be triggered when all the files specified by a prefetch operation have been cached successfully. (Local Event)

onAfmQueueDropped¶

Returns a Callbacks collection, the members of which will be triggered when replication encounters an issue that cannot be corrected. After the queue is dropped, next recovery action attempts to fix the error and continue to replicate. (Local Event)

GPFS 4.2.1+ only

onAfmRPOMiss¶

Returns a Callbacks collection, the members of which will be triggered when Recovery Point Objective (RPO) is missed on DR primary filesets, RPO Manager keeps retrying the snapshots. This event occurs when there is lot of data to replicate for the RPO snapshot to be taken or there is an error such as, deadlock and recovery keeps failing. (Local Event)

GPFS 4.2.1+ only.

onAfmRecoveryEnd¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery ends. (Local Event)

onAfmRecoveryFail¶

Returns a Callbacks collection, the members of which will be triggered when recovery fails. The recovery action is retried after 300 seconds. If recovery keeps failing, fileset is moved to a resync state if the fileset mode allows it. (Local Event)

GPFS 4.2.1+ only

onAfmRecoveryStart¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery starts. (Local Event)

onFilesetLimitExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a fileset quota has been exceeded. (Local Event)

Note

GPFS recommends using the softQuotaExceeded event instead.

onNoDiskSpace¶: Returns a Callbacks collection, the members of which will be triggered when the file system encounters a disk that ran out of space. (Local Event)

Note

This event is triggered every two minutes until the condition is solved.

onSnapshotCreated¶: Returns a Callbacks collection, the members of which will be triggered after a snapshot is created, and run before the file system is resumed. (Local Event)

onSoftQuotaExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a soft quota limit (for either files or blocks) has been exceeded. (Local Event)

onUsageUnderSoftQuota¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that quota usage has dropped below soft limits and grace time is reset. (Local Event)

parentId¶

Returns the fileset ID of the parent fileset

Return type:	long

permChangeFlag¶

Returns the permission change flags

One of [chmodOnly, setAclOnly, chmodAndSetAcl, chmodAndUpdateAcl]

Return type:	str

quotas¶

Provides access to quotas for this fileset.

NB. Per User and Per Group Quotas won’t work unless you have per-fileset quota enforcement enabled for the filesystem.

Return type:	Quotas

rootInode¶

Returns the inode that the fileset is linked at

Return type:	long

snapId¶

Returns the snapId of the inode space

Return type:	long

status¶

Returns the status of the fileset

One of [Linked, Unlinked, Deleted]

Return type:	str

unlink(force=False)¶

Unlinks the Fileset from the fileset’s current ‘junction path’ on the file system.

In effect, the ‘junction path’ is removed from the file system’

Parameters:

force (bool) –

True: Forcibly unlink the fileset if the fileset contains open files.

False (default): Do not unlink the fileset if the fileset contains open files.

Returns: None

Raises: GPFSError

class arcapix.fs.gpfs.fileset.CacheFileset(filesystemName, name, remotePath, remoteHost=None, protocol="nfs", mode="read-only", comment='', **kwargs)¶

Represents an AFM managed Cache fileset.

Instantiates a CacheFileset object.

Parameters:

filesystemName (str) – The name of the filesystem the Cache will be on.
name (str) – The name for the fileset
remotePath (str) – The path on the remote system
remoteHost (str) – For NFS based AFM only, specifies the remote host to cache
protocol (str) – Either ‘nfs’ or ‘gpfs’ (gpfs v4+ only).
mode (str) – Specifies the way the cache works. See main AFM documentation. One of single-writer (sw), read-only (ro), local-updates (lu), independent-writer (iw)
comment (str) – Provides a comment to be attached to the fileset

create()¶

Creates an AFM Cached Fileset, based on the properties which were supplied to the constructor

Returns:	Nothing
Raises:	GPFSError

change(**kwargs)¶

Change properties of the fileset.

NB. Implications of changes to the ‘name’ attribute should be understood, as the API does not modify the parent filesets collection to match the new name

Parameters:	extras – TBC
Type:	extras: TBC

remoteHost¶

remotePath¶

protocol¶

afmTarget¶

afmMode¶

This AFM configuration attribute specifies the mode in which the cache operates. Valid values are the following:

single‐writer | sw Specifies single‐writer mode.

read‐only | ro Specifies read‐only mode. (For mmcrfileset, this is the default value.)

local‐updates | lu Specifies local‐updates mode.

independent‐writer | iw Specifies independent‐writer mode.

Changing from read‐only and single‐writer modes to another mode is supported. When changing from read‐only to single‐writer, the read‐only cache is up‐to‐date. When changing from single‐writer to read‐only, all requests from cache should have been played at home. Changing from local‐updates is restricted.

A typical dataset is set up to include a single cache cluster in single‐writer mode (which generates the data) and one or more cache clusters in local‐updates or read‐only mode.

Return type:	str

afmDirLookupRefreshInterval¶

This AFM configuration attribute (specified in seconds) controls the frequency of data revalidations that are triggered by lookup perations (such as ls or stat). When a lookup operation is done on a directory, if the specified amount of time has passed, AFM sends a message to the home cluster to find out whether the metadata of that directory has been modified since the last time it was checked. If the time interval has not passed, AFM does not check the home cluster for updates to the metadata.

Valid values are 0 through 2147483647. In situations where home cluster data changes frequently, a value of 0 is recommended.

Default:	60
Return type:	long

afmNumReadThreads¶

This tunable defines how many read threads can be used on each participating gateway node during parallel read. The default value of this parameter is 1; that is, one reader thread will be active on every gateway node for each big write operation qualifying for splitting per the parallel read threshold value.

The valid range of values is 1 to 64.

Default:	1
Return type:	long

afmDirOpenRefreshInterval¶

This AFM configuration attribute (specified in seconds) controls the frequency of data revalidations that are triggered by I/O operations (such as read or write). After a directory has been cached, open requests resulting from I/O operations on that object are directed to the cached directory until the specified amount of time has passed. Once the specified amount of time has passed, the open request gets directed to a gateway node rather than to the cached directory.

Valid values are 0 through 2147483647. Setting a lower value guarantees a higher level of consistency.

Default:	60
Return type:	long

afmAsyncDelay¶

This AFM configuration attribute specifies (in seconds) the amount of time by which write operations are delayed (because write operations are asynchronous with respect to remote clusters). For write‐intensive applications that keep writing to the same set of files, this delay is helpful because it replaces multiple writes to the home cluster with a single write containing the latest data. However, setting a very high value weakens the consistency of data on the remote cluster.

This configuration parameter is applicable only for writer caches (SW and IW), where data from cache is pushed to home.

Valid values are 1 through 2147483647.

Default:	15
Return type:	long

afmExpirationTimeout¶

Returns the AFM expiration timeout

Default:	0
Return type:	int

afmEnableAutoEviction¶

This AFM configuration attribute enables eviction on a given fileset.

A True value specifies that eviction is allowed on the fileset.

A False value specifies that eviction is not allowed on the fileset.

Return type:	bool

afmFileLookupRefreshInterval¶

This AFM configuration attribute (specified in seconds) controls the frequency of data revalidations that are triggered by lookup operations (such as ls or stat). When a lookup operation is done on a file, if the specified amount of time has passed, AFM sends a message to the home cluster to find out whether the metadata of the file has been modified since the last time it was checked. If the time interval has not passed, AFM does not check the home cluster for updates to the metadata.

Valid values are 0 through 2147483647. In situations where home cluster data changes frequently, a value of 0 is recommended.

Default:	30
Return type:	int

afmPrefetchThreshold¶

This AFM configuration attribute controls partial file caching and prefetching. Valid values are the following:

0: Enables full file prefetching. This is useful for sequentially accessed files that are read in their entirety, such as image files, home directories, and development environments. The file will be prefetched after three blocks have been read into the cache.
1-99: Specifies the percentage of file size that must be cached before the entire file is prefetched. A large value is suitable for a file accessed either randomly or sequentially but partially, for which it might be useful to ingest the rest of the file when most of it has been accessed.
100: Disables full file prefetching. This value only fetches and caches data that is read by the application. This is useful for large random-access files, such as databases, that are either too big to fit in the cache or are never expected to be read in their entirety. When all data blocks are accessed in the cache, the file is marked as cached.

Default:	0
Return type:	int

afmShowHomeSnapshots¶

This AFM configuration attribute controls the visibility of the home snapshot directory in cache. For this to be visible in cache, this variable has to be set to true, and the snapshot directory name in cache and home should not be the same.

Return type:	bool

afmParallelWriteThreshold¶

Defines the threshold beyond which parallel writes become effective

Values are interpreted in terms of MB. Valid range of values is 0 to 2147483647.

Default:	1024
Return type:	int

afmReadSparseThreshold¶

Specifies the size in MB for files in cache beyond which sparseness is maintained

Return type:	int

afmReadBufferSize¶

Return type:	int

afmParallelReadThreshold¶

Defines the threshold beyond which parallel reads become effective Values are interpreted in terms of MB. Valid range of values is 0 to 2147483647

Default:	1024
Return type:	int

afmNumFlushThreads¶

Defines the number of threads used on each gateway to synchronize updates to the home cluster

The current maximum value is 1024

Default:	4
Return type:	int

afmWriteBufferSize¶

Return type:	int

afmNeedsResync¶: Specifies whether the fileset needs resyncing :rtype: bool

afmPrimaryId¶

Specifies the unique primary ID of the primary fileset for asynchronous data replication

Return type:	str

afmNeedsRecovery¶

Specifies whether the fileset needs recovery

Return type:	bool

afmRPO¶

Specifies the recovery point objective (RPO) interval in minutes for a primary fileset.

Return type:	int

afmAssociatedPrimaryId¶

Return type:	str

afmState¶

Return type:	str

afmLastPSnapId¶

Return type:	str

afmParallelReadChunkSize¶

Defines the threshold beyond which parallel reads become effective

Valid range of values is 0 to 2147483647.

Default:	1024
Return type:	int

afmDRState¶

Return type:	str

afmFileOpenRefreshInterval¶

Controls the frequency of data revalidations that are triggered by such I/O operations as read or write (specified in seconds).

Valid values are 0 through 2147483647.

Default:	30
Return type:	int

afmNumWriteThreads¶

Defines the number of threads that can be used on each participating gateway node during parallel write

Valid values can range from 1 to 64

Default:	1
Return type:	int

afmParallelWriteChunkSize¶

Defines the minimum chunk size of the write that needs to be distributed among the gateway nodes during parallel writes

Valid range of values is 0 to 2147483647

Default:	128
Return type:	int

afmGateway¶

Specifies the user-defined gateway node for an AFM or AFM DR fileset, that gets preference over internal hashing algorithm.

Cluster afmHashVersion value must be already set as ‘5’

New in GPFS 5.0.2 - older versions will have a default value of None

Return type:	str

maxInodes¶

Returns the maximum number of inodes which this fileset may have

Default:	0
Returns:	long

allocInodes¶

Returns the allocated number of inodes which this fileset may have

Default:	0
Returns:	long

cacheState¶

State of the cache

:rtype:str

gatewayNode¶

Metadata server or MDS of the fileset. This gateway node is handling requests for this fileset.

Return type:	str

queueLength¶

Current length of the queue on the MDS.

Return type:	int

numExec¶

Number of operations played at home since the fileset is last Active.

Return type:	int

DRState¶

Return type:	str

resync()¶

In case of inadvertent changes made at home of an SW fileset, such as delete of a file or change of data in a file etc., the administrator can correct the home by sending all contents from cache to home using this option.

The limitation of this option that renamed files at home may not be fixed by resync

This method is available only for SW cache. Using resync requires the cache to be either in NeedsResync or Active state.

expire()¶

When an RO cache is disconnected, the cached contents are still accessible for the user.

However the administrator can define a time from home beyond which access to the cached contents becomes stale.

Such an event would occur automatically after disconnection (when cached contents are no longer accessible) and is called expiration; the cache is said to be expired.

This state can also be forced manually using the expire parameter.

This method is available only for RO cache

unexpire()¶

When the home comes back or reconnects, the cache contents become automatically accessible again and the cache is said to un-expire.

This can be forced manually using the unexpire parameter.

This method is available only for RO cache

resumeRequeued()¶

If there are operations in the queue that were re-queued due to errors at home, the Administrator should correct those errors and can run this option to retry the re-queued operations.

This method is applicable for SW/IW and primary filesets

flushPending(listFile=None, localWorkDirectory=None)¶

Flushes all point-in-time pending messages in the normal queue on the fileset to home. Requeued messages and messages in the priority queue for the fileset are not flushed by this command.

Parameters:	listFile (str) – if specified, messages pending on the files listed in the list file are flushed to home. localWorkDirectory (str) – Specifies the temporary working directory.

failover(newTarget, targetOnly=False, localWorkDirectory=None)¶

This option pushes all the data from cache to home. It should be used only in case home is completely lost due to a disaster and a new home is being set up.

This option is applicable only for SW/IW filesets

Parameters:

newTarget (str) – Specifies a new home server and path
targetOnly (bool) – This is used if the user wants to change the mount path/IP address in the target path. The new NFS server should be in the same home cluster and should be of the same architecture as the existing NFS server in the target path. This option should not be used to change the target location or protocol.
localWorkDirectory (str) – Specifies the temporary working directory.

prefetch(metadataOnly=False, listFile=None, homeFsPath=None, localWorkDirectory=None, retryFailedFileList=False, enableFailedFileList=False, policy=False, directory=None)¶

This option is used for fetching file contents from home before the application requests for the contents. This reduces the network delay when the application is in progress.

You can also use this option to move files over the WAN when the WAN usage is low. These files might be the files that are accessed during high WAN usage. Thus, you can use this option for better WAN management.

Parameters:

metadataOnly (bool) – Prefetches only the metadata and not the actual data
homeFsPath (str) – Specifies the full path to the fileset at the home cluster and can be used in conjunction with ListFile.
listFile (str) – if specified, messages pending on the files listed in the list file are flushed to home.
localWorkDirectory (str) – Specifies the temporary working directory.
retryFailedFileList (bool) – Allows re-trying prefetch of files that failed in the last prefetch operation (5.0.2+) To use this option, you must set enableFailedFileList=True
enableFailedFileList (bool) – Turns on generating a list of files which failed during prefetch operation (5.0.2+)
policy (bool) – Specifies that the listFile is generated using a GPFS Policy (5.0.2+)
directory (str) – Specifies path to the local directory from which you want to prefetch files (5.0.2+)

evict(safeLimit=None, logFile=None, order=None, filenameFilter=None, minFilesize=None, maxFilesize=None, file=None)¶

When cache space exceeds the allocated quota, data blocks from non-dirty are automatically de-allocated with the eviction process.

This option can be used for a file that is specifically to be de-allocated based on some criteria

Parameters:

safeLimit – Specifies target quota limit (which is used as the low water mark) for eviction in bytes; must be less than the soft limit.
order –
Specifies the order in which files are to be chosen for eviction:

LRU - Least recently used files are to be evicted first.

SIZE - Larger-sized files are to be evicted first.
logFile – Specifies the file where the eviction log is to be stored. The default is that no logs are generated.
filenameFilter – Specifies the name of a file to be evicted from the cache. This uses an SQL-type search query. If the same file name exists in more than one directory, it will evict all the files with that name. The complete path to the file should not be given here.
minFilesize – Sets the minimum size of a file to evict from the cache. This value is compared to the number of blocks allocated to a file (KB_ALLOCATED), which may differ slightly from the file size.
maxFilesize – Sets the maximum size of a file to evict from the cache. This value is compared to the number of blocks allocated to a file (KB_ALLOCATED), which may differ slightly from the file size.
file – The fully qualified name of the file that needs to be evicted.

startFailback(failoverTime, localWorkDirectory=None)¶

failback is applicable only for IW filesets

Parameters:	failoverTime – Specifies the point in time at the home cluster, from which the independent-writer cache taking over as writer should sync up. Can be specified in date command format with time zones. It will use the cluster’s time-zone and year by default. localWorkDirectory (str) – Specifies the temporary working directory.

stopFailback(localWorkDirectory=None)¶

An option to be run after the failback process is complete and the fileset moves to FailbackCompleted state. This will move the fileset to Active state.

failback is applicable only for IW filesets

Parameters:	localWorkDirectory (str) – Specifies the temporary working directory.

failoverToSecondary(restore=True)¶

When primary experiences a disaster, all applications will need to be moved to the secondary to ensure business continuity.

The secondary has to be first converted to an acting primary using this option.

Parameters:	restore – If True, restoring of data is done from the latest peer snapshot else restoring from the latest peer snapshot is not done (default=True)

(GPFS 4.1.1+ only)

convertToPrimary(afmTarget=None, inband=False, outband=False, secondarySnapname=None, checkMetadata=None, rpo=None, localWorkDirectory=None)¶

This is to be run on a GPFS fileset or SW/IW fileset which is intended to be converted to primary.

Parameters:

afmTarget – Specifies the secondary that needs to be configured for this primary. Need not be used for AFM filesets as target would already have been defined.
inband – Used for inband trucking. Inband trucking is copying the data while setting up a primary/secondary relationship from GPFS filesets, where primary site has contents and secondary site is empty.
outband – Used for outband trucking. Outband trucking is copying data manually using other ways such as ftp, scp, rsync etc. This should be completed before the relationship is established.
secondarySnapname – Used while establishing a new primary for an existing secondary or acting primary during failback.
checkMetadata – This checks if the disallowed types (like immutable/append-only files) are present in the GPFS fileset on the primary site before the conversion. If True, conversion will fail if such files exist.
rpo – Specifies the RPO interval in minutes for this primary fileset.
localWorkDirectory (str) – Specifies the temporary working directory.

(GPFS 4.1.1+ only)

convertToSecondary(primaryId, force=False)¶

This is to be run on a GPFS fileset on the secondary site. This converts a GPFS independent fileset to a secondary and sets the primary ID.

Parameters:	primaryId – Specifies the ID of the primary with which the secondary will be associated. force – If convertToSecondary failed or got interrupted, it will not create afmctl file at the secondary. The command should be rerun with the force=True

(GPFS 4.1.1+ only)

changeSecondary(newTarget, inband=False, outband=False, targetOnly=False, localWorkDirectory=None)¶

A disaster at the secondary can take place due to which secondary is not available.

Run this command on the primary when a secondary fails and this primary needs to be connected with a new secondary. On the new secondary site a new GPFS independent fileset has to be created. Data on the primary can be copied to the new GPFS fileset that was created with this command using other means such as ftp, scp etc. Alternatively it can be decided that data will be trucked using the relationship.

Parameters:

newTarget – Specifies the new secondary.
inband – Used for inband trucking
outband – Used for outband trucking
targetOnly – Used when you want to change the IP address or NFS server name for the same target path. The new NFS server must be in the same home cluster and must be of the same architecture(power or x86) as the existing NFS server in the target path. This option can be used to move from NFS to a mapping target.
localWorkDirectory (str) – Specifies the temporary working directory.

(GPFS 4.1.1+ only)

replacePrimary()¶

This is used on an acting primary only. This will create a latest snapshot of the acting primary. This command deletes any old peer snapshots on the acting primary and creates a new initial peer snapshot psnap0.

This snapshot will be used in the setup of the new primary.

(GPFS 4.1.1+ only)

startFailbackToPrimary()¶

This is to be run on an old primary that came back after the disaster, or on a new primary that is to be configured after an old primary went down with a disaster. The new primary should have been converted from GPFS to primary using convertToPrimary option.

Restores the primary to the contents from the last RPO on the primary before the disaster. This option will put the primary in read-only mode, to avoid accidental corruption until the failback process is completed. In case of new primary that is setup using convertToPrimary, the startFailback does no change.

(GPFS 4.1.1+ only)

callbacks¶

Returns a collection of the callbacks that are installed on this object

Return type:	Callbacks

comment¶

Returns the user defined optional comment that appears in the output of the mmlsfileset command.

This comment must be less than 256 characters in length. This option cannot be used on the root fileset.

Return type:	str

created¶

Returns the date and time the fileset was created (or None if not created yet or not available)

Return type:	datetime

dataInKB¶

Returns the number of blocks in use for the fileset(when the ‐d flag is included)

Return type:	long

delete(deleteNonEmpty=False, unlink=False, force=False, qosClass=None)¶

Deletes the fileset, optionally forcing the operation if the fileset contains data.

Parameters:	deleteNonEmpty (bool) – Synonym for force True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. force (bool) – Partial Synonym for deleteNonEmpty True: Filesets which are not empty will be forcably deleted. False (default): A Fileset containing data will not be deleted. In addition, setting force to True will forcibly unlink the fileset, if unlink is also specified. unlink (bool) – True: The Fileset will be unlinked prior to deletion. False (default): Fileset will proceed immediately to deletion. qosClass (str) – Quality of Service class to which the operation should be assigned. One of ‘maintenance’ (default) or ‘other’. (GPFS version 4.2+ only)
Returns:	None
Raises:	GPFSError

filesetId¶

Returns the unique ID of the fileset

Return type:	long

filesetMode¶

Returns the mode of the fileset

Also known as Integrated archive manger (IAM) mode

Return type:	str

filesystemName¶

Returns the name of the filesystem on which this fileset is located

Return type:	str

iamMode¶

Integrated archive manger (IAM) mode

Synonym for filesetMode

Return type:	str

id¶

Returns the name of the fileset

Synonym for name

Return type:	str

inodeSpaceId¶

Returns the ID of the inode space which the fileset inherits

Return type:	long

inodeSpaceMask¶

Returns the mask of the inode space

Return type:	long

inodes¶

Returns the number of inodes currently allocated to the fileset

Return type:	long

isInodeSpaceOwner¶

Returns True if the fileset is the inode space owner, false otherwise

Return type:	boolean

is_linked()¶

Check if the fileset is linked to a junction point.

Returns:

True: if the fileset is linked.

False: if the fileset is not linked

Return type:	bool

link(junction_path)¶

Links the Fileset to a ‘junction path’ on the file system.

Parameters:	junction_path (str) – A valid directory path not currently present on the file system
Returns:	None
Raises:	GPFSError

name¶

Returns the name of the fileset

Return type:	str

onAfmCmdRequeued¶

Returns a Callbacks collection, the members of which will be triggered during replication when messages are queued up again because of errors. These messages are retried after 15 minutes (Local Event)

GPFS 4.2.1+ only

onAfmFilesetChange¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is changed. If a fileset is renamed the new name is part of %reason. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetCreate¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is created. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetDelete¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is deleted. (Local Event)

GPFS 5.0.0+ only.

onAfmFilesetExpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset expire. (Global Event)

onAfmFilesetLink¶

Returns a Callbacks collection, the members of which will be triggered when an AFM fileset is linked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnexpired¶: Returns a Callbacks collection, the members of which will be triggered when the contents of a fileset become unexpired. (Global Event)

onAfmFilesetUnlink¶

Returns a Callbacks collection, the members of which will be triggered when a AFM fileset is unlinked. (Local Event)

GPFS 4.2.3+ only.

onAfmFilesetUnmounted¶

Returns a Callbacks collection, the members of which will be triggered when the fileset is moved to an Unmounted state because NFS server is not reachable or remote cluster mount is not available for GPFS Native protocol. (Local Event)

GPFS 4.1.0+ only

onAfmHomeConnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node connects to the home NFS server of the fileset that it is serving. (Local Event)

onAfmHomeDisconnected¶: Returns a Callbacks collection, the members of which will be triggered when a gateway node gets disconnected from the home NFS server of the fileset that it is serving. (Local Event)

onAfmManualResyncComplete¶: Returns a Callbacks collection, the members of which will be triggered when a manual resync is completed. (Local Event)

onAfmPrepopEnd¶: Returns a Callbacks collection, the members of which will be triggered when all the files specified by a prefetch operation have been cached successfully. (Local Event)

onAfmQueueDropped¶

Returns a Callbacks collection, the members of which will be triggered when replication encounters an issue that cannot be corrected. After the queue is dropped, next recovery action attempts to fix the error and continue to replicate. (Local Event)

GPFS 4.2.1+ only

onAfmRPOMiss¶

Returns a Callbacks collection, the members of which will be triggered when Recovery Point Objective (RPO) is missed on DR primary filesets, RPO Manager keeps retrying the snapshots. This event occurs when there is lot of data to replicate for the RPO snapshot to be taken or there is an error such as, deadlock and recovery keeps failing. (Local Event)

GPFS 4.2.1+ only.

onAfmRecoveryEnd¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery ends. (Local Event)

onAfmRecoveryFail¶

Returns a Callbacks collection, the members of which will be triggered when recovery fails. The recovery action is retried after 300 seconds. If recovery keeps failing, fileset is moved to a resync state if the fileset mode allows it. (Local Event)

GPFS 4.2.1+ only

onAfmRecoveryStart¶: Returns a Callbacks collection, the members of which will be triggered when AFM recovery starts. (Local Event)

onFilesetLimitExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a fileset quota has been exceeded. (Local Event)

Note

GPFS recommends using the softQuotaExceeded event instead.

onNoDiskSpace¶: Returns a Callbacks collection, the members of which will be triggered when the file system encounters a disk that ran out of space. (Local Event)

Note

This event is triggered every two minutes until the condition is solved.

onSnapshotCreated¶: Returns a Callbacks collection, the members of which will be triggered after a snapshot is created, and run before the file system is resumed. (Local Event)

onSoftQuotaExceeded¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that a soft quota limit (for either files or blocks) has been exceeded. (Local Event)

onUsageUnderSoftQuota¶: Returns a Callbacks collection, the members of which will be triggered when the file system manager detects that quota usage has dropped below soft limits and grace time is reset. (Local Event)

parentId¶

Returns the fileset ID of the parent fileset

Return type:	long

path¶

Returns the junction path name for the fileset.

A junction is a special directory entry that connects a name in a directory of one fileset to the root directory of another fileset.

Returns None if not linked to a junction

Return type:	str

permChangeFlag¶

Returns the permission change flags

One of [chmodOnly, setAclOnly, chmodAndSetAcl, chmodAndUpdateAcl]

Return type:	str

quotas¶

Provides access to quotas for this fileset.

NB. Per User and Per Group Quotas won’t work unless you have per-fileset quota enforcement enabled for the filesystem.

Return type:	Quotas

rootInode¶

Returns the inode that the fileset is linked at

Return type:	long

snapId¶

Returns the snapId of the inode space

Return type:	long

status¶

Returns the status of the fileset

One of [Linked, Unlinked, Deleted]

Return type:	str

stopFailbackToPrimary(force=False)¶

Used to complete the Failback process. This will put the fileset in read-write mode. The primary is now ready for starting applications.

Parameters:	force – Use if a previous call does not complete due to errors and not allow for failback to be stopped.

(GPFS 4.1.1+ only)

unlink(force=False)¶

Unlinks the Fileset from the fileset’s current ‘junction path’ on the file system.

In effect, the ‘junction path’ is removed from the file system’

Parameters:

force (bool) –

True: Forcibly unlink the fileset if the fileset contains open files.

False (default): Do not unlink the fileset if the fileset contains open files.

Returns: None

Raises: GPFSError

applyUpdates()¶

Run this on the primary after running the startFailback command. All the differences can be brought over in one go or through multiple iterations. For minimizing application downtime, this command can be run multiple times to bring the primary’s contents in sync with the acting primary.

When the contents are as close as possible or minimal, applications should take a downtime and then this command should be run one last time.

It is possible that applyUpdates fails with an error during instances when the acting primary is overloaded. In such cases the command has to be run again.

(GPFS 4.1.1+ only)

classmethod decodeTarget(afmTarget)¶

Decodes a GPFS afmTarget string

Returns:	(Protocol, Path, Host)
Return type:	tuple

classmethod encodeTarget(protocol, path, host=None)¶

Encodes an afmTarget string from the 3 bits

Parameters:	protocol – either nfs or gpfs path – UNIX style path host – The host to connect to (NFS protocol only)

Examples¶

Creating a Dependent Fileset directly¶

>>> from arcapix.fs.gpfs import DependentFileset
>>> import uuid
>>>
>>> # Create a random UUID to use in the fileset's comment files
... uid = uuid.uuid4()
>>>
>>> # Create a dependent fileset object (created in root inode space)
... myfileset = DependentFileset('mmfs1', 'example-dep-fset', uid)
>>> myfileset.create()
>>>
>>> # Link the fileset to the mmfs1 file system
... myfileset.link('/mmfs1/example-dep-fset-dir')
>>>
>>> # Unlink the fileset
... myfileset.unlink()
>>>
>>> # Remove the fileset
... myfileset.delete()

Creating an Independent Fileset directly¶

>>> from arcapix.fs.gpfs import IndependentFileset
>>> import uuid
>>>
>>> # Create a random UUID to use in the fileset's comment files
... uid = uuid.uuid4()
>>>
>>> # Create and independent fileset with a size of 10000 inodes and preallocate 2000 inodes
... myfileset = IndependentFileset('mmfs1', 'indep-fileset1', uid, maxInodes=10000, allocInodes=2000)
>>> myfileset.create()
>>>
>>> # Link the fileset to the mmfs1 file system
... myfileset.link('/mmfs1/example-indep-fset-dir')
>>>
>>> # Unlink the fileset
... myfileset.unlink()
>>>
>>> # Remove the fileset
... myfileset.delete()

Create a Dependent Fileset under an Independent Fileset¶

>>> from arcapix.fs.gpfs import Filesets
>>>
>>> # Get independent fileset from collection
... myfileset = Filesets('mmfs1')['indep-fileset1']
>>>
>>> # Create new Dependent Fileset
... myfileset.dependents.new('dep-fileset1')
>>>
>>> # List dependent filesets of 'indep-fileset1'
... print(myfileset.dependents)
dep-fileset1
>>>
>>> # delete dependent fileset
... myfileset.dependents.destroy('dep-fileset1')

Working with an AFM Cache Fileset¶

>>> from arcapix.fs.gpfs import CacheFileset
>>> import uuid
>>>
>>> # Create fileset (Using default NFS protocol, and read-only cache approach
... myfileset = CacheFileset('mmfs1', 'cache-fileset1', '/mmfs1/projects/project1', 'gw1')
>>> myfileset.create()
>>>
>>> # Change the number of read threads
... myfileset.change(afmNumReadThreads=4)
>>>
>>> # Create another fileset, using GPFS protocol
... myfileset = CacheFileset('mmfs1', 'cache-fileset2', '/remote/mmfs1/projects/project2', protocol='gpfs')