Filesystem¶
The Filesystem Object represents an instance of a PixStor file system and its associated attributes.
The Filesystem Object has child Objects of type:
Filesystems Objects support Callback Functions
Description¶
-
class
arcapix.fs.gpfs.filesystem.RsFlags¶ Flags which may be passed to the
Filesystem restripemethod:- criticalFromSuspended
- allFromSuspended
- repairPoolPlacement
- repairReplicaConflicts
- repairReadOnlyConflicts
- rebalance
- strictRebalance
- defaultReplication
- runDeferedCompression
-
arcapix.fs.gpfs.filesystem.setTempFilePath(path)¶ Change the directory to which temporary files are written
Default:
/<filesystem_default_mount>/.policytmpNote
If you don’t have write permission for the specified directory, you won’t be able to write temporary file or use methods that require writing temporary files
-
class
arcapix.fs.gpfs.filesystem.FilesystemVersion(internal, major, minor, human)¶ “A small structure which provides information about the version of the filesystem in use.
-
full¶ The human readable full version i.e. 3.5.3.4
-
major¶ The major version components 3.5, 4.1 etc.
-
internal¶ The internal version number, 13.32 etc.
-
minor¶ The tail of the full version i.e. 55.67
-
-
class
arcapix.fs.gpfs.filesystem.Filesystem(deviceName)¶ Holds a representation of a GPFS file system.
Parameters: deviceName (str) – the name of the filesystem -
snapDir¶ Returns the name of the directory in which global snapshots are stored
Default: .snapshots Return type: str
-
subdirectorySnapDirs¶ Returns whether all filesystem subdirectories have a snapshot subdirectory
E.G. /mmfs1/dir/file > /mmfs1/.snapshots/snap1/dir/file + /mmfs1/dir/.snapshots/snap1/fileReturn type: bool
-
policy¶ Returns the running placement policy installed on the filesystem
Return type: PlacementPolicy
-
blockAllocationType¶ Returns the block allocation type
Either ‘cluster’ or ‘scatter’
Return type: str
-
creationTime¶ Returns the creation time of the filesystem
Return type: datetime
-
disks¶ Returns the disks associated with the filesystem
Return type: Disks object
-
fastEAenabled¶ Returns if fast extended attributes are enabled.
Returns None if ‘Unknown’
Return type: bool
-
fileLockingSemantics¶ Returns the file locking semantics in effect
One of [nfs4, posix, all]
Return type: str
-
filesetdfEnabled¶ Returns whether the fileset df is enabled.
Returns None if ‘Unknown’
Return type: bool
-
filesystemHighestSupported¶ Returns the highest supported version of the filesystem.
Return type: FilesystemVersion
-
filesystemVersionLocal¶ Returns the supported filesystem version supported locally.
Return type: FilesystemVersion
-
filesystemVersion¶ Returns the supported filesystem version
Return type: FilesystemVersion
-
filesystemVersionManager¶ Returns the supported filesystem version supported by the cluster manager
Return type: FilesystemVersion
-
filesystemVersionOriginal¶ Returns the version the filesystem was created at
Return type: FilesystemVersion
-
quotasEnforced¶ Returns the types of quotas that are enforced on the filesystem
A subset of [user, group, fileset]
Return type: set
-
snapshots¶ Returns the collection of global snapshot objects for the filesystem
This doesn’t include fileset snapshots.
Return type: SnapshotsObject
-
quotas¶ Returns the quota objects for this filesystem
This doesn’t include fileset quotas
Return type: Quotas
-
strictReplication¶ Returns if strict replication is enforced
One of [yes, no, whenpossible]
Note
Whilst at first glance this should be a boolean, it is in fact tristate, so we can only offer a string, which will always be lower case.
Return type: str
-
suppressAtime¶ Returns whether the access time is suppressed
One of [yes, no, relatime]
Note
Whilst at first glance this should be a boolean, it is in fact tristate, so we can only offer a string, which will always be lower case.
Return type: str
-
quotasAccountingEnabled¶ Returns the types of quota that are enabled.
A subset of [user, group, fileset]
Return type: set
-
maintenanceMode¶ Returns whether maintenance mode is enabled.
New in GPFS 5.0.1 - older versions will always return False
Default: False Return type: bool
-
writeCacheThreshold¶ Returns the configured write cache (HAWC) threshold.
Max value 65536
Return type: int
-
subblocksPerFullBlock¶ Returns the number of subblocks per full block.
New in GPFS 5.0.0 - older version will default to 32
Return type: int
-
fileAuditLog¶ Returns whether file audit logging is enabled.
New in GPFS 5.0.0 - older version will always return False
Return type: bool
-
autoInodeLimit¶ Returns whether auto inode limit is enabled.
When enabled, the filesystem automatically increases the maximum number of inodes per inode space.
New in GPFS 5.1.4 - older versions will always return False
Return type: bool
-
resetQoS()¶ Resets Quality of Service to unlimited for all pools
-
change(**kwargs)¶ Change attributes of the filesystem. Available attributes include:
Parameters: - quotasEnabled (bool) – this can only be modified while the file system is not mounted
- defaultMountPoint (str) –
- manager (str) – Node to be made the filesystem manager
- qosEnabled (bool) – Enable/disable Quality of Service for all pools in the filesystem (GPFS 4.2+ only)
- snapDir (str) – Change the directory under which global snapshots are stored (default=’.snapshots’)
- subdirectorySnapDirs (bool) – Specify whether all filesystem subdirectories have a snapshot subdirectory E.G. /mmfs1/dir/file > /mmfs1/.snapshots/snap1/dir/file + /mmfs1/dir/.snapshots/snap1/file
- maintenanceMode (bool) – Enable/disable filesystem maintenance mode (GPFS 5.0.2+ only). Note - the filesystem must be unmounted on all nodes before maintenance mode can be enabled.
Note
Enabling QoS will restore any previously set limits. Limits can be changed via Pool objects. If no limits have been set previously, they will be ‘unlimited’ (None) by default
-
restripe(flag, nodes=None, qosClass=None, metadata_only=False)¶ Performs a restripe command on all files in the filesystem
Parameters: - flag – The restripe action to perform.
One of
RsFlags - nodes – A node or nodes to participate in the restripe (default=all)
- qosClass (str) – Quality of Service class to which the operation should be assigned. One of ‘maintenance’ (default) or ‘other’. (GPFS version 4.2+ only)
- metadata_only (bool) – limit operations to metadata blocks (GPFS version 4.2+ only)
- flag – The restripe action to perform.
One of
-
suspend()¶ Suspends the processing of all subsequent I/O requests.
Flushes the internal buffers on all nodes, and brings the file system to a consistent state on disk.
-
suspend_write()¶ Suspends the processing of subsequent write I/O requests.
Flushes all pending requests on all nodes, and brings the file system to a consistent state on disk.
-
resume()¶ Resumes the normal processing of I/O requests on all nodes.
-
mount(nodes=None, async_=False, timeout=300, **kwargs)¶ Mounts the filesystem on one, some, or all nodes.
If nodes is not passed, or is empty, then the filesystem is mounted on the node the API function is called on.
Otherwise, the node is mounted on the nodes given (the set can either be names, or node objects)
To emulate the -a behaviour, one can do
>>> Filesystem("mmfs1").mount(Nodes())
Parameters: - nodes (set) – a set of Nodes to mount the filesystem on. In no set is given, the calling node is mounted.
- async (bool) – when True, the call will return as soon as the mount process has started. Otherwise, it will block until the mount has completed
- timeout (int) – If the filesystem has not unmounted by the end of the timeout, an Exception is raised. The default is 300 seconds. To simulate an infinite timeout, use for example 86400, equating to 1 day.
Raises: GPFSError
-
unmount(nodes=None, async_=False, timeout=300, force=False, **kwargs)¶ Unmounts the filesystem on one, some, or all nodes.
If nodes is not passed, or is empty, then the filesystem is mounted on the node the API function is called on.
Otherwise, the node is mounted on the nodes given (the set can either be names, or node objects)
To emulate the -a behaviour, one can do
>>> Filesystem("mmfs1").unmount(Nodes())
Parameters: - nodes (set) – a set of Nodes to unmount the filesystem on. If no set is given, the calling node is unmounted.
- async (bool) – when True, the call will return as soon as the unmount process has started. Otherwise, it will block until the unmount has completed
- timeout (int) – If the filesystem has not unmounted by the end of the timeout, an Exception is raised. The default is 300 seconds. To simulate an infinite timeout, use for example 86400, equating to 1 day.
- force (bool) – Forces the unmount to take place even though the filesystem may be still in use.
Warning
Use force with extreme caution. Using force may cause outstanding write operations to be lost. Because of this, forcing an unmount can cause data integrity failures and should be used with caution.
Raises: GPFSError
-
mounted¶ Returns true if the filesystem is mounted on this node
-
mountedOn(refresh=False)¶ Returns a set of Nodes on which the filesystem is mounted.
NB. This will trigger a load of the Nodes() collection, if you have not created the filesystem via the Cluster() object.
Parameters: refresh (bool) – rescan the system, rather than using Cached information Return type: dict of Node objects
-
tempFilePath¶ Path into which temporary files will be created.
Used by
NamedTemporaryFileandmkdtemp
-
NamedTemporaryFile(**kwargs)¶ Return a temporary file, located in the globally accessible file space on this filesystem
Takes all the same parameters as tempfile.NamedTemporaryFile, excepting dir which is not valid
Return type: tempfile.NamedTemporaryFile
-
mkdtemp(**kwargs)¶ Return a temporary directory, located in the globally accessible file space on this filesystem.
Takes all the same parameters as tempfile.mkdtemp, excepting dir which is not valid
Returns: absolute pathname of the new directory Return type: str
-
totalFreeInodes¶ Returns the total number of unallocated inodes for the filesystem.
Returns: unallocated inodes Return type: long
-
totalUsedInodes¶ Returns the total number of used inodes for the filesystem.
Returns: Used inodes Return type: long
-
totalAllocatedInodes¶ Returns the total number of allocated inodes for the filesystem.
Returns: Allocated inodes Return type: long
-
totalMaxInodes¶ Returns the maximum number of inodes for the filesystem
Returns: Maximum inodes Return type: long
-
totalSizeBytes¶ Returns the total capacity of the filesystem in bytes
Returns: Total filesystem capacity (bytes) Return type: long
-
contiguousSpaceBytes¶ Returns the amount of contiguous space in bytes for the filesystem
Returns: Total free contiguous space (bytes) Return type: long
-
fragmentedSpaceBytes¶ Returns the amount of fragmented space in bytes for the filesystem
Returns: Total free space in fragments (bytes) Return type: long
-
totalDataSizeBytes¶ Returns the amount of space in bytes allocated for data usage for the filesystem
Returns: Total filesystem data capacity (bytes) Return type: long
-
contiguousDataSpaceBytes¶ Returns the amount of contiguous space in bytes allocated for data usage for the filesystem
Returns: Total free data space (bytes) Return type: long
-
fragmentedDataSpaceBytes¶ Returns the amount of fragmented space in bytes allocated for data usage for the filesystem
Returns: Free data space in fragments (bytes) Return type: long
-
totalMetadataSizeBytes¶ Returns the amount of space in bytes allocated for metadata usage for the filesystem
Returns: Total filesystem metadata capacity (bytes) Return type: long
-
contiguousMetadataSpaceBytes¶ Returns the amount of contiguous space in bytes allocated for metadata usage for the filesystem
Returns: Total contiguous metadata space (bytes) Return type: long
-
fragmentedMetadataSpaceBytes¶ Returns the amount of fragmented space in bytes allocated for metadata usage for the filesystem
Returns: Total free metadata space in fragments (bytes) Return type: long
-
callbacks¶ Returns a collection of the callbacks that are installed on this object
Return type: Callbacks
-
onAfmCmdRequeued¶ Returns a
Callbackscollection, the members of which will be triggered during replication when messages are queued up again because of errors. These messages are retried after 15 minutes (Local Event)GPFS 4.2.1+ only
-
onAfmFilesetChange¶ Returns a
Callbackscollection, the members of which will be triggered when an AFM fileset is changed. If a fileset is renamed the new name is part of %reason. (Local Event)GPFS 4.2.3+ only.
-
onAfmFilesetCreate¶ Returns a
Callbackscollection, the members of which will be triggered when an AFM fileset is created. (Local Event)GPFS 4.2.3+ only.
-
onAfmFilesetDelete¶ Returns a
Callbackscollection, the members of which will be triggered when an AFM fileset is deleted. (Local Event)GPFS 5.0.0+ only.
-
onAfmFilesetExpired¶ Returns a
Callbackscollection, the members of which will be triggered when the contents of a fileset expire. (Global Event)
-
onAfmFilesetLink¶ Returns a
Callbackscollection, the members of which will be triggered when an AFM fileset is linked. (Local Event)GPFS 4.2.3+ only.
-
onAfmFilesetUnexpired¶ Returns a
Callbackscollection, the members of which will be triggered when the contents of a fileset become unexpired. (Global Event)
-
onAfmFilesetUnlink¶ Returns a
Callbackscollection, the members of which will be triggered when a AFM fileset is unlinked. (Local Event)GPFS 4.2.3+ only.
-
onAfmFilesetUnmounted¶ Returns a
Callbackscollection, the members of which will be triggered when the fileset is moved to an Unmounted state because NFS server is not reachable or remote cluster mount is not available for GPFS Native protocol. (Local Event)GPFS 4.1.0+ only
-
onAfmHomeConnected¶ Returns a
Callbackscollection, the members of which will be triggered when a gateway node connects to the home NFS server of the fileset that it is serving. (Local Event)
-
onAfmHomeDisconnected¶ Returns a
Callbackscollection, the members of which will be triggered when a gateway node gets disconnected from the home NFS server of the fileset that it is serving. (Local Event)
-
onAfmManualResyncComplete¶ Returns a
Callbackscollection, the members of which will be triggered when a manual resync is completed. (Local Event)
-
onAfmPrepopEnd¶ Returns a
Callbackscollection, the members of which will be triggered when all the files specified by a prefetch operation have been cached successfully. (Local Event)
-
onAfmQueueDropped¶ Returns a
Callbackscollection, the members of which will be triggered when replication encounters an issue that cannot be corrected. After the queue is dropped, next recovery action attempts to fix the error and continue to replicate. (Local Event)GPFS 4.2.1+ only
-
onAfmRPOMiss¶ Returns a
Callbackscollection, the members of which will be triggered when Recovery Point Objective (RPO) is missed on DR primary filesets, RPO Manager keeps retrying the snapshots. This event occurs when there is lot of data to replicate for the RPO snapshot to be taken or there is an error such as, deadlock and recovery keeps failing. (Local Event)GPFS 4.2.1+ only.
-
onAfmRecoveryEnd¶ Returns a
Callbackscollection, the members of which will be triggered when AFM recovery ends. (Local Event)
-
onAfmRecoveryFail¶ Returns a
Callbackscollection, the members of which will be triggered when recovery fails. The recovery action is retried after 300 seconds. If recovery keeps failing, fileset is moved to a resync state if the fileset mode allows it. (Local Event)GPFS 4.2.1+ only
-
onAfmRecoveryStart¶ Returns a
Callbackscollection, the members of which will be triggered when AFM recovery starts. (Local Event)
-
onDiskFailure¶ Returns a
Callbackscollection, the members of which will be triggered on the file system manager node when the disk status in the files system changes to down. (Local Event)
-
onDiskIOHang¶ Returns a
Callbackscollection, the members of which will be triggered when the GPFS daemon detects that a local I/O request has been pending in the kernel for more than five minutes. (Local Event)GPFS 5.0.2+ only
-
onFilesetLimitExceeded¶ Returns a
Callbackscollection, the members of which will be triggered when the file system manager detects that a fileset quota has been exceeded. (Local Event)Note
GPFS recommends using the
softQuotaExceededevent instead.
-
onFsstruct¶ Returns a
Callbackscollection, the members of which will be triggered when the file system manager detects a file system structure (FS Struct) error. (Local Event)GPFS 4.2.1+ only
-
onLowDiskSpace¶ Returns a
Callbackscollection, the members of which will be triggered when the file system manager detects that disk space is running below the low threshold that is specified in the current policy rule. (Local Event)Note
This event is triggered every two minutes until the condition is solved.
-
onMount¶ Returns a
Callbackscollection, the members of which will be triggered when a file system is mounted successfully. (Local Event)
-
onNoDiskSpace¶ Returns a
Callbackscollection, the members of which will be triggered when the file system encounters a disk that ran out of space. (Local Event)Note
This event is triggered every two minutes until the condition is solved.
-
onPreMount¶ Returns a
Callbackscollection, the members of which will be triggered when a file system is about to be mounted. (Local Event)
-
onPreUnmount¶ Returns a
Callbackscollection, the members of which will be triggered when a file system is about to be unmounted. (Local Event)
-
onSnapshotCreated¶ Returns a
Callbackscollection, the members of which will be triggered after a snapshot is created, and run before the file system is resumed. (Local Event)
-
onSoftQuotaExceeded¶ Returns a
Callbackscollection, the members of which will be triggered when the file system manager detects that a soft quota limit (for either files or blocks) has been exceeded. (Local Event)
-
onUnmount¶ Returns a
Callbackscollection, the members of which will be triggered when a file system is unmounted successfully. (Local Event)
-
onUsageUnderSoftQuota¶ Returns a
Callbackscollection, the members of which will be triggered when the file system manager detects that quota usage has dropped below soft limits and grace time is reset. (Local Event)
-
Examples¶
Utilising Filesystem from the Cluster Object¶
>>> from arcapix.fs.gpfs import Cluster
>>>
>>> # Load the cluster
... mycluster = Cluster()
>>>
>>> # Print the block size of all file systems
... for filesys in mycluster.filesystems.values():
... print("Name: {0}\tBlockSize: {1}".format(filesys.deviceName, filesys.blockSize))
...
Name: mmfs1 Block size: 4194304
Utilising Filesystem Directly¶
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> # Create a new file system object representing file system mmfs1
... # This file system already exists and we know exactly what we are doing
... filesys = Filesystem('mmfs1')
>>>
>>> print(filesys.blockSize)
4194304
Mounting/Unmounting¶
>>> from arcapix.fs.gpfs import Filesystem, Node, Nodes
>>>
>>> # Create a new file system object representing file system mmfs1
... # This file system already exists and we know exactly what we are doing
... filesys = Filesystem('mmfs1')
>>>
>>> # Unmount filesystem on this node
... filesys.unmount()
>>>
>>> # Mount filesystem on all nodes
... filesys.mount(Nodes())
>>>
>>> # Start filesystem unmount, but don't wait for it to complete
... filesys.unmount([Node('xxx')], async_=True)
>>>
>>> # Start filesystem mount, timeout if not complete in 10 seconds and raise an Exception
... filesys.unmount([Node('xxx')], timeout=10)
Restripe the Filesystem After a Deferred Move¶
>>> from arcapix.fs.gpfs import File, Filesystem
>>> from arcapix.fs.gpfs.filesystem import RsFlags
>>>
>>> f = File('/mmfs1/test.tmp")
... fs = Filesystem('mmfs1')
>>>
>>> # perform a defered move of the file to sas1
... f.change(storagepoolName='sas1', action='defer')
>>>
>>> # the file is now 'illplaced'
... print(f.flags)
['illplaced']
>>>
>>> # perform a restripe on the filesystem
... fs.restripe(RsFlags.repairPoolPlacement)
>>>
>>> # the file is no longer illplaced
... print(f.flags)
[]
Checking Free Inodes Before Creating a Preallocated Independent Fileset¶
>>> import uuid
>>> from arcapix.fs.gpfs import IndependentFileset, Filesystem
>>>
>>> # Create a random UUID to use in the fileset's comment files
... uid = uuid.uuid4()
>>>
>>> # Create the filesystem object
... filesys = Filesystem('mmfs1')
>>>
>>> # Required IndependentFileset size in inodes
... requiredInodes = 200000
>>>
>>> # Check the free inodes on the filesystem then Create and independent fileset with a
>>> # size of 200000 nodes and preallocate 200000 inodes (I.E. all of the inodes)
>>>
>>> # PixStor automatically expands inode spaces for large directories etc. as data is created
>>> # therefore we should never actually allocate 100% of the storage, PixStor will do that
>>> # automatically through use (and run perfectly up to and including 100% full).
>>> # Therefore ensure we only allocate up to a maximum of 80% of the available storage
... comfortRatio = 0.8
>>>
>>> # Calculate the maximum size of the IndependentFileset inode space
>>> # This formula is applicable to filesystems with dedicated metadata disks and data
>>> # capacity significantly higher than metadata capacity
... maxAvailableInodes = (comfortRatio * (filesys.contiguousMetadataSpaceBytes +
... filesys.fragmentedMetadataSpaceBytes)) /
... filesys.inodeSize / filesys.defaultMetadataReplicas
>>>
>>> # For file systems with mixed-use [clustered] disks we need to take into account
>>> # the subblock size (32)
>>> # maxAvailableInodes = (comfortRatio * ( filesys.contiguousMetadataSpaceBytes +
>>> # filesys.fragmentedMetadataSpaceBytes)) /
>>> # (filesys.inodeSize + (filesys.blockSize / 32)) /
>>> # filesys.defaultMetadataReplicas
>>>
>>> # Check if enough inodes are free to create the preallocated IndependentFileset
... if requiredInodes < maxAvailableInodes:
>>>
>>> # Create the fileset with pre-allocated inodes
... myfileset = IndependentFileset('mmfs1', 'indep-fileset1', uid,
... maxInodes=requiredInodes, allocInodes=requiredInodes)
...
... myfileset.create()
...
... # Link the fileset to the mmfs1 file system
... myfileset.link('/mmfs1/example-indep-fset-dir')
...
... # Unlink the fileset
... myfileset.unlink()
...
... # Remove the fileset
... myfileset.delete()
Calculating the Total Percentage of Space Remaining on a Filesystem¶
>>> from arcapix.fs.gpfs import Filesystem
>>> fs = Filesystem('mmfs1')
>>> #calculate the percentage of data space avalible
... percentage = fs.contiguousSpaceBytes/fs.totalSizeBytes*100
>>>
>>> print("mmfs1 has %.2f per cent total storage space available" % percentage)
mmfs1 has 97.22 per cent total storage space available