File

The File object represents a file on the filesystem. It can be used to look-up and change attributes of a file.

This class can’t be used to create, destroy, or edit the contents of files.

Important

All time-based attributes are provided as naive datetime objects. However, creation time is given in the localtime zone whereas all other times are in Universal Coordinate Time (UTC). This is a quirk of GPFS. To convert all times to the same timezone, consider using the pytz library

Description

class arcapix.fs.gpfs.file.RsFlags

Flags which may be passed to the File restripe method:

  • criticalFromSuspended
  • allFromSuspended
  • repairPoolPlacement
  • repairReplicaConflicts
  • repairReadOnlyConflicts
  • rebalance
  • strictRebalance
  • relocateBlocks
  • runDeferedCompression
class arcapix.fs.gpfs.file.File(path)

File objects represent files in the GPFS filesystem

Create a File object

Parameters:path (str) – path to a file within the GPFS filesystem

Note: This is just a representation of an existing file. This does not create a new file on the filesystem.

change(**kwargs)

Change certain file attributes

Parameters:
  • action – ‘yes’ or ‘defer’ (default=’yes’)
  • storagepoolName – Name of pool to move file to
  • dataReplicas (int) – number of data replicas to make of file
  • maxDataReplicas (int) – maximum number of data replicas to make of file
  • metadataReplicas (int) – number of metadata replicas to make of file
  • maxMetadataReplicas (int) – maximum number of metadata replics to make of file
  • immutable (bool) – set the (im)mutability of the file
  • appendOnly (bool) – set whether the file should be append only

Note

maxDataReplicas and maxMetadataReplicas can only be changed for zero length files.

restripe(flag)

Perform a restripe action on the file.

Parameters:flag – The restripe action to perform. One of RsFlags
name

Returns the name of the file (excluding path)

Return type:str
id

Returns the name of the file (excluding path)

synonym for name

Return type:str
path

Returns the pathname of the file

Return type:str
metadataReplicas

Returns the number of metadata replicas of the file

Return type:int
maxMetadataReplicas

Returns the maximum number of metadata replicas that can be created of the file

Return type:int
dataReplicas

Returns the number of data replicas of the file

Return type:int
maxDataReplicas

Returns the maximum number of data replicas that can be created of the file

Return type:int
immutable

Returns whether the file is immutable

Return type:bool
appendOnly

Returns whether the file is in append only mode

Return type:bool
flags

Return any flags associated with the file

Return type:list
storagepoolName

Returns the name of the storage pool which the file resides in

Return type:str
poolName

Returns the name of the storage pool which the file resides in

Synonym for storagepoolName

Return type:str
filesetName

Returns the name of the fileset which the file resides in

Return type:str
snapshotName

Returns the name of the snapshot which the file belongs to

Returns None if not a snapshot file

Return type:str
creationTime

Returns the file’s creation time (local timezone)

Return type:datetime
miscAttributes

Returns miscellaneous attributes of the file such as whether the file is ARCHIVE or READONLY.

Older versions of GPFS call these ‘Windows Attributes’

Return type:list
encrypted

Returns whether the file is encrypted

Return type:bool
xattrs

Returns extended attributes of the file (if any)

Return type:XAttrs
accessTime

Returns the time when the file was last accessed (UTC)

Return type:datetime
userId

Returns the ID of the file owner

Return type:int

Returns the number of hard links to the file

Return type:int
modificationTime

Returns the time when the file (UTC) was last modified

Return type:datetime
inode

Returns the file’s inode number

Return type:int
groupId

Returns the ID of the group which owns the file

Return type:int
size

Returns the current size of the file (bytes)

Return type:int
deviceId

Returns the ID of the device that contains the directory entry

Return type:int
blocks

Returns the number of blocks that make up the file

Return type:int
blocksize

Returns the size of each block of the file (bytes)

Return type:int
changeTime

Returns time that the file metadata (UTC) was last changed.

Return type:datetime
sqlMiscAttributes

Returns various micalleneous attributes of the file

This is a string of characters, which indicate properties such as object type.

See MISC_ATTRIBUTES for a explanation of the returned string

Return type:str
generation

Returns the file’s generation number

A number that is incremented when an inode number is reused.

Return type:int
modificationSnapId

Returns the ID of the snapshot after which the file was last changed.

Return type:int
mode

Returns the file type and permissions

E.G ‘-rw-rw-r–’

Return type:str
kbAllocated

Returns the disk space allocated to the file (kilobytes)

Return type:int
rdeviceId

Returns the device type for a device

Return type:int
class arcapix.fs.gpfs.file.XAttrs(path)

Class representing the extended attributes of a file

Note

Only user extended attributes can be created, deleted, or changed.

You do not need to specify user.*

user

Returns user defined extended attributes

Return type:dict
system

Returns system extended attributes

Return type:dict
dmapi

Returns dmapi extended attributes

Return type:dict
new(key, value=None, force=False)

Set a user extended attribute

>>> f.xattrs.new('foo', 'bar')
>>> f.xattrs.user['foo']
bar
Parameters:
  • key – Name of new extended attribute
  • value – Value for new extended attribute
  • force – Whether this attribute should overwite any existing with the same name (default=False)
Raises:

GPFSExecuteException If attribute with name ‘key’ already exists and ‘force’ isn’t True

change(key, value=None)

Change a user extended attribute

Parameters:
  • key – Name of the user extended attribute to be changed
  • value – New value for the extended attribute
Raises:

KeyError If attribute with name ‘key’ doesn’t exist

destroy(key)

Delete a user extended attribute

Parameters:key – Name of the user extended attribute to delete
Raise:KeyError If attribute with name ‘key’ doesn’t exist

Examples

Make a file immutable

>>> from arcapix.fs.gpfs import File
>>>
>>> # create a new File object
... f = File('/mmfs/data/myfile.txt')
>>>
>>> # check the file's mutability
... f.immutable
False
>>>
>>> # make the file immutable
... f.change(immutable=True)
>>>
>>> f.immutable
True

Change the data replication for all .txt files using a Policy

>>> from arcapix.fs.gpfs import ManagementPolicy, ListProcessingRule, Criteria
>>>
>>> # define the processing function
... def processor(files):
...     files_changed = 0
...     for f in files:
...         f.change(dataReplicas=1)
...         files_change += 1
...     return files_changed
...
>>> # create a policy
... p = ManagementPolicy()
>>>
>>> # create the processing rule
... r = p.rules.new(ListProcessingRule, 'txt_repls', processor=processor)
>>>
>>> # add a criteria to only select txt files
... r.criteria.new(Criteria.like('NAME', '*.txt'))
>>>
>>> # Run the policy
... p.run('mmfs1')
{'txt_repls': 345578}
>>>
>>> f = File('/mmfs1/data/myfile.txt')
>>> f.dataReplicas
1

Restripe a File After a Defered Move

>>> from arcapix.fs.gpfs.file import File, RsFlags
>>>
>>> f = File('/mmfs1/test.tmp")
>>>
>>> # perform a defered move of the file to sas1
... f.change(storagepoolName='sas1', action='defer')
>>>
>>> # the file is now 'illplaced'
... print f.flags
['illplaced']
>>>
>>> # perform a restripe on the file
... f.restripe(RsFlags.repairPoolPlacement)
>>>
>>> # the file is no longer illplaced
... print f.flags
[]