Policy¶

GPFS provides two types of Policy:

Placement Policy: tells its associated Filesystem where to place new files

Management Policy: can be used to move, delete, etc. files in a Filesystem or directory

All policies are made up of Rules.

Description¶

class arcapix.fs.gpfs.policy.PlacementPolicy(filesystem)¶

“A placement policy is a set of rules for defining how to handle new files on a filesystem

Parameters:	filesystem (Filesystem or str) – The filesystem that the PlacementPolicy applies to

id¶

Returns the identifier for this policy

Synonym for filesystem

Return type:	str

filesystem¶

Returns the name of the filesystem the policy is applied to

Return type:	str

save(testOnly=False, cleanup=True)¶

Save (apply) the policy to the filesystem (using mmchpolicy)

Parameters:	testOnly (bool) – Runs mmchpolicy in test mode (default=False) cleanup (bool) – Remove temporary policy file on completion (default=True)

Warning

In order to save changes to the placement policy, a temporary file needs to be written in your /<filesystem_default_mount>/.policytmp folder.

If you don’t have write permission for this folder, an error will be raised.

You can change the temporary directory to one you do have permission for using filesystem.setTempFilePath

validate()¶: Checks that the policy is valid.

Note

This won’t necessarily catch all errors that GPFS might thrown.

export_to_file(filename, **kwargs)¶

Export the policy to a file, e.g for backup purposes.

Parameters:	filename (str) – The file to export the policy to. overwrite (bool) – If the specified filename exists, pass overwrite=True else an error will be raised excludeDisabled (bool) – Don’t write disabled (commented out) rules to file (default=False) excludeComments (bool) – Don’t write comments to file (default=False)

import_from_file(filename)¶

Import the contents of an existing file into the policy.

Overwrites any existing content of the PlacementPolicy object with the contents of the file.

Parameters:	filename (str) – The file to import from

name¶

Returns the name of the policy

Either the filename or the filesystem it’s defined on, depending on the policy type.

Return type:	str

rules¶

Returns the collection of policy rules

Return type:	Rules

sameas(tocompare)¶

Verifies whether the policy has been modified vs the file given.

Parameters:	tocompare – name of policy file to compare this policy object to
Raises:	Exception if the file does not exist.
Returns:	True if the current policy is the same as the file (NB. The comparison ignores white space etc.
Return type:	bool

class arcapix.fs.gpfs.policy.ManagementPolicy(filename=None)¶

A management policy is a set of rules for manipulating existing files on a filesystem or directory

Parameters:	filename (str) – Name of a file from which to an existing policy, or to which you’d like to export the policy.

filename¶

Name of the file the policy is read from/written to.

Synonym for id

Return type:	str

summary(target=None, refresh=False, **kwargs)¶

The response from running a management policy.

This is parsed into a dictionary summarising the action and applicability of the policy to the target filesystem or directory

This includes information like the number of files chosen by each rule in the policy or how pool occupancy will be affected by the policy.

Parameters:	target (str) – filesystem or directory to run policy against refresh (bool) – reload the summary (if the policy has changed since the last call to ‘run’ or ‘summary’ (default=False)
Return type:	dict

save(**kwargs)¶

Export the policy object to a (text) file, e.g. for backup purposes

Parameters:	filename (str) – Name of the file to save to overwrite (bool) – If filename already exists, specify True to force overwrite else an error will be raised (default=False) excludeDisabled (bool) – Don’t write disabled (commented out) rules to file (default=False) excludeComments (bool) – Don’t write comments to file (default=False)

run(target, **kwargs)¶

Run the policy on the specified target (using mmapplypolicy)

Parameters:

target – target filesystem or directory
nodes – list of names of nodes to run on (default=local ONLY)
action – ‘yes’, ‘defer’, ‘test’, ‘prepare’ (default=’yes’)
localWorkDirectory – Specifies the directory to be used for temporary storage (default=’<filesystem_default_mount>/.policytmp’)
globalWorkDirectory – Specifies a global directory to be used for temporary storage (must be within the shared filesystem)
threadLevel – Number of threads used in policy the execution phase (default=24).
dirThreadLevel – Number of threads used during the directory scan phase (default=24).
maxFiles – Specifies how many files are passed for each invocation of the EXEC script (default=8000).
singleInstance – Only one policy with the singleInstance option can execute at one time on a filesystem (default=False)
cleanup – remove any temporary policy file after completion (default=True)

Warning

If you haven’t saved your policy to a file, it will be saved to a temporary file in your /<filesystem_default_mount>/.policytmp folder.

If you don’t have write permission for this folder, an error will be raised.

In this case you should call save() before calling run().

Alternatively, you can change the temporary directory to one you do have permission for using filesystem.setTempFilePath

name¶

Returns the name of the policy

Either the filename or the filesystem it’s defined on, depending on the policy type.

Return type:	str

rules¶

Returns the collection of policy rules

Return type:	Rules

sameas(tocompare)¶

Verifies whether the policy has been modified vs the file given.

Parameters:	tocompare – name of policy file to compare this policy object to
Raises:	Exception if the file does not exist.
Returns:	True if the current policy is the same as the file (NB. The comparison ignores white space etc.
Return type:	bool

validate()¶: Checks that the policy is valid.

Note

This won’t necessarily catch all errors that GPFS might thrown.

Examples¶

Defining a default Placement Policy¶

>>> from arcapix.fs.gpfs import Filesystem, SetPoolRule
>>>
>>> # Get the placement policy object for Filesystem 'mmfs1'
... mypolicy = Filesystem('mmfs1').policy
>>>
>>> # Create a new 'Set Pool' rule
... mypolicy.rules.new(SetPoolRule, name='default', target='system')
>>>
>>> # Validate
... mypolicy.validate()
>>>
>>> # Apply policy to the filesystem
... mypolicy.save()

Load the running Placement Policy and export to file¶

>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> # Get the placement policy object for Filesystem 'mmfs1'
... mypolicy = Filesystem('mmfs1').policy
>>>
>>> # Print default placement rule string
... print mypolicy.rules['default'].toGpfsString()

"RULE 'default' SET POOL 'system'"

>>>
>>> # Export policy to file
... mypolicy.export_to_file('mmfs1-running.pol')

Creating and run a Management Policy to delete temporary files¶

>>> from arcapix.fs.gpfs import Cluster, ManagementPolicy, Criteria, DeleteRule
>>>
>>> # Create a management policy object
... mypolicy = ManagementPolicy()
>>>
>>> # Create a new 'Delete' rule
... r = mypolicy.rules.new(DeleteRule, source='pool_1')
>>>
>>> # Add criteria that select temporary files
... r.criteria.new(Criteria.like('name', '*.tmp'))
>>>
>>> # Get target filesystem mmfs1
... mmfs1 = Cluster().filesystems['mmfs1']
>>>
>>> # Run the policy on the filesystem
... Cluster().runPolicy(mypolicy, mmfs1)

Load and modify an existing Management Policy file¶

>>> from arcapix.fs.gpfs import ManagementPolicy, Criteria, ExcludeRule
>>>
>>> # Load a management policy object from file
... mypolicy = ManagementPolicy('tidyup.pol')
>>>
>>> # Create a new 'Exclude' rule
... r = mypolicy.rules.new(ExcludeRule)
>>>
>>> # Add criteria to select files less than 30 days old for exclusion
... r.criteria.new(Criteria.lt('creation', 30))
>>>
>>> # Save changes to file
... mypolicy.save()