Policy¶
GPFS provides two types of Policy:
- Placement Policy: tells its associated
Filesystem
where to place new files- Management Policy: can be used to move, delete, etc. files in a
Filesystem
or directory
All policies are made up of Rules
.
Description¶
-
class
arcapix.fs.gpfs.policy.
PlacementPolicy
(filesystem)¶ “A placement policy is a set of rules for defining how to handle new files on a filesystem
Parameters: filesystem (Filesystem or str) – The filesystem that the PlacementPolicy applies to -
save
(testOnly=False, cleanup=True)¶ Save (apply) the policy to the filesystem (using mmchpolicy)
Parameters: Warning
In order to save changes to the placement policy, a temporary file needs to be written in your
/<filesystem_default_mount>/.policytmp
folder.If you don’t have write permission for this folder, an error will be raised.
You can change the temporary directory to one you do have permission for using
filesystem.setTempFilePath
-
validate
()¶ Checks that the policy is valid.
Note
This won’t necessarily catch all errors that GPFS might thrown.
-
export_to_file
(filename, **kwargs)¶ Export the policy to a file, e.g for backup purposes.
Parameters: - filename (str) – The file to export the policy to.
- overwrite (bool) – If the specified filename exists, pass overwrite=True else an error will be raised
- excludeDisabled (bool) – Don’t write disabled (commented out) rules to file (default=False)
- excludeComments (bool) – Don’t write comments to file (default=False)
-
import_from_file
(filename)¶ Import the contents of an existing file into the policy.
Overwrites any existing content of the PlacementPolicy object with the contents of the file.
Parameters: filename (str) – The file to import from
-
name
¶ Returns the name of the policy
Either the filename or the filesystem it’s defined on, depending on the policy type.
Return type: str
-
sameas
(tocompare)¶ Verifies whether the policy has been modified vs the file given.
Parameters: tocompare – name of policy file to compare this policy object to Raises: Exception if the file does not exist. Returns: True if the current policy is the same as the file (NB. The comparison ignores white space etc. Return type: bool
-
-
class
arcapix.fs.gpfs.policy.
ManagementPolicy
(filename=None)¶ A management policy is a set of rules for manipulating existing files on a filesystem or directory
Parameters: filename (str) – Name of a file from which to an existing policy, or to which you’d like to export the policy. -
summary
(target=None, refresh=False, **kwargs)¶ The response from running a management policy.
This is parsed into a dictionary summarising the action and applicability of the policy to the target filesystem or directory
This includes information like the number of files chosen by each rule in the policy or how pool occupancy will be affected by the policy.
Parameters: Return type:
-
save
(**kwargs)¶ Export the policy object to a (text) file, e.g. for backup purposes
Parameters: - filename (str) – Name of the file to save to
- overwrite (bool) – If filename already exists, specify True to force overwrite else an error will be raised (default=False)
- excludeDisabled (bool) – Don’t write disabled (commented out) rules to file (default=False)
- excludeComments (bool) – Don’t write comments to file (default=False)
-
run
(target, **kwargs)¶ Run the policy on the specified target (using mmapplypolicy)
Parameters: - target – target filesystem or directory
- nodes – list of names of nodes to run on (default=local ONLY)
- action – ‘yes’, ‘defer’, ‘test’, ‘prepare’ (default=’yes’)
- localWorkDirectory – Specifies the directory to be used for temporary storage (default=’<filesystem_default_mount>/.policytmp’)
- globalWorkDirectory – Specifies a global directory to be used for temporary storage (must be within the shared filesystem)
- threadLevel – Number of threads used in policy the execution phase (default=24).
- dirThreadLevel – Number of threads used during the directory scan phase (default=24).
- maxFiles – Specifies how many files are passed for each invocation of the EXEC script (default=8000).
- singleInstance – Only one policy with the singleInstance option can execute at one time on a filesystem (default=False)
- cleanup – remove any temporary policy file after completion (default=True)
Warning
If you haven’t saved your policy to a file, it will be saved to a temporary file in your
/<filesystem_default_mount>/.policytmp
folder.If you don’t have write permission for this folder, an error will be raised.
In this case you should call
save()
before callingrun()
.Alternatively, you can change the temporary directory to one you do have permission for using
filesystem.setTempFilePath
-
name
¶ Returns the name of the policy
Either the filename or the filesystem it’s defined on, depending on the policy type.
Return type: str
-
sameas
(tocompare)¶ Verifies whether the policy has been modified vs the file given.
Parameters: tocompare – name of policy file to compare this policy object to Raises: Exception if the file does not exist. Returns: True if the current policy is the same as the file (NB. The comparison ignores white space etc. Return type: bool
-
validate
()¶ Checks that the policy is valid.
Note
This won’t necessarily catch all errors that GPFS might thrown.
-
Examples¶
Defining a default Placement Policy¶
>>> from arcapix.fs.gpfs import Filesystem, SetPoolRule
>>>
>>> # Get the placement policy object for Filesystem 'mmfs1'
... mypolicy = Filesystem('mmfs1').policy
>>>
>>> # Create a new 'Set Pool' rule
... mypolicy.rules.new(SetPoolRule, name='default', target='system')
>>>
>>> # Validate
... mypolicy.validate()
>>>
>>> # Apply policy to the filesystem
... mypolicy.save()
Load the running Placement Policy and export to file¶
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> # Get the placement policy object for Filesystem 'mmfs1'
... mypolicy = Filesystem('mmfs1').policy
>>>
>>> # Print default placement rule string
... print mypolicy.rules['default'].toGpfsString()
"RULE 'default' SET POOL 'system'"
>>>
>>> # Export policy to file
... mypolicy.export_to_file('mmfs1-running.pol')
Creating and run a Management Policy to delete temporary files¶
>>> from arcapix.fs.gpfs import Cluster, ManagementPolicy, Criteria, DeleteRule
>>>
>>> # Create a management policy object
... mypolicy = ManagementPolicy()
>>>
>>> # Create a new 'Delete' rule
... r = mypolicy.rules.new(DeleteRule, source='pool_1')
>>>
>>> # Add criteria that select temporary files
... r.criteria.new(Criteria.like('name', '*.tmp'))
>>>
>>> # Get target filesystem mmfs1
... mmfs1 = Cluster().filesystems['mmfs1']
>>>
>>> # Run the policy on the filesystem
... Cluster().runPolicy(mypolicy, mmfs1)
Load and modify an existing Management Policy file¶
>>> from arcapix.fs.gpfs import ManagementPolicy, Criteria, ExcludeRule
>>>
>>> # Load a management policy object from file
... mypolicy = ManagementPolicy('tidyup.pol')
>>>
>>> # Create a new 'Exclude' rule
... r = mypolicy.rules.new(ExcludeRule)
>>>
>>> # Add criteria to select files less than 30 days old for exclusion
... r.criteria.new(Criteria.lt('creation', 30))
>>>
>>> # Save changes to file
... mypolicy.save()