Execute¶

Dry Run¶

Dry run allows users to see what PixStor commands would be run by a piece of code without executing them. Commands are added to a command cache that can be retrieved using the getCmdCache function. When dry run mode is activated, commands are added to the cache until dry run mode is de-activated. Each time dry run mode is activated, the command cache is cleared.

Caveats¶

Dry run may cause some unexpected or undefined behaviour within the API, or may cause API methods to fail. In particular, when the API expects some response from the command call, an error will likely be raised.

The decorator or context manager will attempt to catch these dry run side-effect errors and write them to logging. Be aware that this may also catch errors that are not side effects of dry run, and that would otherwise be raised during normal execution. It is good practice to check logging for errors before re-running code outside of dry run mode.

Changing objects in dry run mode may result in those objects being changed within the API, but these chages will not be reflected on the Filesystem - E.G.

>>> setDryRun(True)
>>>
>>> fs.snapshots.new('test-snap')
>>>
>>> 'test-snap' in fs.snapshots
True
>>> fs = Filesystem('mmfs1')  # reload Filesystem object
>>> 'test-snap' in fs.snapshots
False

When the commands returned by dry run are actually executed on the Filesystem they may raise errors which aren’t presented by the dry run. E.G. In the decorator example below, attempting to delete the root fileset would raise an error outside of dry run.

In practice the returned commands would be run as sudo, however this is ommitted from the returned commands.

Dry run functionality is meant to be used with commands that don’t already provide test run functionality - E.G. when running a ManagementPolicy you should use the action='test' option rather than using dry run. Dry run will only return the mmapplypolicy command that would be run on PixStor.

Be aware that decorators and context managers automatically deactivate dry run mode on exit - even if setDryRun(True) has been used.

Description¶

arcapix.fs.gpfs.execute.FORCE_SUDO_ENV = 'AP_PYAPI_MUST_SUDO'¶: Environment variable which, when set, causes commands to be called with sudo for user root. For non-root users, commands are always called with sudo.

arcapix.fs.gpfs.execute.setShowDryRunCmds(activate=False)¶

Set whether the commands, which are captured by dry run, are shown on screen. E.G.

>>> setShowDryRunCmds(True)
>>>
>>> fs.snapshots['global-snapshot1'].delete()
INFO:arcapix.dryrun:mmdelsnapshot mmfs1 global-snapshot1

class arcapix.fs.gpfs.execute.CommandRunner¶

Baseclass for a command runner.

run(command)¶

Run a command.

Parameters:	command – command to run as a list
Returns:	stdout, stderr, returncode

class arcapix.fs.gpfs.execute.PopenRunner¶

Run a command using subprocess Popen.

run(command)¶

Run a command.

Parameters:	command – command to run as a list
Returns:	stdout, stderr, returncode

class arcapix.fs.gpfs.execute.StreamingPopenRunner(stdout_processor=None, stderr_processor=None)¶

Popen runner which allows processing streaming output.

The processor functions will receive an iterator of output lines, and should yield processed lines (if relevant)

This might be used to filter output, similar to piping output to grep in the cli

>>> def processor(stream):
...     for line in stream:
...         if line.startswith('[I]'):
...             yield line

Warning

If your processor modifies the stdout stream, it may cause parts of the API to break.

The stream lines will be byte strings, so will need to be decoded accordingly. Lines will include a trailing newline character.

The output from the processor functions must also be byte strings. The lines will be joined and returned from the run method

Hint

Stream processing is performed with separate threads for stdout and stderr. If your processor functions use ‘print’, you may need to apply locking to prevent output from overlapping.

stdout_processor(pipe, outbuf)¶

Process data from a stdout pipe.

Processed results should be inserted into the out buffer

stderr_processor(pipe, outbuf)¶

Process data from a stderr pipe.

Processed results should be inserted into the out buffer

run(command)¶

Run a command.

Parameters:	command – command to run as a list
Returns:	stdout, stderr, returncode

class arcapix.fs.gpfs.execute.DryRunRunner(readonly_runner=None)¶

Command dry-runner.

Read-only commands are executed as normal, but commands which would modify the filesystem are caught and the command in question is added to the commands list

Parameters:	readonly_runner – runner to execute readonly commands

run(command)¶

Run a command.

Parameters:	command – command to run as a list
Returns:	stdout, stderr, returncode

class arcapix.fs.gpfs.execute.GPFSCommandExecutor(runner=None)¶

Execute GPFS mm-commands and return parsed output.

For commands which support ‘programmer-friendly’ output the result is returned as a parsed dictionary.

Otherwise, the output is returned as a list of lines.

NOTE: blank lines are returned as None

Raises:	GPFSExecuteException is a command returns non-zero
Parameters:	runner – command runner to execute commands and return raw output

execute(cmd, ignorerc=False)¶

Parameters:	cmd – command to run as a string ignorerc – whether an error should be raised for non-zero return codes

class arcapix.fs.gpfs.execute.command_runner(runner)¶

Execute commands with an alternate CommandRunner.

Can be used as a context manager

>>> with command_runner(DryRunRunner()):
...     Cluster().filesystems.keys()

or decorator

>>> @command_runner(RemoteRunner())
... def example():
...     # do stuff

or can be manually activated/deactivated

>>> runner = command_runner(MockRunner())
>>> runner.activate()
>>> # do stuff

Note - if multiple command runners are nested the inner runner will replace any outer runners

activate()¶: Activate alternative command runner.

deactivate()¶: Deactivate alternative command runner.

arcapix.fs.gpfs.execute.setDryRun(activate=False)¶

Sets dry run mode, which will stop non-readonly commands from executing on the filesystem.

Any ‘caught’ commands will written to logging.

The commands are also stored in a command cache which can be retrieved using the getCmdCache function.

arcapix.fs.gpfs.execute.isDryRun()¶: Returns whether dry run mode is currently active.

arcapix.fs.gpfs.execute.resetCmdCache()¶: Clear the command cache

arcapix.fs.gpfs.execute.getCmdCache(index=None)¶

Returns the cache of commands which were caught and not executed during a dry run.

Return type:	list

class arcapix.fs.gpfs.execute.dryrun¶

Special class to run certain pieces of code as dry run.

Can be used as a context manager or a decorator

>>> with dryrun():
...     # do stuff

>>> @dryrun
... def my_func():
...     # do stuff

Examples¶

Set Dry Run globally¶

>>> from arcapix.fs.gpfs.execute import setDryRun, getCmdCache
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> setDryRun(True)
>>>
>>> fs = Filesystem('mmfs1')
>>>
>>> for s in fs.snapshots.values():
...    s.delete()
...
>>> print(getCmdCache())
['mmdelsnapshot mmfs1 global-snapshot1',
'mmdelsnapshot mmfs1 snap_test']

Dry Run decorator¶

>>> from arcapix.fs.gpfs.execute import dryrun, getCmdCache
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> fs = Filesystem('mmfs1')
>>>
>>> @dryrun
... def delete_filesets():
...     for f in fs.filesets.values():
...         f.delete(True, True, True)
...
>>> delete_filesets()
>>>
>>> print(getCmdCache())
['mmunlinkfileset mmfs1 root -f',
'mmdelfileset mmfs1 root -f',
'mmdelfileset mmfs1 sata1-cg-projects -f']

Dry Run a single method¶

>>> from arcapix.fs.gpfs.execute import dryrun, getCmdCache
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> fs = Filesystem('mmfs1')
>>>
>>> dryrun(fs.change)(manager='sn02')
>>>
>>> print(getCmdCache())
['mmchmgr mmfs1 sn02']

Dry Run context manager¶

>>> from arcapix.fs.gpfs.execute import dryrun, getCmdCache
>>> from arcapix.fs.gpfs import Filesystem
>>>
>>> fs = Filesystem('mmfs1')
>>>
>>> with dryrun():
...     fs.snapshots.new('snaptest')
...
>>> print(getCmdCache())
['mmcrsnapshot mmfs1 snaptest']