Recalling

Transparent Recall

With Ngenea HSM Transparent Recall, files are recalled from external storage when opened for reading or writing by any application.

See Transparent Recall file system policy for details of the required policy configuration.

Policy Based Recall

Files may be recalled programmatically using a Spectrum Scale/GPFS policy. A policy will select candidates for recall based on specified criteria, and then call ngrecall in the execution phase to recall files from external storage.

Example Recall Policy

This example policy will recall all offline files in the /mmfs1/policytest/ folder tree.

/* Set the default filesystem storage pool */
define(targetpool,'sas1')

RULE EXTERNAL POOL 'NGENEA_DEFAULT'
    EXEC '/var/mmfs/etc/mmpolicyExec-ngenea-hsm'
    OPTS '-v1'

RULE 'ngenea_recall' MIGRATE FROM POOL 'NGENEA_DEFAULT' TO POOL targetpool

/* Select offline files only */
WHERE XATTR('dmapi.APXguuid') IS NOT NULL AND CountSubstr(MISC_ATTRIBUTES,'V')>0

/* Recall files in the policytest folder */
AND PATH_NAME LIKE '/mmfs1/policytest/%'

A more comprehensive example can be found here.

ngrecall

Synopsis

ngrecall [-vLEVEL] [--config-file=FILE] [--[no-]delete-remote]
         FILE1 ... FILEn
ngrecall [-vLEVEL] [--config-file=FILE] --premigrate [--overwrite]
         [--default-mode=MODE] [--default-uid=USER]
         [--default-gid=GROUP] FILENAME1 ... FILENAMEn
ngrecall [-vLEVEL] [--config-file=FILE] --premigrate [--overwrite]
         [--default-mode=MODE] [--default-uid=USER]
         [--default-gid=GROUP] -L REMOTE_LOC
ngrecall [-vLEVEL] [--config-file=FILE] --stub [--overwrite]
         [--default-stub-size=LENGTH|--force-stub-size=LENGTH]
         [--default-mode=MODE] [--default-uid=USER]
         [--default-gid=GROUP] FILENAME1 ... FILENAMEn
ngrecall [-vLEVEL] [--config-file=FILE] --stub [--overwrite]
         [--default-stub-size=LENGTH|--force-stub-size=LENGTH]
         [--default-mode=MODE] [--default-uid=USER]
         [--default-gid=GROUP] -L REMOTE_LOC
ngrecall [-v2|3] [--config-file=FILE]
         [--log-target=syslog] :FSID:INO:IGEN

Description

Recalls files from a Storage Target to the local GPFS file system.

Options

--config-file=FILE
                path to a configuration file to use.
                Default: /opt/arcapix/etc/ngenea.conf
--default-mode=MODE
                file mode bits (in octal format) to set for a
                stubbed/premigrated file if there are no file mode bits
                associated with a remote object.
                Default: apply umask in the usual way to file mode 666.
                Compatible with the options: -p, --premigrate; --stub
--default-uid=USER
                file owner (a numeric user ID or a non-numeric user name) to
                set for a stubbed/premigrated file if there is no file owner
                associated with a remote object.
                Default: use the effective user ID as a file owner.
                Compatible with the options: -p, --premigrate; --stub
--default-gid=GROUP
                file group (a numeric group ID or a non-numeric group name) to
                set for a stubbed/premigrated file if there is no file group
                associated with a remote object.
                Default: use the effective group ID as a file group.
                Compatible with the options: -p, --premigrate; --stub
--default-stub-size=LENGTH
                default approximate length of a file stub. This setting can be
                overridden in a configuration file.
                Default: 0 (do not retrieve file content).
                Compatible with the option: --stub
--delete-remote delete each successfully recalled remote object from a storage
                endpoint where the object was stored (for storage endpoints
                supporting object deletion).
--no-delete-remote
                do not delete recalled remote objects from storage endpoints
                where they were stored.
--force-stub-size=LENGTH
                forcibly set file stub size.
                Default: do not forcibly set the stub size.
                Compatible with the option: --stub
--help          display this help and exit.
--log-target=syslog
                redirect all logging to the syslog.
--overwrite     overwrite local files if they already exist.
                Default: if a local file already exists, then report an error
                         and do not overwrite the local file.
                Compatible with the options: -p, --premigrate; --stub
-p, --premigrate
                retrieve content of local files and make them reference
                locations at storage endpoints where those files were stored
                (i.e. make each file premigrated just after retrieving its
                content).
                       Implies the option: --no-delete-remote
                Conflicts with the option: --stub
-L, --remote-loc-xattr=REMOTE_LOC
                location of a remote object to premigrate or create a local stub
                file.
                Compatible with the options: -p, --premigrate; --stub
--stub          create local stub files.
                       Implies the option: --no-delete-remote
                Conflicts with the option: -p, --premigrate
-v, --verbose[=LEVEL]
                verbosity level:
                0 = standard messages (also used when this option is absent);
                1 = additional informational and warning messages (default);
                2 = debug messages;
                3 = enable core dump and messages related to file locking;
                    print PID and current time with microsecond precision.
-V, --version   display version information and exit.

Examples

To recall a file from the associated Storage Target:

ngrecall /mmfs1/data/file1

To recall multiple files from their associated Storage Target:

ngrecall /mmfs1/data/file1 /mmfs1/data/file2

To recall all files starting with name file from the associated Storage Target:

ngrecall /mmfs1/data/file*

To recall all files starting with name file and name newfile from the associated Storate Target:

ngrecall /mmfs1/data/file* /mmfs1/data/newfile*

To recall all files, except hidden ("dot") files, within a directory from the associated Storage Target:

ngrecall /mmfs1/data/*

To recall all files, including hidden ("dot") files, within a directory from the associated Storage Target:

ngrecall /mmfs1/data/{.??,}*

To recall all files, except hidden ("dot") files, within two different directories from their associated Storage Target(s):

ngrecall /mmfs1/data/dir1/* /mmfs1/data/dir2/*

To recall all files, except hidden ("dot") files, within a directory from a storage target, overriding the current config with (E.G. a custom storage target):

ngrecall --config-file=/path/to/custom.conf /mmfs1/data/delivery/*

See Also