Querying¶

Listing file status¶

The ngenea status of files can be easily queried with the ngls tool.

For example:

# ngls sample.txt
  SIZE    RSS ST FILE NAME
    29      0 m sample.txt

The information displayed shows:

SIZE: file size (in bytes)

RSS: resident size on disk (in bytes)

ST: migration state: m for migrated, - for resident, p for premigrated (resident at both ends)

FILE NAME: file name

ngls¶

Synopsis¶

ngls [-h | --help] [-V | --version] [-H | --header] [-n | -a] [ -l ] [-d N | --depth=N]
    | [--output='format-specifier' || --output=ALL ]
    | [-m pattern | --match=pattern] file-or-dir-paths

Description¶

Lists Ngenea status of files.

Options¶

-n, -a                      natural(-n) or alphabetical (-a) sort of file names
                            natural sort is alphabetical, with embedded version numbers sorted numerically
-l                          display remote location
-d                          same as "-d1" or "-depth=1"
-dNNN, --depth=NNN          descend directories to a depth of NNN
-D                          same as "-d15"
-H, --header                don't print the directory header line
--output='format-specifier' output in the specified format, using printf-style format specifiers
--output=ALL                same as --output='%10s %6r %8p %08w %S %-50n %l'
                               %s file size
                               %r resident size
                               %p file permissions
                               %w win attrs
                               %S migration state ('-', 'm=migrated', 'p=premigrated')
                               %n file name
                               %l remote location
                            default is --output='%10s %6r %S %-50n'
                            or '%10s %6r %S %-50n %l' if -l also selected
-m'glob', --match='glob'    match the filename against the glob string
-r'regex', --regex='regex'  match the filename against the regular expression
                            (note: directory names are not matched)

-?, --help                  display this help and exit.
-V, --version               display version information and exit.

Examples¶

Default Output Format¶

Display data for jpg files in the current directory

ngls *.jpg

Display data for JPG files in the current directory and one level below

ngls -d --match='*.jpg'

Display data for JPG files in the current directory and 5 levels below

ngls -d5 --match='*.jpg'

Custom Output Format¶

Display selected items in comma-separated format

ngls --output='%n,%s,%r' -d

Querying extended attributes¶

All ngenea metadata associated with a file's migration are stored as extended attributes within the file inode.

As such, standard Spectrum Scale/GPFS tools can be used to show this information.

For example - mmlsattr can be used to show the migration information about a file:

# mmlsattr -d -L sample.txt
file name:            sample.txt
metadata replication: 2 max 2
data replication:     1 max 3
immutable:            no
appendOnly:           no
flags:
storage pool name:    sas1
fileset name:         root
snapshot name:
creation time:        Wed May 30 16:54:18 2018
Misc attributes:      ARCHIVE OFFLINE
Encrypted:            no
dmapi.APXguuid:       "a7e82a60-62ae-4c25-8da6-91d05f265714"
dmapi.APXrmtlc:       "ngenea_target1:test/sample.txt"
dmapi.APXsh512:       "0cfcd93fb5eae3f5c9962117b38882de79ea4f9c432d574e8c9a4b1b65cded4620d25dc1b29ef0e60d238219797cb4f7d3946ce3a0f718b849cff2ae41fffd1c"
dmapi.APXrmtsz:       "29"
gpfs.dmapi.region:    0xFFFFFFFFFFFFFFFF00000000000000000700000000000000

The relevant extended attributes include:

Misc attributes: the presence of the OFFLINE status here shows that the file is migrated, and is no longer resident on the file system

dmapi.APXguuid: Unique ID for the file, used to authoritatively match an inode to a remote object

dmapi.APXrmtlc: Remote location of the file, as defined in the RemoteLocationXAttrRegex setting for the activated storage target in the master configuration file, at the point it was migrated.

dmapi.APXsh512: SHA512 checksum of the file at the point it was migrated.

dmapi.APXrmtsz: Size of the object sent to the external storage.

gpfs.dmapi.region: Internal Spectrum Scale/GPFS attribute which reflects the offline status of the file

Note that these attributes must not be changed, except under explicit direction from ArcaStream / Pixit Media support. Doing so may result in data becoming unavailable, and may even result in data corruption/loss.