GPFS Poor man's checksumming

Spectrum Scale has support for end-to-end checksumming if you have an ESS, but not with non-ESS storage. As a poor man's checksumming, one can store file checksums in the extended attributes. Storing checksums here can be useful for validating if there has been bitrot at a later point, without leaving around .md5sum-files everywhere. Manually this can be done with the “mmchattr” command:

# md5sum testfile
d2aa3e121b13274d84c8388f453b0ca5  testfile
# mmchattr --set-attr user.cksum=md5:d2aa3e121b13274d84c8388f453b0ca5 testfile

# mmlsattr --get-attr user.cksum testfile
file name:            testfile
user.cksum:           "md5:d2aa3e121b13274d84c8388f453b0ca5"

To efficiently set this attribute on all files, we can utilize the policy engine. First we create a script that parses a policy engine file listing, and apply the checksum attribute on every file listed:

# cat <<'EOF' > /root/ 
#! /bin/bash -
# Arguments to this script:
# $@ = LIST /mnt/gpfs01/.mmSharedTmpDir/mmPolicy.ix.19751.02F6A4F0.1 7

IFS=$(echo -en "\n\b")

case $1 in
        TEST )
                exit 0
        LIST )
                awk -F ' -- ' '{print $2}' "$2" | while read file
                        MD5SUM=$(md5sum "$file")
                        mmchattr --set-attr user.cksum=md5:"${MD5SUM% *}" "$file"
# chmod +x /root/

Then we create an external list policy that call this script on all files that has not yet gotten the user.cksum attribute defined:

# cat <<'EOF' > apply-checksum.policy
RULE EXTERNAL LIST 'setChecksum' EXEC '/root/'
RULE 'findNonChecksummed' LIST 'setChecksum' WHERE xattr('user.cksum') IS NULL

And run it using:

# mmapplypolicy /path/to/folder -P apply-checksum.policy -I yes

Now this attribute can be used f.ex. in a migration policy, to make sure we only migrate out files to Spectrum Archive or TCT after a checksum has been defined for the files:

RULE 'archivePoolRule'
    EXTERNAL POOL 'ltfsee'
        EXEC '/opt/ibm/ltfsee/bin/eeadm'
        OPTS '-p pool1@library1,pool2@library2'

RULE 'migrateOffline'
    MIGRATE TO POOL 'ltfsee' 
        WHERE xattr('user.cksum') IS NOT NULL