Lustre Best Practices
These guidelines were taken mainly from NASA.
1 – Always avoid using “ls -l”. The additional information requested (ownership, file permission, size) is stored on the metadata servers (MDT) and Object Store Targets (OST) so this information needs to access these servers for each file. Try the simple “ls” command or, if you know the name of the file, use it.
2 – Avoid placing a large number of files in a single directory. Placing many files on one directory creates contention. It is strongly recommended to split a large number of files (in the 100s or more) into multiple sub-directories.
3 – Avoid accessing small files on Lustre File Systems.
4 – Avoid accessing executables on the Lustre FS
5 – Avoid having multiple processes open the same file(s) at the same time