Some Cheyenne users have recently reported some application slowdown when using the MPT library. These slowdowns were traced to using binaries that were compiled with mpt/2.22 in jobs that load the now-default mpt/2.25 MPI library. Incompatibilities between the remote-data transfer backends of each MPT version explain this substandard performance seen in certain workflows.
This scenario illustrates the benefits of setting your job environment (including loading modules) within the batch script itself. It is especially important if you have not yet recompiled your program to use the new default module. Include a line like this in your script to specify the versions of modules your older job needs to run properly:
module load intel/19.0.5 mpt/2.22
This practice simplifies debugging tasks and typically increases reproducibility. Explicit environment configuration in the job script serves as a record of your compile-time configuration and protects your job against any changes to the default configuration the job would otherwise be assigned.
