Fix up UWM testing logic in automated builds

[AlexanderRichert-NOAA]

Right now, test_ufswm.sh requires using a fork of ufs-weather-model in order to point to the modules to be tested. We should be able to use some search-and-replace on the modulefiles so that we can use the head of develop in the authoritative repo.

[climbfuji]

Fixed.

[AlexanderRichert-NOAA]

@rickgrubin-noaa where does this happen in the weekly build scripts as far as pointing ufs-weather-model to the new modules?

[rickgrubin-noaa]

@rickgrubin-noaa where does this happen in the weekly build scripts as far as pointing ufs-weather-model to the new modules?

The setting for BRANCH handles differing modules without having to change automation scripts.

[AlexanderRichert-NOAA]

The idea here is to be able to point test_ufswm.sh to the ufs-weather-model head of develop and have it run with the newly installed modules on any system. Re-opening the issue.
I think the approach here is to either have our weekly build logic build UWM modulefiles for a given system, or, probably more simply, do a search-and-replace on the path, and set appropriate environment variables to set each package version.

[rickgrubin-noaa]

The idea here is to be able to point test_ufswm.sh to the ufs-weather-model head of develop and have it run with the newly installed modules on any system. Re-opening the issue. I think the approach here is to either have our weekly build logic build UWM modulefiles for a given system, or, probably more simply, do a search-and-replace on the path, and set appropriate environment variables to set each package version.

test_ufswm.sh does point to ufs-weather-model@develop by default. Not setting any env variables would "run with the newly installed modules" on a system.

To me, it seems more work to modify scripts or CI/CD, but that said, no worries, as you see fit.

[AlexanderRichert-NOAA]

I don't follow-- how would ufs-weather-model know to look at the newly installed modules for a given system? Unless I'm missing something then I would predict that running ./rt.sh would lead it to automatically infer the platform and load the modules present in develop from modulesfiles/*lua. Can you walk through how running test_ufswm.sh would run ufs-weather-model/rt.sh in a way that it knows to point to the packages being tested?

[rickgrubin-noaa]

I don't follow-- how would ufs-weather-model know to look at the newly installed modules for a given system? Unless I'm missing something then I would predict that running ./rt.sh would lead it to automatically infer the platform and load the modules present in develop from modulesfiles/*lua. Can you walk through how running test_ufswm.sh would run ufs-weather-model/rt.sh in a way that it knows to point to the packages being tested?

ufs-weather-model@some-branch modifies modulefiles/ufs_<host>.<compiler>.lua so that it references appropriate pathing for MODULEPATH and stack compiler+MPI etc., and modifies ufs_common.lua to reference appropriately versioned stack components.

Using orion as an example, UFSWM_BRANCH can be set to some-branch and then test_ufswm.sh knows what branch to use to get the aforementioned modulefiles/ufs_<host>.<compiler>.lua and ufs_common.lua files.

This is how I tested it on orion, pointing at a specific branch in Ratko's ufs-weather-model repo that was being used to test latest spack-stack component versions.