The OPAL Discussion Forum

[Rob Nagler <opal-rtnkh AT q33.us>]

I am trying to upgrade our build environment to Fedora 32, which uses gcc 10.2.1. The compile started thrashing on an 8GB virtual machine (VM) so I bumped it up to 16GB, and it still thrashed. My first attempts were with Trilinos 13.0.1, which is not recommended. However, I determined the memory explosion was caused by a single file MGPoissonSolver.cpp, and more specifically line 396 (and 408):

        prec_mp = MueLu::CreateTpetraPreconditioner(At, MueLuList_m);

If I comment out the CreateTpetraPreconditioner calls, the compile proceeds without the memory issue (still uses 3.5GB) and finishes in 3 minutes.

I downgraded to Trilinos 12.10.1, and was able to compile with the CreateTpetraPreconditioner calls, but it still required a VM with 16GB. Anything smaller, and the compile would thrash. When the compile is successful, it takes 10 minutes (just this file). The resultant output file (MGPoissonSolver.cpp.o) is 384MB with gcc10 and 344MB with gcc8 (also 10 minutes to compile). With CreateTpetraPreconditioner line commented out,  MGPoissonSolver.cpp.o is 150MB (gcc10) and 128MB (gcc8).

The OPAL executable 513MB and libOPAL.a is 1.1GB. With the commented out version of MGPoissonSolver.cpp.o, libOPAL.a is 841MB and the executable is 413MB. 

I suspect there is some type of template issue that is exacerbated in Trilinos 13.0.1, but certainly is there in 12.10.1.

Since I was poking around at file sizes, I thought you'd like to know about some other issues. SDDSParser/column.cpp has one include file and three type declarations, but it compiles to 30MB. Many of the SDDSParser object files are over 10MB, which suggests another template issue. Even SDDSParser/version.cpp.o is 6.5MB, which seems to exist to get the version from an SDDS file, a simple operation. I suspect the use of templates in these files may be problematic. The SDDSParser object directory is 146MB. Other large files are OptimizeCmd.cpp.o (61MB), P3MPoissonSolver.cpp.o (23MB), Distribution.cpp.o (22MB), SampleCmd.cpp.o (25MB), and PartBunch.cpp.o (32MB).

However, MGPoissonSolver.cpp is the primary issue, which perhaps is a Trilinos issue. If you could address that, compiles would be a lot faster and smaller.

I am tracking this in https://github.com/radiasoft/download/issues/136. I didn't see how to register on gitlab.psi.ch, or I would have submitted an issue there.

Thanks,

Rob