Skip to Content.
Sympa Menu

opal - Re: [Opal] OPAL Monitor Memory Usage

opal AT

Subject: The OPAL Discussion Forum

List archive

Re: [Opal] OPAL Monitor Memory Usage

Chronological Thread  
  • From: Adelmann Andreas <andreas.adelmann AT>
  • To: Christopher Pierce <cmpierce AT>
  • Cc: "opal AT" <opal AT>, "opal-developers AT" <opal-developers AT>
  • Subject: Re: [Opal] OPAL Monitor Memory Usage
  • Date: Tue, 28 Nov 2023 06:48:58 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; 1; spf=pass (sender ip is; dmarc=pass (p=none sp=none pct=100) action=none; dkim=none (message not signed); arc=none (0)
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed;; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=enKSK5VdM/F1NNU6LP8FKFEBAnuDgS+XJMRqMwmbkMk=; b=ZcaQsxFWLLrIX6vg0lYfCnTl/45b/EdhNdoUIko0QVdfxmie443jgHKPXHJWK+O75JEZoOfGP4RNXl2rzu5TZqzrzJYCVxpKTHrMkjerf469F2QKjSRv9oBjTooOTGxQgXHoXY9bIF+FofsWskHbjzjKxANlZoW222As8SVvM0gt6OjpCMFB9w/vvnvOu8OGtfMTNgUqAQNTtb+nmBvCu/39m43hkJcBrYRaxANhGtdoCy4/PEETK4LBSMm9aHfwGH2KAsVtG4K3c8kLHcyG2UvCJWDL1LVRSEuKj1L/ZQwXgGV+Rls/yS7bzKa0D/ieDKZeRaLPzkLY77pehVUmyQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901;; cv=none; b=YHbcgfymkitCpwlF2xvVvoFw1Fk4Wa5yLfCNrkpmGjGbyosjHsDb14qzouNVpQHkGOKJ9c5PrCyxxQiHUFRhATeCrnaN5j+4eQd3VvYXVLkHaKw6ip7q89YmiDgRE6cA5BMTp7lCVWfRPZsdYsHWOd7/ZK/pPP3SeVTdDPO9qiUmZA/tc+rbSSU4UlsCMVhOhweSy/UwwYH/2te6gXkQZMozv1j4vVGTOWz2g1USV5JJfxWT6tMHGyt3E9vssqxetf8Ufkro3wI8SRIgP3i1N5BHIhpyeCQxbiQQWGl8lBU9zQ/Rr+NAU2iEe9N3DnFIXQWMT/xDWuYQi5n8VUaJ7g==
  • Authentication-results:; iprev=pass ( smtp.remote-ip=; spf=pass; dkim=pass header.s=selector2 header.a=rsa-sha256; dmarc=pass
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none;

Hi Chris, thank you so much for the flowers and reporting the bug will open an issue today. 
Maybe we can add your fix this to the 2023.1 release. Please send me the changed file so I can
see exactly what you did!
Cheers and many thanks  Andy 
Dr. sc. math. Andreas (Andy) Adelmann
Paul Scherrer Institut OHSA/D17 CH-5232 Villigen PSI
Phone Office: xx41 56 310 42 33 Fax: xx41 56 310 31 91
Zoom ID: 470-582-4086 Password: AdA
Zoom Link:

Friday: ETH HPK G 28   +41 44 633 3076
The more exotic, the more abstract the knowledge, 
the more profound will be its consequences.
Leon Lederman 

On 27 Nov 2023, at 17:00, Christopher Pierce <cmpierce AT> wrote:

Hey folks,

I have been using OPAL for some space charge particle tracking on a recent project and have been having a great experience with it so far. Thanks for the nice code!

One thing I ran into recently is an out-of-memory issue when I run it with many "monitor" elements. What seems to be happening is that the memory allocated while saving the particles to disk doesn't get freed after the file is output. In my case, I was using 2 million particles and I could see the memory usage rise by 200MB as the first monitor output is saved to disk and then rise by another 200MB at the second and so on. Since I was producing video-like output with 48 monitors, it didn't take long to exceed our cluster's 4GB per core limit.

I did some digging in the source code and on line 367 in `src/Classic/Structure/LossDataSink.cpp` I saw that the vectors storing particle information in the monitor objects do get cleared. Reading online, though (since I'm not a C++ expert) I saw some people saying this might not free memory ( I implemented their suggestion, ie the following code to replace all of the clear methods.


After recompiling the code, the memory growth seems to have gone away and I can run it on the cluster again without going over our RAM limits.

I wanted to pass this along to see if it's the right thing to do to deal with the memory problem I was having and also in case  anyone else finds it useful.



Archive powered by MHonArc 2.6.24.

Top of Page