opal AT lists.psi.ch

Subject: The OPAL Discussion Forum

List archive

[Opal] Increase in External Field Evaluation Time

From: Chris Hall <chall AT radiasoft.net>
To: opal AT lists.psi.ch
Subject: [Opal] Increase in External Field Evaluation Time
Date: Mon, 11 Jan 2021 13:43:51 -0700

Hi All,

I recently noticed that some OPAL simulations were taking much longer than expected. We upgraded to OPAL 2.4.0 from 2.2.0 not that long ago so I reran an old simulation I still had outputs for. From this I see that the average Wall and CPU times for "External field eval." have increased by almost a factor of 10 when I compare 2.4 to 2.2. This leads to a doubling of the total wall clock run time. None of the other individual timing results show significant difference between the two runs.

Is anyone aware of changes between the two versions that might have caused this, or any other factors that should be investigated that would lead to this?

I've attached the run logs (with step updates cut out) for the runs on versions 2.2 and 2.4 for reference.

Thanks!

Chris hall

Research Scientist | RadiaSoft

720-502-3928 x709 | chall AT radiasoft.net

radiasoft.net | sirepo.com

Ippl> CommMPI: Parent process waiting for children ...
Ippl> CommMPI: Child 12 ready.
Ippl> CommMPI: Child 7 ready.
Ippl> CommMPI: Child 8 ready.
Ippl> CommMPI: Child 15 ready.
Ippl> CommMPI: Child 1 ready.
Ippl> CommMPI: Child 2 ready.
Ippl> CommMPI: Child 4 ready.
Ippl> CommMPI: Child 5 ready.
Ippl> CommMPI: Child 6 ready.
Ippl> CommMPI: Started job 1 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 2 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 3 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 4 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 5 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 6 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 7 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 8 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 9 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 10 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 11 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 12 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 13 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 14 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Started job 15 on host `fnl11b.bivio.biz'.
Ippl> CommMPI: Child 9 ready.
Ippl> CommMPI: Child 10 ready.
Ippl> CommMPI: Child 11 ready.
Ippl> CommMPI: Child 13 ready.
Ippl> CommMPI: Child 14 ready.
Ippl> CommMPI: Child 3 ready.
Ippl> CommMPI: Initialization complete.
{0}> ____ _____ ___
{0}> / __ \| __ \ /\ | |
{0}> | | | | |__) / \ | |
{0}> | | | | ___/ /\ \ | |
{0}> | |__| | | / ____ \| |____
{0}> \____/|_| /_/ \_\______|
OPAL{0}>
OPAL{0}> This is OPAL (Object Oriented Parallel Accelerator Library) Version
2.2.0
OPAL{0}> git rev.
OPAL{0}>
OPAL{0}>
OPAL{0}> (c) PSI, http://amas.web.psi.ch
OPAL{0}>
OPAL{0}>
OPAL{0}> The optimiser (former opt-Pilot) is integrated
OPAL{0}>
OPAL{0}> Please send cookies, goodies or other motivations (wine and beer ...
)
OPAL{0}> to the OPAL developers opal AT lists.psi.ch
OPAL{0}>
OPAL{0}> Time: 19:18:57 date: 10/08/2020
OPAL{0}>
OPAL{0}> Couldn't find startup file "/home/vagrant/init.opal".
OPAL{0}> Note: this is not mandatory for an OPAL simulation!
OPAL{0}>
OPAL{0}> * Reading input stream "opal_0.in".
OPAL{0}> *
**********************************************************************************

OPAL{0}> * Selected Tracking Method == PARALLEL-T, NEW TRACK
OPAL{0}> *
**********************************************************************************

OPAL{0}>
OPAL{0}> *
**********************************************************************************
OPAL{0}> * Write initial distribution to file "data/opal_0_GEN_DIST.dat"
OPAL{0}> *
**********************************************************************************
Ippl{0}> Creating new file for data: opal_0.stat
Ippl{0}> Creating new file for data: opal_0.lbal
OPAL{0}>
OPAL{0}> * ************* D I S T R I B U T I O N
********************************************
OPAL{0}> *
OPAL{0}> * Number of particles: 50000
OPAL{0}> *
OPAL{0}> * Distribution type: FLATTOP
OPAL{0}> *
OPAL{0}> * SIGMAX = 5.880351e-03 [m]
OPAL{0}> * SIGMAY = 5.880351e-03 [m]
OPAL{0}> * Sigma Time Rise = 1.204546e-12 [sec]
OPAL{0}> * TPULSEFWHM = 0.000000e+00 [sec]
OPAL{0}> * Sigma Time Fall = 1.204546e-12 [sec]
OPAL{0}> * Longitudinal cutoff = 4.000000e+00 [units of Sigma Time]
OPAL{0}> * Flat top modulation amplitude = 0.000000e+00 [Percent of
distribution amplitude]
OPAL{0}> * Flat top modulation periods = 0.000000e+00
OPAL{0}> *
OPAL{0}> * Number of energy bins = 9
OPAL{0}> * Distribution is emitted.
OPAL{0}> * Emission time = 6.799879e-12 [sec]
OPAL{0}> * Time per bin = 7.555421e-13 [sec]
OPAL{0}> * Delta t during emission = 6.868565e-14 [sec]
OPAL{0}> *
OPAL{0}> * ------------- THERMAL EMITTANCE MODEL
--------------------------------------------
OPAL{0}> * THERMAL EMITTANCE in ASTRA MODE
OPAL{0}> * Kinetic energy (thermal emittance) = 2.000000e-01 [eV]
OPAL{0}> *
----------------------------------------------------------------------------------
OPAL{0}> *
OPAL{0}> *
**********************************************************************************
OPAL{0}>
OPAL{0}> * ************* B E A M
************************************************************
OPAL{0}> * BEAM BEAM1
OPAL{0}> * PARTICLE ELECTRON
OPAL{0}> * CURRENT 8.568000e-01 A
OPAL{0}> * FREQUENCY 2.856000e+03 MHz
OPAL{0}> * CHARGE -e * 1.000000e+00
OPAL{0}> * REST MASS 5.109990e-04 GeV
OPAL{0}> * MOMENTUM 1.196160e-06
OPAL{0}> * NPART 5.000000e+04
OPAL{0}> *
**********************************************************************************

OPAL{0}>
OPAL{0}> * ************* F I E L D S O L V E R
**********************************************
OPAL{0}> * FIELDSOLVER FS_SC
OPAL{0}> * TYPE FFT
OPAL{0}> * N-PROCESSORS 16
OPAL{0}> * MX 3.200000e+01
OPAL{0}> * MY 3.200000e+01
OPAL{0}> * MT 3.200000e+01
OPAL{0}> * BBOXINCR 1.000000e+00
OPAL{0}> * GRRENSF INTEGRATED
OPAL{0}> * XDIM serial
OPAL{0}> * YDIM serial
OPAL{0}> * Z(T)DIM parallel
OPAL{0}>
OPAL{0}> *
**********************************************************************************

OPAL{0}>
Ippl{0}>
Ippl{0}> * ************* P C G
**************************************************************
Ippl{0}> PCG_phi = 298.0022 [deg], corresp. in Astra = 33.1686 [deg],
Ippl{0}> E = 5.2415 [MeV], phi_nom = -5.1665 [deg]
Ippl{0}> Ez_0 = 111.3150 [MV/m]
Ippl{0}> time = 0.0000 [ns], dt = 0.1000 [ps]
Ippl{0}> *
**********************************************************************************
Ippl{0}>
Ippl{0}>
Ippl{0}> * ************* L 1
****************************************************************
Ippl{0}> L1_phi = 330.5441 [deg], corresp. in Astra = 124.4550 [deg],
Ippl{0}> E = 39.8775 [MeV], phi_nom = -27.0743 [deg]
Ippl{0}> Ez_0 = 17.5940 [MV/m]
Ippl{0}> time = 6.2667 [ns], dt = 0.1000 [ps]
Ippl{0}> *
**********************************************************************************
Ippl{0}>
OPAL{0}>
OPAL{0}> Size of map 5 sections
OPAL{0}> Key: (0.000000 - 0.148488) number of overlapping elements 2
OPAL{0}> PCG
OPAL{0}> MS
OPAL{0}> Key: (0.148488 - 0.504249) number of overlapping elements 1
OPAL{0}> MS
OPAL{0}> Key: (0.504249 - 1.864823) number of overlapping elements 0
OPAL{0}> Key: (1.864823 - 4.872675) number of overlapping elements 1
OPAL{0}> L1
OPAL{0}> Key: (4.872675 - 5.500000) number of overlapping elements 0
OPAL{0}>
Ippl{0}> Creating new file for data: data/opal_0_ElementPositions.sdds
Ippl{0}> Saved phases in the h5 file: Cav-1-name -> PCG --- Cav-1-value ->
5.2011
Ippl{0}> Saved phases in the h5 file: Cav-2-name -> L1 --- Cav-2-value ->
0.1288
OPAL{0}>
OPAL{0}> Track start at: 19:19:01, t= -3.400 [ps]; zstart at: 0.000 [um]
ParallelTTracker {0}> Change dT from 1e-13 [s] to 6.86856e-14 [s] during
emission
OPAL{0}> Executing ParallelTTracker, initial dt= 100.000 [fs];
OPAL{0}> max integration steps 20000000, next step= 0
OPAL {0}>
OPAL {0}> * ************** B U N C H
*********************************************************
OPAL {0}> * NP = 50000
OPAL {0}> * Qtot = 300.000 [pC] Qi = 6.000
[fC]
OPAL {0}> * Ekin = 6.985 [keV] dEkin = 2.912
[keV]
OPAL {0}> * rmax = ( 5.87613 , 5.87733 , 0.35637 )
[mm]
OPAL {0}> * rmin = ( -5.87681 , -5.87444 , -0.06781 )
[mm]
OPAL {0}> * rms beam size = ( 2.95489 , 2.93497 , 0.05336 )
[mm]
OPAL {0}> * rms momenta = ( 2.25444e-03 , 2.24958e-03 , 3.48199e-02 )
[beta gamma]
OPAL {0}> * mean position = ( -1.34813 , -2.38681 , 58.36988 )
[um]
OPAL {0}> * mean momenta = ( -3.19928e-06 , -4.05636e-06 , 1.62272e-01 )
[beta gamma]
OPAL {0}> * rms emittance = ( 1.78879e-05 , 1.76971e-05 , 1.81071e-06 )
(not normalized)
OPAL {0}> * rms correlation = ( -8.95291e-01 , -8.95688e-01 , 9.86841e-01 )
OPAL {0}> * hr = ( 386.71192 , 386.67322 , 13.81769 )
[um]
OPAL {0}> * dh = 1.00000e+00 [%]
OPAL {0}> * t = 3.794 [ps] dT = 100.000
[fs]
OPAL {0}> * spos = 69.192 [um]
OPAL {0}> *
**********************************************************************************

...

OPAL {0}> * ************** B U N C H
*********************************************************
OPAL {0}> * NP = 50000
OPAL {0}> * Qtot = 300.000 [pC] Qi = 6.000
[fC]
OPAL {0}> * Ekin = 39.867 [MeV] dEkin = 322.523
[keV]
OPAL {0}> * rmax = ( 11.34304 , 11.77731 , 1.12542 )
[mm]
OPAL {0}> * rmin = ( -11.49049 , -11.48420 , -1.20980 )
[mm]
OPAL {0}> * rms beam size = ( 5.19789 , 5.22678 , 0.40761 )
[mm]
OPAL {0}> * rms momenta = ( 6.20707e-02 , 6.23494e-02 , 6.31211e-01 )
[beta gamma]
OPAL {0}> * mean position = ( -3.25190 , 2.83798 , -12.31333 )
[um]
OPAL {0}> * mean momenta = ( -3.20042e-05 , 3.79428e-05 , 7.90106e+01 )
[beta gamma]
OPAL {0}> * rms emittance = ( 2.36115e-07 , 2.37382e-07 , 2.65240e-07 )
(not normalized)
OPAL {0}> * rms correlation = ( 9.98327e-01 , 9.98342e-01 , -9.96677e-01 )
OPAL {0}> * hr = ( 751.29195 , 765.37389 , 76.83619 )
[um]
OPAL {0}> * dh = 1.00000e+00 [%]
OPAL {0}> * t = 18.399 [ns] dT = 500.000
[fs]
OPAL {0}> * spos = 5.500 [m]
OPAL {0}> *
**********************************************************************************

OPAL {0}>
OPAL{0}> done executing ParallelTTracker at 19:30:36
OPAL{0}> * End of input stream "opal_0.in".
Timings{0}> -----------------------------------------------------------------
Timings{0}> Timing results for 16 nodes:
Timings{0}> -----------------------------------------------------------------
Timings{0}> mainTimer........... Wall tot = 698.587, CPU tot = 672.33
Timings{0}>
Timings{0}> Binaryrepart........ Wall max = 0.0804251, CPU max = 0.13
Timings{0}> Wall avg = 0.0792938, CPU avg = 0.078125
Timings{0}> Wall min = 0.0762691, CPU min = 0.04
Timings{0}>
Timings{0}> Boundingbox......... Wall max = 21.9074, CPU max = 22.05
Timings{0}> Wall avg = 21.2645, CPU avg = 21.2381
Timings{0}> Wall min = 20.2972, CPU min = 19.86
Timings{0}>
Timings{0}> Boundingbox-bounds.. Wall max = 3.23519, CPU max = 3.3
Timings{0}> Wall avg = 2.29564, CPU avg = 2.33812
Timings{0}> Wall min = 1.43132, CPU min = 1.19
Timings{0}>
Timings{0}> Boundingbox-update.. Wall max = 13.0062, CPU max = 13.74
Timings{0}> Wall avg = 12.9693, CPU avg = 12.9756
Timings{0}> Wall min = 12.8177, CPU min = 12.44
Timings{0}>
Timings{0}> Compute Statistics.. Wall max = 26.1269, CPU max = 26.62
Timings{0}> Wall avg = 24.6571, CPU avg = 24.815
Timings{0}> Wall min = 22.6815, CPU min = 22.65
Timings{0}>
Timings{0}> ComputePotential.... Wall max = 221.744, CPU max = 222.57
Timings{0}> Wall avg = 219.925, CPU avg = 219.121
Timings{0}> Wall min = 215.744, CPU min = 214.95
Timings{0}>
Timings{0}> Create Distr........ Wall max = 0.0196037, CPU max = 0.03
Timings{0}> Wall avg = 0.01854, CPU avg = 0.019375
Timings{0}> Wall min = 0.0183704, CPU min = 0.01
Timings{0}>
Timings{0}> External field eval. Wall max = 83.0337, CPU max = 83.09
Timings{0}> Wall avg = 37.3202, CPU avg = 37.67
Timings{0}> Wall min = 2.13352, CPU min = 2.24
Timings{0}>
Timings{0}> Fast inside test.... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Histogram........... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Initialize geometry. Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Inside test......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Load Distr.......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Particle Inside..... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Ray tracing......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Secondary emission.. Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> SelfField total..... Wall max = 398.178, CPU max = 399.5
Timings{0}> Wall avg = 359.377, CPU avg = 358.998
Timings{0}> Wall min = 330.56, CPU min = 330.17
Timings{0}>
Timings{0}> SF: GreensFTotal.... Wall max = 140.186, CPU max = 141.31
Timings{0}> Wall avg = 139.146, CPU avg = 138.486
Timings{0}> Wall min = 137.841, CPU min = 135.62
Timings{0}>
Timings{0}> TIntegration1....... Wall max = 21.4125, CPU max = 21.77
Timings{0}> Wall avg = 20.1085, CPU avg = 19.935
Timings{0}> Wall min = 18.3236, CPU min = 18.12
Timings{0}>
Timings{0}> TIntegration2....... Wall max = 40.1375, CPU max = 40.82
Timings{0}> Wall avg = 38.7925, CPU avg = 38.8013
Timings{0}> Wall min = 37.9342, CPU min = 37.21
Timings{0}>
Timings{0}> WakeField........... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Write H5-File....... Wall max = 47.1853, CPU max = 21.09
Timings{0}> Wall avg = 47.1643, CPU avg = 18.93
Timings{0}> Wall min = 47.0338, CPU min = 15.95
Timings{0}>
Timings{0}> Write Stat.......... Wall max = 9.99641, CPU max = 3.93
Timings{0}> Wall avg = 2.74196, CPU avg = 2.36437
Timings{0}> Wall min = 2.17362, CPU min = 2.09
Timings{0}>
Timings{0}> -----------------------------------------------------------------
Ippl> CommMPI: Parent process waiting for children ...
Ippl> CommMPI: Child 8 ready.
Ippl> CommMPI: Child 10 ready.
Ippl> CommMPI: Child 13 ready.
Ippl> CommMPI: Child 14 ready.
Ippl> CommMPI: Child 15 ready.
Ippl> CommMPI: Started job 1 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 2 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 3 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 4 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 5 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 6 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 7 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 8 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 9 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 10 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 11 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 12 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 13 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 14 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Started job 15 on host `fnl12b.bivio.biz'.
Ippl> CommMPI: Child 1 ready.
Ippl> CommMPI: Child 2 ready.
Ippl> CommMPI: Child 3 ready.
Ippl> CommMPI: Child 4 ready.
Ippl> CommMPI: Child 5 ready.
Ippl> CommMPI: Child 6 ready.
Ippl> CommMPI: Child 7 ready.
Ippl> CommMPI: Child 9 ready.
Ippl> CommMPI: Child 11 ready.
Ippl> CommMPI: Child 12 ready.
Ippl> CommMPI: Initialization complete.
{0}> ____ _____ ___
{0}> / __ \| __ \ /\ | |
{0}> | | | | |__) / \ | |
{0}> | | | | ___/ /\ \ | |
{0}> | |__| | | / ____ \| |____
{0}> \____/|_| /_/ \_\______|
OPAL{0}>
OPAL{0}> This is OPAL (Object Oriented Parallel Accelerator Library) Version
2.4.0
OPAL{0}> git rev. unknown
OPAL{0}>
OPAL{0}>
OPAL{0}> (c) PSI, http://amas.web.psi.ch
OPAL{0}>
OPAL{0}>
OPAL{0}> The optimiser (former opt-Pilot) is integrated
OPAL{0}>
OPAL{0}> Please send cookies, goodies or other motivations (wine and beer ...
)
OPAL{0}> to the OPAL developers opal AT lists.psi.ch
OPAL{0}>
OPAL{0}> Time: 18:24:17 date: 11/01/2021
OPAL{0}>
OPAL{0}> Couldn't find startup file "/home/vagrant/init.opal".
OPAL{0}> Note: this is not mandatory for an OPAL simulation!
OPAL{0}>
OPAL{0}> * Reading input stream "opal_0.in".
OPAL{0}> *
**********************************************************************************

OPAL{0}> * Selected Tracking Method == PARALLEL-T, NEW TRACK
OPAL{0}> *
**********************************************************************************

OPAL{0}>
OPAL{0}> *
**********************************************************************************
OPAL{0}> * Write initial distribution to file "data/opal_0_GEN_DIST.dat"
OPAL{0}> *
**********************************************************************************
Ippl{0}> Creating new file for data: opal_0.stat
Ippl{0}> Creating new file for data: opal_0.lbal
OPAL{0}>
OPAL{0}> * ************* D I S T R I B U T I O N
********************************************
OPAL{0}> *
OPAL{0}> * Number of particles: 50000
OPAL{0}> *
OPAL{0}> * Distribution type: FLATTOP
OPAL{0}> *
OPAL{0}> * SIGMAX = 5.880351e-03 [m]
OPAL{0}> * SIGMAY = 5.880351e-03 [m]
OPAL{0}> * Sigma Time Rise = 1.204546e-12 [sec]
OPAL{0}> * TPULSEFWHM = 0.000000e+00 [sec]
OPAL{0}> * Sigma Time Fall = 1.204546e-12 [sec]
OPAL{0}> * Longitudinal cutoff = 4.000000e+00 [units of Sigma Time]
OPAL{0}> * Flat top modulation amplitude = 0.000000e+00 [Percent of
distribution amplitude]
OPAL{0}> * Flat top modulation periods = 0.000000e+00
OPAL{0}> *
OPAL{0}> * Number of energy bins = 9
OPAL{0}> * Distribution is emitted.
OPAL{0}> * Emission time = 6.799879e-12 [sec]
OPAL{0}> * Time per bin = 7.555421e-13 [sec]
OPAL{0}> * Delta t during emission = 6.868565e-14 [sec]
OPAL{0}> *
OPAL{0}> * ------------- THERMAL EMITTANCE MODEL
--------------------------------------------
OPAL{0}> * THERMAL EMITTANCE in ASTRA MODE
OPAL{0}> * Kinetic energy (thermal emittance) = 2.000000e-01 [eV]
OPAL{0}> *
----------------------------------------------------------------------------------
OPAL{0}> *
OPAL{0}> *
**********************************************************************************
OPAL{0}>
OPAL{0}> * ************* B E A M
************************************************************
OPAL{0}> * BEAM BEAM1
OPAL{0}> * PARTICLE ELECTRON
OPAL{0}> * CURRENT 8.568000e-01 A
OPAL{0}> * FREQUENCY 2.856000e+03 MHz
OPAL{0}> * CHARGE -e * 1.000000e+00
OPAL{0}> * REST MASS 5.109990e-04 GeV
OPAL{0}> * MOMENTUM 1.196160e-06
OPAL{0}> * NPART 5.000000e+04
OPAL{0}> *
**********************************************************************************

OPAL{0}>
OPAL{0}> * ************* F I E L D S O L V E R
**********************************************
OPAL{0}> * FIELDSOLVER FS_SC
OPAL{0}> * TYPE FFT
OPAL{0}> * N-PROCESSORS 16
OPAL{0}> * MX 3.200000e+01
OPAL{0}> * MY 3.200000e+01
OPAL{0}> * MT 3.200000e+01
OPAL{0}> * BBOXINCR 1.000000e+00
OPAL{0}> * GRRENSF INTEGRATED
OPAL{0}> * XDIM serial
OPAL{0}> * YDIM serial
OPAL{0}> * Z(T)DIM parallel
OPAL{0}>
OPAL{0}> *
**********************************************************************************

OPAL{0}>
Ippl{0}>
Ippl{0}> * ************* P C G
**************************************************************
Ippl{0}> PCG_phi = 298.0022 [deg], corresp. in Astra = 33.1686 [deg],
Ippl{0}> E = 5.2415 [MeV], phi_nom = -5.1665 [deg]
Ippl{0}> Ez_0 = 111.3150 [MV/m]
Ippl{0}> time = 0.0000 [ns], dt = 0.1000 [ps]
Ippl{0}> *
**********************************************************************************
Ippl{0}>
Ippl{0}>
Ippl{0}> * ************* L 1
****************************************************************
Ippl{0}> L1_phi = 330.4914 [deg], corresp. in Astra = 124.4023 [deg],
Ippl{0}> E = 39.8770 [MeV], phi_nom = -27.0743 [deg]
Ippl{0}> Ez_0 = 17.5940 [MV/m]
Ippl{0}> time = 6.2667 [ns], dt = 0.1000 [ps]
Ippl{0}> *
**********************************************************************************
Ippl{0}>
OPAL{0}>
OPAL{0}> Size of map 5 sections
OPAL{0}> Key: (0.000000 - 0.148488) number of overlapping elements 2
OPAL{0}> MS
OPAL{0}> PCG
OPAL{0}> Key: (0.148488 - 0.504249) number of overlapping elements 1
OPAL{0}> MS
OPAL{0}> Key: (0.504249 - 1.864823) number of overlapping elements 0
OPAL{0}> Key: (1.864823 - 4.872674) number of overlapping elements 1
OPAL{0}> L1
OPAL{0}> Key: (4.872674 - 5.500000) number of overlapping elements 0
OPAL{0}>
Ippl{0}> Creating new file for data: data/opal_0_ElementPositions.sdds
Ippl{0}> Saved phases in the h5 file: Cav-1-name -> PCG --- Cav-1-value ->
5.2011
Ippl{0}> Saved phases in the h5 file: Cav-2-name -> L1 --- Cav-2-value ->
0.1279
OPAL{0}>
OPAL{0}> Track start at: 18:24:31, t= -3.400 [ps]; zstart at: 0.000 [um]
ParallelTTracker {0}> Change dT from 1e-13 [s] to 6.86856e-14 [s] during
emission
OPAL{0}> Executing ParallelTTracker, initial dt= 100.000 [fs];
OPAL{0}> max integration steps 20000000, next step= 0
OPAL {0}>
OPAL {0}> * ************** B U N C H
*********************************************************
OPAL {0}> * NP = 50000
OPAL {0}> * Qtot = 300.000 [pC] Qi = 6.000
[fC]
OPAL {0}> * Ekin = 6.985 [keV] dEkin = 2.912
[keV]
OPAL {0}> * rmax = ( 5.87613 , 5.87733 , 0.35637 )
[mm]
OPAL {0}> * rmin = ( -5.87681 , -5.87444 , -0.06781 )
[mm]
OPAL {0}> * rms beam size = ( 2.95489 , 2.93497 , 0.05336 )
[mm]
OPAL {0}> * rms momenta = ( 2.25444e-03 , 2.24958e-03 , 3.48199e-02 )
[beta gamma]
OPAL {0}> * mean position = ( -1.34813 , -2.38681 , 58.36988 )
[um]
OPAL {0}> * mean momenta = ( -3.19928e-06 , -4.05636e-06 , 1.62272e-01 )
[beta gamma]
OPAL {0}> * rms emittance = ( 1.78879e-05 , 1.76971e-05 , 1.81071e-06 )
(not normalized)
OPAL {0}> * rms correlation = ( -8.95291e-01 , -8.95688e-01 , 9.86841e-01 )
OPAL {0}> * hr = ( 386.71192 , 386.67322 , 13.81769 )
[um]
OPAL {0}> * dh = 1.00000e+00 [%]
OPAL {0}> * t = 3.794 [ps] dT = 100.000
[fs]
OPAL {0}> * spos = 69.192 [um]
OPAL {0}> *
**********************************************************************************

...

OPAL {0}> * ************** B U N C H
*********************************************************
OPAL {0}> * NP = 50000
OPAL {0}> * Qtot = 300.000 [pC] Qi = 6.000
[fC]
OPAL {0}> * Ekin = 39.851 [MeV] dEkin = 323.256
[keV]
OPAL {0}> * rmax = ( 11.34901 , 11.78365 , 1.12527 )
[mm]
OPAL {0}> * rmin = ( -11.49655 , -11.49029 , -1.20967 )
[mm]
OPAL {0}> * rms beam size = ( 5.20104 , 5.22995 , 0.40758 )
[mm]
OPAL {0}> * rms momenta = ( 6.21123e-02 , 6.23914e-02 , 6.32646e-01 )
[beta gamma]
OPAL {0}> * mean position = ( -3.25786 , 2.83414 , -12.31562 )
[um]
OPAL {0}> * mean momenta = ( -3.21617e-05 , 3.77190e-05 , 7.89802e+01 )
[beta gamma]
OPAL {0}> * rms emittance = ( 2.36234e-07 , 2.37505e-07 , 2.65332e-07 )
(not normalized)
OPAL {0}> * rms correlation = ( 9.98331e-01 , 9.98346e-01 , -9.96692e-01 )
OPAL {0}> * hr = ( 751.68758 , 765.78317 , 76.82682 )
[um]
OPAL {0}> * dh = 1.00000e+00 [%]
OPAL {0}> * t = 18.399 [ns] dT = 500.000
[fs]
OPAL {0}> * spos = 5.500 [m]
OPAL {0}> *
**********************************************************************************

OPAL {0}>
OPAL{0}> done executing ParallelTTracker at 18:46:30
OPAL{0}> * End of input stream "opal_0.in".
Timings{0}> -----------------------------------------------------------------
Timings{0}> Timing results for 16 nodes:
Timings{0}> -----------------------------------------------------------------
Timings{0}> mainTimer........... Wall tot = 1333.67, CPU tot = 1304.62
Timings{0}>
Timings{0}> Binaryrepart........ Wall max = 0.0760612, CPU max = 0.12
Timings{0}> Wall avg = 0.0748878, CPU avg = 0.076875
Timings{0}> Wall min = 0.0710163, CPU min = 0.04
Timings{0}>
Timings{0}> Boundingbox......... Wall max = 21.1333, CPU max = 21.51
Timings{0}> Wall avg = 20.492, CPU avg = 20.3581
Timings{0}> Wall min = 19.5912, CPU min = 18.98
Timings{0}>
Timings{0}> Boundingbox-bounds.. Wall max = 3.31469, CPU max = 3.25
Timings{0}> Wall avg = 2.33188, CPU avg = 2.32125
Timings{0}> Wall min = 1.45284, CPU min = 1.43
Timings{0}>
Timings{0}> Boundingbox-update.. Wall max = 12.3236, CPU max = 12.66
Timings{0}> Wall avg = 12.2843, CPU avg = 12.1619
Timings{0}> Wall min = 12.2186, CPU min = 11.72
Timings{0}>
Timings{0}> Compute Statistics.. Wall max = 24.9249, CPU max = 25.18
Timings{0}> Wall avg = 23.5228, CPU avg = 23.6244
Timings{0}> Wall min = 21.6156, CPU min = 21.35
Timings{0}>
Timings{0}> ComputePotential.... Wall max = 212.61, CPU max = 213.74
Timings{0}> Wall avg = 211.029, CPU avg = 210.689
Timings{0}> Wall min = 207.543, CPU min = 207.01
Timings{0}>
Timings{0}> Create Distr........ Wall max = 0.0197351, CPU max = 0.03
Timings{0}> Wall avg = 0.0186866, CPU avg = 0.020625
Timings{0}> Wall min = 0.0185006, CPU min = 0.01
Timings{0}>
Timings{0}> External field eval. Wall max = 711.153, CPU max = 711.74
Timings{0}> Wall avg = 361.201, CPU avg = 361.191
Timings{0}> Wall min = 2.25931, CPU min = 2.35
Timings{0}>
Timings{0}> Fast inside test.... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Histogram........... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Initialize geometry. Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Inside test......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Load Distr.......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Particle Inside..... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Ray tracing......... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> SelfField total..... Wall max = 380.273, CPU max = 380.01
Timings{0}> Wall avg = 344.384, CPU avg = 344.401
Timings{0}> Wall min = 317.067, CPU min = 318.26
Timings{0}>
Timings{0}> SF: GreensFTotal.... Wall max = 134.448, CPU max = 134.95
Timings{0}> Wall avg = 133.307, CPU avg = 132.716
Timings{0}> Wall min = 131.767, CPU min = 130.46
Timings{0}>
Timings{0}> TIntegration1....... Wall max = 20.5137, CPU max = 20.53
Timings{0}> Wall avg = 19.1446, CPU avg = 19.1488
Timings{0}> Wall min = 17.3008, CPU min = 17.47
Timings{0}>
Timings{0}> TIntegration2....... Wall max = 38.3021, CPU max = 38.52
Timings{0}> Wall avg = 36.8959, CPU avg = 36.8775
Timings{0}> Wall min = 35.997, CPU min = 35.53
Timings{0}>
Timings{0}> WakeField........... Wall max = 0, CPU max = 0
Timings{0}> Wall avg = 0, CPU avg = 0
Timings{0}> Wall min = 0, CPU min = 0
Timings{0}>
Timings{0}> Write H5-File....... Wall max = 49.9556, CPU max = 21.73
Timings{0}> Wall avg = 49.9353, CPU avg = 20.2519
Timings{0}> Wall min = 49.8026, CPU min = 17.74
Timings{0}>
Timings{0}> Write Stat.......... Wall max = 9.01406, CPU max = 2.07
Timings{0}> Wall avg = 0.833675, CPU avg = 0.405
Timings{0}> Wall min = 0.281822, CPU min = 0.22
Timings{0}>
Timings{0}> -----------------------------------------------------------------

[Opal] Increase in External Field Evaluation Time, Chris Hall, 01/11/2021
- Re: [Opal] Increase in External Field Evaluation Time, Christof Metzger-Kraus, 01/12/2021
  - Re: [Opal] Increase in External Field Evaluation Time, Christof Metzger-Kraus, 01/16/2021
    - Re: [Opal] Increase in External Field Evaluation Time, Nicole R Neveu, 01/17/2021
      - Re: [Opal] Increase in External Field Evaluation Time, Christof Metzger-Kraus, 01/18/2021
        
        Re: [Opal] Increase in External Field Evaluation Time, Chris Hall, 01/18/2021
      - Re: [Opal] Increase in External Field Evaluation Time, Christof Metzger-Kraus, 01/25/2021