h5part AT lists.psi.ch
Subject: H5Part development and discussion
List archive
- From: "Matthew Bettencourt" <mbetten AT sandia.gov>
- To: "Gsell Achim" <achim.gsell AT gmail.com>
- Cc: h5part AT lists.psi.ch
- Subject: Re: [EXTERNAL] Re: [H5part] H5PartSetCanonicalView
- Date: Wed, 31 Oct 2012 12:28:30 -0600
- List-archive: <https://lists.web.psi.ch/pipermail/h5part/>
- List-id: H5Part development and discussion <h5part.lists.psi.ch>
In the doc you say both start and end need to be -1,
"unset" by calling \c H5PartSetView(file,-1,-1);
What I want is to be able to have one set of code which works in parallel for both if we have more particles than processors and if we have less particles than processor. Right now I tried both the H5PartSetCanonicalView and setting by hand, it is impractical to set all the indices that I want to read and read them in for a very large run. Ideally one should have to have both start and end to -1, (which would most likely fix the other bug)
I have a work around to check to make sure that everyone has a valid range they expect and then revert to one processor only read, but it is ugly.
M
On 10/30/2012 01:36 PM, Gsell Achim wrote:
Hi Matthew
On Oct 26, 2012, at 12:32 AM, Matthew Bettencourt wrote:
Also, H5PartSetView doesn't work if one has less than 1 part/proc, example...This is actually an undocumented feature. H5PartSetView was not designed to select zero
my_pid is rank, and num_procs is 3. s = {0,0,0}, e={-1,-1,2} and nproc2
should be {0,0,2} but it is {2,2,2}
h5part_int64_t npart = H5PartGetNumParticles(file_); //<<<<<<<<this
returns 2
h5part_int64_t npart_per_proc = npart/num_procs;
h5part_int64_t npart_remainder = npart%num_procs;
h5part_int64_t s = npart_per_proc*my_pid;
h5part_int64_t e= npart_per_proc*(my_pid+1)-1;
if ( my_pid == num_procs-1 )
e += npart_remainder;
npart = e-s+1;
if (H5PartSetView(file_, s, e) != 0 )
error_out("Cannot set file view for h5part file\n");
h5part_int64_t npart2 = H5PartGetNumParticles(file_); //<<<<<<<<<<<<<
This returns 2
assert (npart == npart2);
particles. If you pass -1 as end, this will not be interpreted as "select zero
particles" but as "select all particles from the value given in start to the end.
So if you do something like
H5PartSetView (f, 0, -1);
you actually select all particles in the dataset. We will improve the
documentation in this point. A workaround is to use
H5PartSetViewIndices (f, NULL, 0);
if you want to select zero particles.
OK, we will check this.
-------- Original Message --------
Subject: H5PartSetCanonicalView
Date: Thu, 25 Oct 2012 16:22:00 -0600
From: Matthew Bettencourt <mbetten AT sandia.gov>
To: h5part AT lists.psi.ch
It appears that H5PartSetCanonicalView does not work correctly if you
have fewer particles that you have processors. I have 3 procs and 2
particles.
if (H5PartSetCanonicalView(file_) != 0 )
error_out("Cannot set file view for h5part file\n");
h5part_int64_t npart = H5PartGetNumParticles(file_);
returns an npart equal to 2 and doesnt trip the error.
Achim
- [H5part] H5PartSetCanonicalView, Matthew Bettencourt, 10/26/2012
- [H5part] Fwd: H5PartSetCanonicalView, Matthew Bettencourt, 10/26/2012
- Re: [H5part] H5PartSetCanonicalView, Gsell Achim, 10/30/2012
- Re: [EXTERNAL] Re: [H5part] H5PartSetCanonicalView, Matthew Bettencourt, 10/31/2012
- Re: [H5part] H5PartSetCanonicalView, Gsell Achim, 10/30/2012
- [H5part] Fwd: H5PartSetCanonicalView, Matthew Bettencourt, 10/26/2012
Archive powered by MHonArc 2.6.19.