H5Part development and discussion

["Matthew Bettencourt" <mbetten AT sandia.gov>]

In the doc you say both start and end need to be -1,

"unset" by calling \c H5PartSetView(file,-1,-1);

What I want is to be able to have one set of code which works in 
parallel for both if we have more particles than processors and if we 
have less particles than processor.  Right now I tried both the 
H5PartSetCanonicalView and setting by hand, it is impractical to set all 
the indices that I want to read and read them in for a very large run.  
Ideally one should have to have both start and end to -1, (which would 
most likely fix the other bug)

I have a work around to check to make sure that everyone has a valid 
range they expect and then revert to one processor only read, but it is 
ugly.

M



On 10/30/2012 01:36 PM, Gsell Achim wrote:
> Hi Matthew
>
> On Oct 26, 2012, at 12:32 AM, Matthew Bettencourt wrote:
>
> > Also, H5PartSetView doesn't work if one has less than 1 part/proc, example...
> >
> > my_pid is rank, and num_procs is 3.  s = {0,0,0}, e={-1,-1,2} and nproc2 
> > should be {0,0,2} but it is {2,2,2}
> >
> >
> >    h5part_int64_t npart = H5PartGetNumParticles(file_);  //<<<<<<<<this 
> > returns 2
> >
> >    h5part_int64_t npart_per_proc = npart/num_procs;
> >    h5part_int64_t npart_remainder = npart%num_procs;
> >
> >
> >    h5part_int64_t s = npart_per_proc*my_pid;
> >    h5part_int64_t e= npart_per_proc*(my_pid+1)-1;
> >    if ( my_pid == num_procs-1 )
> >      e += npart_remainder;
> >
> >
> >    npart = e-s+1;
> >    if (H5PartSetView(file_, s, e) != 0 )
> >      error_out("Cannot set file view for h5part file\n");
> >
> >    h5part_int64_t npart2 =  H5PartGetNumParticles(file_);  //<<<<<<<<<<<<<  
> > This returns 2
> >
> >    assert (npart == npart2);
> This is actually an undocumented feature. H5PartSetView was not designed to select zero 
> particles. If you pass -1 as end, this will not be interpreted as "select zero 
> particles" but as "select all particles from the value given in start to the end. 
> So if you do something like
>
> H5PartSetView (f, 0, -1);
>
> you actually select all particles in the dataset. We will improve the 
> documentation in this point. A workaround is to use
>
> H5PartSetViewIndices (f, NULL, 0);
>
> if you want to select zero particles.
>
> > -------- Original Message --------
> > Subject:        H5PartSetCanonicalView
> > Date:   Thu, 25 Oct 2012 16:22:00 -0600
> > From:   Matthew Bettencourt <mbetten AT sandia.gov>
> > To:     h5part AT lists.psi.ch
> >
> > It appears that H5PartSetCanonicalView does not work correctly if you
> > have fewer particles that you have processors.  I have 3 procs and 2
> > particles.
> >
> >    if (H5PartSetCanonicalView(file_) != 0 )
> >      error_out("Cannot set file view for h5part file\n");
> >    h5part_int64_t npart = H5PartGetNumParticles(file_);
> >
> > returns an npart equal to 2 and doesnt trip the error.
> OK, we will check this.
>
>
> Achim