Skip to Content.
Sympa Menu

h5part - Re: [EXTERNAL] Re: [H5part] H5PartSetCanonicalView

h5part AT lists.psi.ch

Subject: H5Part development and discussion

List archive

Re: [EXTERNAL] Re: [H5part] H5PartSetCanonicalView


Chronological Thread 
  • From: "Matthew Bettencourt" <mbetten AT sandia.gov>
  • To: "Gsell Achim" <achim.gsell AT gmail.com>
  • Cc: h5part AT lists.psi.ch
  • Subject: Re: [EXTERNAL] Re: [H5part] H5PartSetCanonicalView
  • Date: Wed, 31 Oct 2012 12:28:30 -0600
  • List-archive: <https://lists.web.psi.ch/pipermail/h5part/>
  • List-id: H5Part development and discussion <h5part.lists.psi.ch>

In the doc you say both start and end need to be -1,

"unset" by calling \c H5PartSetView(file,-1,-1);

What I want is to be able to have one set of code which works in parallel for both if we have more particles than processors and if we have less particles than processor. Right now I tried both the H5PartSetCanonicalView and setting by hand, it is impractical to set all the indices that I want to read and read them in for a very large run. Ideally one should have to have both start and end to -1, (which would most likely fix the other bug)

I have a work around to check to make sure that everyone has a valid range they expect and then revert to one processor only read, but it is ugly.

M



On 10/30/2012 01:36 PM, Gsell Achim wrote:
Hi Matthew

On Oct 26, 2012, at 12:32 AM, Matthew Bettencourt wrote:

Also, H5PartSetView doesn't work if one has less than 1 part/proc, example...

my_pid is rank, and num_procs is 3. s = {0,0,0}, e={-1,-1,2} and nproc2
should be {0,0,2} but it is {2,2,2}


h5part_int64_t npart = H5PartGetNumParticles(file_); //<<<<<<<<this
returns 2

h5part_int64_t npart_per_proc = npart/num_procs;
h5part_int64_t npart_remainder = npart%num_procs;


h5part_int64_t s = npart_per_proc*my_pid;
h5part_int64_t e= npart_per_proc*(my_pid+1)-1;
if ( my_pid == num_procs-1 )
e += npart_remainder;


npart = e-s+1;
if (H5PartSetView(file_, s, e) != 0 )
error_out("Cannot set file view for h5part file\n");

h5part_int64_t npart2 = H5PartGetNumParticles(file_); //<<<<<<<<<<<<<
This returns 2

assert (npart == npart2);
This is actually an undocumented feature. H5PartSetView was not designed to select zero
particles. If you pass -1 as end, this will not be interpreted as "select zero
particles" but as "select all particles from the value given in start to the end.
So if you do something like

H5PartSetView (f, 0, -1);

you actually select all particles in the dataset. We will improve the
documentation in this point. A workaround is to use

H5PartSetViewIndices (f, NULL, 0);

if you want to select zero particles.


-------- Original Message --------
Subject: H5PartSetCanonicalView
Date: Thu, 25 Oct 2012 16:22:00 -0600
From: Matthew Bettencourt <mbetten AT sandia.gov>
To: h5part AT lists.psi.ch

It appears that H5PartSetCanonicalView does not work correctly if you
have fewer particles that you have processors. I have 3 procs and 2
particles.

if (H5PartSetCanonicalView(file_) != 0 )
error_out("Cannot set file view for h5part file\n");
h5part_int64_t npart = H5PartGetNumParticles(file_);

returns an npart equal to 2 and doesnt trip the error.
OK, we will check this.


Achim






Archive powered by MHonArc 2.6.19.

Top of Page