H5Part development and discussion

[John Shalf <jshalf AT lbl.gov>]

On Apr 19, 2007, at 10:45 AM, Achim Gsell wrote:
> On Thursday 19 April 2007 17:53, John Biddiscombe wrote:
> > 2) Added option to use INDEPENDENT_IO instead of
> > COLLECTIVE_IO, I need this when using it on a system with a
> > non-parallel file-system, using collective IO causes too large
> > performance drop on our cluster.
>
> INDEPENDENT_IO vs COLLECTIVE_IO in H5Part, good point. A couple
> of weeks ago a college found a bug in H5Part concerning
> COLLECTIVE_IO (or we understand something completely wrong): The
> H5FD_MPIO_COLLECTIVE property is added to the property list
> f->xfer_prop, but this property list is not used in I/O calls.
> Thus we are using independent I/O. It's the same in version 1.0!
> John (Shalf), any idea/comments about this?

Either it is a mistake or a misunderstanding about the operation of  
parallelIO with HDF5.  I was under the impression that if you open a  
file with the COLLECTIVE_IO property, that the implicit mode of  
operation for the files is collective unless you tell it otherwise.   
If this is not the case, then we should make sure the collective  
property is added to all of the xfer lists (but I assume it is  
implicit if the file was opened in collective mode).

Lets do some regression testing with the benchmark (bench.c) to see  
if there is in fact a difference or not.

> > 6) Added a function to query IsTimeStep Present, so
> > that I don't overwrite an existing one if redoing the same
> > time
>
> For me it is OK to add such a function, but SetStep already
> returns an error, if the step exists.

Yes, that was the reason we didn't add that function.

> > If I update my copy to work with your 1.3.2 (svn head), can I
> > send you a large patch with all of it in, or would you prefer
> > smaller patches which address individual issues (much more
> > work)
>
> Since I want to review the patches, I prefer a set of smaller
> patches.
>
> Achim
> _______________________________________________
> H5Part mailing list
> H5Part AT lists.psi.ch
> https://lists.web.psi.ch/mailman/listinfo/h5part