H5Part development and discussion

[Wes Bethel <ewbethel AT lbl.gov>]

Gang,

Here's the notes I took during our confc call earlier today.

Current Status
- particles & structured fields
- reader, weblink to H5root
- attributes and statistical information

Next Steps
- Performance Issue. Achim found and fixed a performance problem reading
attributes from 80K timesteps. In H5root, took 20 minutes to read
attributes. Problem was/is reading attributes. The solution is to read
attributes by index as well as name. When reading by name, have to do
lots of searching. Cristina believes there is something like this in the
API already.

- Unstructured grids (H5topo). PSI will use for FEM projects. PSI will
have 2 or 3 PhD students on this work. Would like H5part field as input.
PSI feels having common data format central for several projects. Wes to
send link to ITAPS stuff to H5part team. The only link I can find right
now is http://www.scidac.gov/math/ITAPS.html.
        
Cristina asks for "live data" examples of H5block. Andreas is using a
modified version of Impact code, but not "instrumented" yet to dump
field data. Andreas says he'll send sample data in a week or so.
Cristina asks for copy of Impact code modified to dump fields using
H5part so we have examples.

Latest release 1.3.1., 1.3.2 has a few bug fixes. Achim will publish a
"new release", we'll download and update the GForge server at LBNL.

Kurt: looking through example programs to gain understanding. Integrate
FastBit indexing with H5part. Working directly with users (Rob, Ji).

No H5block documentation (yet). Need to nail down Doxygen "procedures"
and "standards" so we can (1) keep docs up to date and (2) make it easy
for folks to add documentation.

H5block currently supports 1 or 3 dimensions. Kurt asks about having
more dimensions. Achim says implementing additional dimensions would be
straightforward. The H5part API encapsulates the HDF5 hyperslabbing
capability.

Performance. Andreas has a spreadsheet containing performance numbers.
Three types of performance (didn't catch the name of each of them).
Current benchmark does only particles, not fields. Shalf wrote initial
version of this benchmark. LBL should pick up this part of the project.

Suggest: to maintain performance numbers across platforms and releases
so that (1) we have ready-at-hand performance numbers (2) we have a
baseline for comparison when we add new features.

The python interface never really worked in earlier version. What will
its fate be in later versions?

Cristina to work some more on making the VisIt reader "nicer." E.g.,
easier to do some introspection and then do selection for
reading/vis/etc. The problem is compilation. E.g., Mac. PSI wants to use
the parallel version (VisIt reader). Ji wants to use a Mac (which is a
problem). PSI to use VisIt on their new small vis cluster. VisIt plugin
code is in repository. The reader should be OK, not so sure about binner
plugin.

Open Issue: communication about checking out/checking in changes. In
general, we all need to be verbose about our intent to avoid "stepping
on each other's toes."

Open Issue: we need to have a set of regression and performance tests
that need to be run before doing a release. Presently, there is a "test
fortran", "test C", "test parallel", "test serial". Are there any tests
for H5block? attributes? LBL will take it as  a to-do item to do an
inventory of where we are with this stuff, and make a list of missing
bits. The overall goal is to have good code for releases. :-)

ToDo: test autogen/configure to make sure H5part compiles on all LBL and
NERSC machines.

Kurt: benchmarks, docs, FastBit, schedule H5block API design review with
Shalf to look at current API and consider future features.
Cristina: reader, VisIt compilation, check that H5part and H5block
readers work with latest version of VisIt. Will check whether current
H5part autogen/configure works on bassi, jacquard and davinci.
Andreas: sample data and code.