H5Part development and discussion

[John Biddiscombe <biddisco AT cscs.ch>]

Thanks Kurt and John for the explanations.

One further question....does H5Block have a concept of time? I would like to store a series of volumes as time steps. There are 800 time steps and the data is 600^3 (it's quite large). I could manage with one time step per file, but my preference is to lump it all together.

An Observation.

I see that you have made a vector field out of 3 components inside a group. This is a nice and simple way of handling things. However, for the H5Part data I am generating, I have done the following
Density
Pressure
Position_0
Position_1
Position_2
K
sigma
Velocity_0
Velocity_1
Velocity_2

and added a simple group name parsing routine which detects when N components with the same name exist and allows you to read them into a 3component array instead of 3 one component arrays.

The reason I did this was because of the possibility to read any scalar variable as X coordinate, Y or Z, and recombine arrays arbitrarily. The code parses the group names and by default reconstructs your arrays based on the naming scheme above - but if you request "Position_0 Pressure Velocity_2" as coordinates
then you can read these three as a single 'field' and use them as (say) the position coordinate (and so on with velocity etc)

I'm not saying that you can't do it by lumping them into the same group as you have done with the block data - but have you any comments on my approach? - I can seem myself having two grouping schemes if I use H5Block for volume data too?

JB



Kurt Stockinger wrote:

Hi Johns,

H5Block allows you to store 3D scalar and vector fields. Internally,
H5Block stores a 3D scalar field as a 3D dataset. A 3D vector field is
stored as three 3-D datasets. Andreas has recently sent me some H5Block
dataset. See an example of the HDF5 file structure below for the 3D
scalar field "potential" and the 3D vector field "Electrical Field".


   GROUP "Block" {
         GROUP "Potential" {
            DATASET "0" {
               DATATYPE  H5T_IEEE_F64LE
               DATASPACE  SIMPLE { ( 128, 16, 16 ) / ( 128, 16, 16 ) }
            }
         }
         GROUP "Electrical Field" {
            DATASET "0" {
               DATATYPE  H5T_IEEE_F64LE
               DATASPACE  SIMPLE { ( 128, 16, 16 ) / ( 128, 16, 16 ) }
            }
            DATASET "1" {
               DATATYPE  H5T_IEEE_F64LE
               DATASPACE  SIMPLE { ( 128, 16, 16 ) / ( 128, 16, 16 ) }
            }
            DATASET "2" {
               DATATYPE  H5T_IEEE_F64LE
               DATASPACE  SIMPLE { ( 128, 16, 16 ) / ( 128, 16, 16 ) }
            }


Unlike in H5Part, where the processor layout is automatically specified
based on the numberOfParticles, in H5Block you need to define the "field
layout", i.e. the partitioning strategy of your 3D dataset. In the
dataset below, the scalar field "Potential" has the dimensionality 16,
16, 128. A possibly partitioning strategy for two processors is:

if (processor == 0)
    H5BlockDefine3DFieldLayout(fh, 0, 15, 0, 15, 0, 63);
else
    H5BlockDefine3DFieldLayout(fh, 0, 15, 0, 15, 64, 127);

The data can then be read with:

H5Block3dReadScalarField ( f, "Potential", data );

H5Block3dRead3dVectorField (f, "ElectricalField", xval, yval, zval);
 
Kurt

John Shalf wrote:
  

But of course, you would be the first H5block "user".
It'll be interesting to see if it meets your needs. :-)

-john

On Jun 15, 2007, at 1:53 PM, John Shalf wrote:
    

I think H5Block provides the capability that you are looking for.  It
provides the ability to store the block-structured grids that are
associated with the PIC calculation together with the particle data.

The H5Block interface should automatically re-integrate the chunks
(the 64 volumes) into the single/un-domain-decomposed logical volume
used in the calculation.

-john

On Jun 15, 2007, at 9:03 AM, John Biddiscombe wrote:
      

I've been happily using H5Part particles for some time, but I need
to write a bunch of volume chunks into HDF as well.
I have (say) 64 volumes which abut each other - the points along the
edges are not duplicated (0->7, 8->15, 16->23, 24->31 etc etc). I
was about to start a little utility to dump these volume chunks into
hdf files and I thought. I wonder if this is what H5Block does?

What is H5Block ? I have not looked at it at all (yet).

thanks

JB

--John Biddiscombe,                            email:biddisco @ cscs.ch
http://www.cscs.ch/about/BJohn.php
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Cantonale, 6928 Manno, Switzerland      | Fax:  +41 (91) 610.82.82

_______________________________________________
H5Part mailing list
H5Part AT lists.psi.ch
https://lists.web.psi.ch/mailman/listinfo/h5part
        

_______________________________________________
H5Part mailing list
H5Part AT lists.psi.ch
https://lists.web.psi.ch/mailman/listinfo/h5part
      

_______________________________________________
H5Part mailing list
H5Part AT lists.psi.ch
https://lists.web.psi.ch/mailman/listinfo/h5part
    

-- 
John Biddiscombe,                            email:biddisco @ cscs.ch
http://www.cscs.ch/about/BJohn.php
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Cantonale, 6928 Manno, Switzerland      | Fax:  +41 (91) 610.82.82