H5Part development and discussion

[Mohamad Chaarawi <chaarawi AT hdfgroup.org>]

Hi Quincey,

On 05/09/2012 09:34 AM, Quincey Koziol wrote:
> Hi Mohamad,
>
> On May 9, 2012, at 9:26 AM, Mohamad Chaarawi wrote:
>
> > Hi All,
> >
> > > I'm guessing that H5MM_xstrdup is returning a malloc'd string to H5P_remove 
> > > that isn't being free'd.
> > The string is free'd again in H5P_close, so I don't see why a memory leak 
> > would be possible there.
>         Maybe the property list is internal to the h5part code and isn't 
> being freed there?

Ah nevermind.. I see the problem now... It's an HDF5 internal issue on 
how properties are inserted/removed when doing collective I/O. I'll talk 
to Quincey off list and get a fix.

Thanks,
Mohamad

>         Mark, is that possible?
>
> > Is there a short version of the code that I could use to work with and 
> > replicate the problem, because just looking at the internal HDF5 library 
> > doesn't get me anywhere :-)
> >
> > Quincey,
> > Could this be related to the memory leak discovered in the derived datatype 
> > construction which you fixed and checked in the trunk and the 1_8_9 release 
> > branch?
>         Hmm, probably not...
>
>         Quincey
>
> > Thanks,
> > Mohamad
> >
> > > Thanks, Mark On May 2, 2012, at 7:47 AM, Stefan Adami wrote:
> > > > > Dear H5Part-Developers,
> > > > >
> > > > > I am using H5Part for my SPH simulation output and found recently a
> > > > > problem when writing in parallel. To be more precise, I get a memory
> > > > > leak when writing things in parallel. I checked the test-code you
> > > > > provide in the test-folder and when adding the preproc-flag
> > > > > -DPARALLEL_IO to the Makefile (by default --enable-parallel does not set
> > > > > this correctly, the testf.f90 is actually compiled in serial version) I
> > > > > can reproduce this memory leak:
> > > > >
> > > > > 23 bytes in 1 blocks are definitely lost in loss record 13 of 20
> > > > > ==9990==    at 0x4024F20: malloc (vg_replace_malloc.c:236)
> > > > > ==9990==    by 0x811770C: H5MM_xstrdup (H5MM.c:170)
> > > > > ==9990==    by 0x8153A47: H5P_remove (H5Pint.c:3932)
> > > > > ==9990==    by 0x80BB1E3: H5FD_mpi_teardown_collective (H5FDmpi.c:525)
> > > > > ==9990==    by 0x808F664: H5D_inter_collective_io (H5Dmpio.c:1511)
> > > > > ==9990==    by 0x808DBA6: H5D_contig_collective_write (H5Dmpio.c:532)
> > > > > ==9990==    by 0x808C06E: H5Dwrite (H5Dio.c:265)
> > > > > ==9990==    by 0x804EF03: H5PartWriteDataFloat64 (H5Part.c:1134)
> > > > > ==9990==    by 0x804C9F8: h5pt_writedata_r8_ (H5PartF.c:501)
> > > > > ==9990==    by 0x804C196: MAIN__ (testf.F90:63)
> > > > > ==9990==    by 0x804B563: main
> > > > > (in /scratch/adami/source/ppm/work/H5Part-1.6.6/test/test
> > > > >
> > > > > ==9990== LEAK SUMMARY:
> > > > > ==9990==    definitely lost: 270 bytes in 12 blocks
> > > > > ==9990==    indirectly lost: 0 bytes in 0 blocks
> > > > > ==9990==      possibly lost: 0 bytes in 0 blocks
> > > > > ==9990==    still reachable: 5,564 bytes in 8 blocks
> > > > > ==9990==         suppressed: 0 bytes in 0 blocks
> > > > > ==9990==
> > > > > ==9990== For counts of detected and suppressed errors, rerun with: -v
> > > > > ==9990== Use --track-origins=yes to see where uninitialised values come
> > > > > from
> > > > > ==9990== ERROR SUMMARY: 1060 errors from 191 contexts (suppressed: 29
> > > > > from 10)
> > > > >
> > > > > Is there a special compiler-adjustment required or anything else the
> > > > > user should take care of? I tested this test-code on two different
> > > > > machines with a 32 and 64bit system with the same result...
> > > > > Do you have any ideas or are you aware of this leak?
> > > > >
> > > > > Thank you very much for any help,
> > > > >
> > > > > Stefan
> > > > >
> > > > > --
> > > > > ________________________________________
> > > > > Dipl.-Ing. Stefan Adami
> > > > > Lehrstuhl für Aerodynamik und Strömungsmechanik
> > > > > Technische Universität München
> > > > > Boltzmannstr. 15, 85748 Garching
> > > > > Tel.: +49-89-289-16122
> > > > > Fax.: +49-89-289-16139
> > > > > http://www.aer.mw.tum.de
> > > > >
> > > > > _______________________________________________
> > > > > H5Part mailing list
> > > > > H5Part AT lists.psi.ch
> > > > > https://lists.web.psi.ch/mailman/listinfo/h5part