[pytables-users] segmentation fault by reindex table column of big data

[s***@googlemail.com]

Hi all,

I imported about 1-2 million of rows to my biggest of 5 tables of pytables
(3.2). The other tables have some 100thousands rows.
If I import without any indexes I can import about 4 Mio or more.
Everything works fine.

With indexing the first imports do all go quite well including indexing one
column of each table. Anythings perfect.
Each import does have nearly the same size of 300.000.

After about that 5th, 6th or 8th import I'll get that size of about 1-2
million entries in this biggest table and I get a segmentation fault while
trying to reindex() or reindex_dirty(). That worked the times before also
quite well. It sometimes succeeds with more sometimes with less imports
until seg fault.
Reindex_dirty causes even faster after 3 times seg fault.

I also tried to flush_rows_to_index() but I am not sure if that is enough
to create a correct index?

Is there any chance to get a correct index on even much bigger tables (need
to import 100th of these)?
Reindexing directly after flushing increases the time enormously to much
(except flush_rows_to_index) which caused me to close the file and reopend
it again. Reindexing the first times worked quite fast - about a second.

Maybe there is any chance to create the index of a column directly by hdf5?
or by pretrack? But I did not find it.
I am quite new to all this.

I did not find any similar case concerning this issue.
Thanks very very much for your help!!
Isa