[pytables-users] Best practices and code snippet for nested arrays

[JC]

Hello, I am a bit new to python programming and have been beating my head
against the wall for day trying to figure out a problem. I finally decided
it was time to ask for help. I wanted to ask what the best-practices are
for storing multiple (30k) mixed data type python dicts. My main cncern is
that the keys need to be tied to the corresponding data.
Here's an example:

entry1 = { 'key1' : 3CharString,
'key2' : 6CharString,
'key3' : float32,
'key4' : string OR date,
'data1' : variable length list (1d) of variable length
strings}


entry2 = { 'key5' : 3CharString,
'key6' : 6CharString,
'key7' : float32,
'key8' : string OR date,
'data2' : variable length list (1d) of variable length
strings}

and so on...

My original approach was to try and load this in a table for quick sorting
and utilization of the keys and to nest a list into a single columns for
the data. I do not any indexing capability of the data, just a way to store
it so I can quickly and easily get it back into pandas Series or numpy
array. It turns out, if this can be done I could not find an example to
guide my code. Furthermore it looks like nested variable length arrays of
variable length strings into table is not currently supported by pytables.

This also makes me wonder if there is a better way to do this in pytables.
I thought about putting the keys as attributes, but worried I would lose
the quick pytables filtering and indexing that I want.

Could you advise me the best way to handle this problem that address my
concerns about tying the keys to the data and quick easy access to data
using the keys as filters or indexes? Could you also help me with a quick
code snippet for the best way to proceed?

Thank you for your time and help.