[pytables-users] Roundtrip numpy record arrays to PyTables Table

[Matthew Rocklin]

I asked about roundtripping numpy datetimes to PyTables Table objects in
this stackoverflow post

http://stackoverflow.com/questions/25706423/store-and-extract-numpy-datetimes-in-pytables

Thought I'd cross-post here to get more traction. Please let me know if
this dual use of SO and mailing is or is not appreciated.

Best,
-Matt

[Matthew Rocklin]

Looks like Phil Cloud managed a solution on the SO post by explicitly
manipulating the numpy datetime64 into the spec that the PyTables docs give
to the TimeCol64 type. I guess my question evolves to the following:

Is PyTables intended to do this work for me? It seems strange to have a
metadata type that isn't enforced or supported by the library.

http://stackoverflow.com/questions/25706423/store-and-extract-numpy-datetimes-in-pytables/25706686#25706686

you can't store regular python datetimes - I am not sure why it does not
throw an error
you have to use np.datetime64 in the first place (in the rec array)
Yeah, datetimes are indeed a PITA to encode. I'm hoping that the PyTables
TimeCol64 has some sane defaults.
But really, I'm actually just very confused about the behavior I saw in
the SO post. What is going on?
I'm aware that datetimes are a messy topic. At least on paper PyTables
seems to have some approach (or so the TimeCol64 descriptor leads me to
believe.) What is the story here?

a nice goal but you almost always need meta data for datetimes to ensure
proper round tripping
eg time zone; frequency
Ideally the table would be self-describing. I want to be able to hand
the path / datapath to someone and know that they will interpret it
correctly. To that end the TimeFoo descriptors are really valuable. I
just want to know how to use them appropriately.

I would store your datetime64 as Int64 columns (I did this in pandas
originally because TimeCol64 was somewhat buggy in PyTables < 3.0.0)
I guess it should work though / not sure if their is any efficiency
difference or not
I asked about roundtripping numpy datetimes to PyTables Table objects in
this stackoverflow post
http://stackoverflow.com/questions/25706423/store-and-extract-numpy-datetimes-in-pytables
Thought I'd cross-post here to get more traction. Please let me know if
this dual use of SO and mailing is or is not appreciated.
Best,
-Matt
--
You received this message because you are subscribed to the Google
Groups "pytables-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.