Discussion:
Table slice and dice (by column and random rows)
(too old to reply)
Ken Walker
2018-03-22 21:24:22 UTC
Permalink
I'm looking for advice to slice and dice data from a table. I want to slice
the table based on both 1) a value in a column and 2) values in rows.

I am familiar with both .where() and .iterrows() operators to grab data
slices.
The column slices will be easy to get with .where('my_col_val=#')
The challenge is many single row 'slices'. I don't want contiguous row
data (say rows 100-200) - that would be easy with .iterrows(). Instead I
want 100 randomly spaced rows (say 11, 24, 109, 333, etc out of 100,000).
And, I need to do this 100s of times: 100+ slices w/ .where(), then search
and save the row data (same row IDs each time).

My approach is to slice based on column value, then iterate thru that
table, only keeping the rows of interest.
All those row searches sound expensive. Is there a better way to do this
from a performance standpoint?
I'm looking for suggestions.
Thanks!
--
You received this message because you are subscribed to the Google Groups "pytables-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytables-users+***@googlegroups.com.
To post to this group, send an email to pytables-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...