[pytables-users] Proposal for HDF5 and Python ecosystem

Francesc Alted

2015-07-10 14:14:55 UTC

Hi Anthony and others,

Looks like you had a productive meeting indeed :) I basically agree on
your decisions (PyTables 4.0 on top of h5py and a long term maintainer for
PyTables). As a freelancer, I might even have time to put on the 4.0
effort (starting on October). And glad to see that this effort can be
sponsored, as this facilitates things a lot.

Anyway, thanks everybody to make this moving forward and ensure the
continuity of the PyTables community!

Francesc

Post by Anthony Scopatz
Hello All,
Yesterday at SciPy 2015, a few of us representing HDF5 (Mike, John), h5py
(Andrew), pytables (Andrea, myself) had a productive meeting with some of
our users about the future development and maintenance plan for PyTables,
h5py, etc.
--------------
From the folks present (at least), there was broad consensus that the a
more maintainable, more consistent future would have PyTables as an
interface built on top of h5py, plus potentially some direct C-API calls
for things like indexing and querying. This will also take some effort on
the h5py side of things to support some of the current PyTables data
types.
The projects would remain separate, but would gain 100% compatibility and
greatly reduce the amount of low-level binding maintenance that is
currently being done twice. This would probably entail a PyTables v4.0,
but would not break the current PyTables API.
Who does the work?
------------------------------
As with many of these things, we are stretched thin already - which is
part of the whole reason for wanting to move to a more maintainable
future. From our discussion yesterday, we envisioned two roles and two
* Refactor work, short term intensive effort
* Maintainer (releases, CI, mailing list), long term quarter person.
As per the short term intensive effort, I am going to talk to Leah Silen
of Numfocus about where we can get some money to fund this. This is
advantageous because they are able to hire contractors, take larger sums of
money, and coordinate internationally. I have some potential funding
sources in mind, that you can ping me about off-list. Francesc, Antonio,
and Andrea we should talk about disburse this going forward.
For the maintainer position, at the University of SC, I am planning on
putting together a staff person for this kind of work in my research group.
(If you know anyone who might be interested, pleas let me know!) Here is
the job ad: http://shuk-sc.github.io/jobs/0004.html HDF Group has a call
for proposals that we can submit to compete for this money.
Next Steps
---------------------
First off, please comment in this thread about your thoughts and opinions
on this plan.
Secondly, I am planning on leading the quest get funds and support for
these things, but don't hesitate to help :)
Also, David Dotson (who has some really neat research relying on h5py,
pandas, and pytables) has volunteered to write a blog post for HDF Group
about his work.
As part of trying to lead this forward, I have also volunteered to write a
blog post for HDF Group about the proposal here.
Lastly, I think we should have roughly monthly calls so that we don't slip
too far behind on this stuff. In a few weeks I'll send out another call
email.
Thanks everybody! I am super happy to see this move forward.
Be Well
Anthony
--
You received this message because you are subscribed to the Google Groups
"pytables-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
Francesc Alted
--
You received this message because you are subscribed to the Google Groups "pytables-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pytables-users+***@googlegroups.com.
To post to this group, send an email to pytables-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.