Project priorities and roadmap

[martinvuyk]

Hi, this project looks really good.
This is a huge enterprise, what is the direction you want to go? I see a lot of native code for the Arrow spec but also some start of a C data interface. I've read the spec like 2 times and it still blows me away how huge it is. And seeing these kinda discourages me, given that there are much more popular languages there and the spec is still incomplete for many of them.

Given that the stated goal of Mojo is to become a superset of Python how do you see this repo evolving over time?

I've also tried reading some of the python pyarrow code and I can barely wrap my head around what is happening in the code. Is it pure C data interface? Do you want to go in that same direction or try and implement everything natively?

[sa-]

Hey @martinvuyk, thanks for the interest here!

The goal is to implement everything in Mojo and solve the interop story with pyarrow/c/cpp/rust via the c data interface. Although how much of that goal is actually realized depends on whether this project gets enough support either via funding or code contributions.

This project requires some very domain specific knowledge to contribute so finding people who are both able and motivated is a challenge.