DataFrame library Polars announced a successful round of seed funding worth US$4 million, led by Bain Capital Ventures (BCV) with participation from individual investors. The step to develop a company around Polars comes around 3 years after our former Xomnia colleague Ritchie Vink started his quest to create the DataFrame library.
Ritchie started Polars as a “pet project” in 2020 in the midst of the Covid-19 pandemic. He was triggered by the lack of mature DataFrame libraries in the Rust programming language. The outcome is one of the fastest DataFrame libraries, which can be used in Rust, Python, R, and NodeJS.
Polars quickly attracted the attention of data scientists, engineers and thought leaders in the data industry. According to the DataFrame library’s founders, Polars has so far been downloaded 6 millions times, drawing close to 20 thousand stars in Github. Moreover, Polars query engine is now used in nushell and explorer.
Due to its success, Polars announced in 2023 that it has become an incorporated company, adding Xomnia’s former CTO Chiel Peters to its team as a co-founder. Together, Ritchie and Chiel want to utilize the funds to enable data processing at scale with Polars, and are looking to grow their team with talented Rust developers.
What makes the Polars DataFrame Library unique?
“Polars is a multi-threaded DataFrame library, meaning that it allows using all the cores of a computer at the same time to achieve its full processing potential,” explains Ritchie. “Even though there are libraries in Python that allow concurrency with multi-processing, they require copying data between processes, which makes them inefficient.”
This makes Polars unique compared to some of the most widely-used DataFrame libraries in Python, such as Pandas, which uses only one CPU core, while other cores remain idle. Ritchie thoroughly explained the technical novelties of Polars, going over its memory model, which is based on Apache Arrow’s memory model.
“It might not seem like much to speed up something that is already in the millisecond realm, but we are very excited about the possibilities brought by the development of Polars,” comments Tim Paauw, CTO at AI consultancy Xomnia, which is proudly sponsoring Polars.
Polars can speed up any application that uses DataFrames, like API's or ETL procedures, but also other analysis applications or decision support systems.
“Speed and memory efficiency matter when you consider that these DataFrame techniques are very widely used. It enables processing of larger datasets without the need for all kinds of extra infrastructure. Such improvements have a big impact on project agility and sustainable computing, among others,” adds Tim.
“Xomnia firmly believes in the application of open source software; we have been using that to help many of our clients over the years,” commented Xomnia’s co-founder Ollie Dapper in 2021. “By powering the development of Polars, we aim to give back to the open source community and support the passions of our colleagues and alumni, which is something that we highly value at Xomnia.”
***************
As sharing knowledge is Xomnia’s #1 value, we believe in giving back to the open source community. We are proud to do this by fully supporting Polars co-founders Ritchie Vink and Chiel Peters as they advance the DataFrame library. Their contribution to the open source community delivered one of the fastest DataFrame libraries in Rust, Python and NodeJS. We are happy to see that our sponsorship allows Polars to grow, get more stable, add new features, and secure wider community adoption.