Microsoft Corp.-owned professional networking site LinkedIn is donating another project to the open-source community.
It said today that it’s handing over control of the Feathr feature store for machine learning projects to the Linux Foundation’s AI & Data Foundation.
Feathr is a tool that was developed by LinkedIn and first put into production back in 2017. It’s designed to make feature serving in machine learning easy, faster and more scalable, especially for real-time artificial intelligence applications. LinkedIn engineers Hangfei Lin and Jinghui Mo said in a blog post that the company’s AI teams use Feathr to store, transform, serve and share features with low latency and high throughput.
More specifically, Feathr serves as an abstraction layer between the raw data and machine learning models that helps to standardize and simplify feature definition, transformation, serving, storage and access within machine learning workflows and applications. Developers can then focus more on feature engineering, leaving Feathr to take care of data serialization formats by connecting to various databases. It also provides performance optimization and credential management capabilities.
Using Feathr, machine learning developers can define features just once and use them in multiple scenarios, such as model training and model serving. They can also connect to various offline data sources, such as data lakes and data warehouses, and transform the data within them into features for machine learning.
Since its launch, Feathr has come to power multiple AI applications at LinkedIn, where it’s used to manage thousands of features. According to Lin and Mo, it has helped teams to reduce the time it takes to add and experiment with new features from weeks to days, while performing up to 50% faster than the custom feature processing pipelines it replaced.
Feathr was first open-sourced under an Apache-2 license in April 2022. At the same time, the company announced native integration and support for Feathr on Microsoft Azure.
Lin and Mo said that since doing so, Feathr has achieved rapid adoption among the machine learning operations community and is now used at companies of various sizes, in multiple industries. What’s more, those people are not just using the software but also helping to contribute to its development. “It’s clear that many others experience the same pain points that Feathr aims to address,” Lin and Mo said.
By donating Feathr to the LF AI & Data, it’s hoped that Feathr will be able to grow and evolve faster, increasing its visibility, user base and contributor base. At the same time, the core Feathr development team expects to gain more opportunities to collaborate with other companies and projects that are using the software.
One target is to achieve richer support for online stores via an integration with Milvus and JanusGraph, for example. In addition, Feathr’s backers want to adopt the open data lineage standard from OpenLineage.
“We’re excited to welcome Feathr to LF AI & Data and for it to be part of our technical project portfolio with more than 16,000 developers,” said Dr. Ibrahim Haddad, executive director of LF AI & Data. “We aim to support Feathr to expand its user base, grow its community of developers, become a leader within its own category, and enable collaboration and integration opportunities with other projects.”
Image: LinkedIn
Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.
Source: https://siliconangle.com/2022/09/12/linkedins-open-source-feathr-feature-store-machine-learning-joins-lf-ai-data-foundation/
