A team of original Apache Flink® contributors founded data Artisans in 2014 because we believed that existing data processing frameworks weren’t adequately addressing the needs of organizations and their engineering teams. From the global saturation of smartphones, to the rapid adoption of the Internet of Things and connected devices, the very nature of data and how it is generated had evolved far more quickly than the tools available to manage that data.
When Stephan Ewen, data Artisans’ CTO, and the rest of the initial Flink team wrote the first lines of what would eventually become Flink in 2010 as researchers at TU Berlin, we had set out to find a conceptually simple solution to a difficult problem: to minimize the time span between data creation and when insights could be generated by that data and to do so in a way that is computationally efficient and easy to use. We never imagined that in a few short years, there would be a brilliant and thriving open source Flink community and that companies like Alibaba, King, and many more would be using Flink to power real-time streaming applications in production at such enormous scale. The latest Flink Forward conference was indeed a testament to the maturity and scale of Flink deployments in some of the largest data infrastructures of the world.
After many discussions with users and prospective users, we decided it is time for us to take a step forward in our ability to support large-scale deployments inside of companies. And so today, we’re announcing the dA Platform: a distribution of Apache Flink, quality assured and supported by data Artisans’ team of Flink committers and experts. Platform customers will have access to 24/7/365 enterprise support and consulting services by data Artisans to ensure that applications powered by Flink run flawlessly and allow organizations to achieve business objectives without disruption.
At data Artisans, we remain wholly committed to the evolution of Apache Flink and we’ll continue to support and contribute to the open source community in the same way and capacity that we have done so far. We see Flink and the dA platform as complementary projects. New features that we develop that are a natural fit for Flink will be proposed to the Flink community. Features that go beyond the scope of a stream processing engine but fall within the scope of the platform will be included in the platform instead. This means Flink remains focused on what it does best: providing the APIs, execution, connectors, libraries, and fault tolerance for large-scale streaming jobs. Enterprises who are looking for additional resources have access to the enterprise-grade support provided by data Artisans.
This is the virtuous cycle that dA Platform’s launch enables: Flink will benefit from the adoption of the dA Platform as the framework is used by more companies, and the platform will benefit from ongoing development in Flink. We look forward to collaborating with enterprises who use our platform to develop improvements that can be proposed to the community for inclusion in future versions of Flink.
We are grateful for all of the support we’ve received from users and the big data community during our first 2 years, and we look forward to developing the future of streaming applications with you and the rest of the Apache Flink® community.