Empower real-time analytics on Apache Flink
The dbt-Flink-adapter enables running SQL-defined pipelines from a dbt project on Apache Flink. This enables all analytics engineers who know dbt to define real-time pipelines. At the same time, all users of Flink SQL get a very powerful and popular tool to maintain their pipeline projects - dbt.
This adapter also supports the typical batch processing pipelines on Flink. Thus, leveraging the broad library of connectors, both as sources and destinations (Elasticsearch, Kafka, DynamoDB, Kinesis, JDBC and others) makes this a very powerful system for all ETL (Extract, Transform, and Load) operations, driven by dbt.
Why Choose dbt-Flink-adapter?
- Leverages dbt as a powerful tool for maintaining Flink pipeline projects.
- Enables analytics engineers to take ownership of ETL processes using Flink, and create, manage, deploy, and maintain real-time analytical pipelines in a familiar environment.
- Apache Flink provides a powerful big data processing framework for both batch and stream processing, with high performance and low latency capabilities.
- Real-time analytics is made accessible and cost-effective, dispelling the myth that it is not worth the investment.
- Integration of dbt and Flink facilitates the future of data processing, with focus on real-time and streaming analytics.
- Apache Flink
- Please click here to read the installation process for dbt-Flink-adapter
See it in Action
Created Pipeline in Flink: