Confluent has added a number of new features to its platform for its Confluent Q2 ‘22 cloud bundle release, including enhanced role-based access control (RBAC) and Grafana-metrics support. The release represents its latest effort to expand on data streaming support and services built on open source Kafka for organizations requiring streaming to maintain real- or near real-time access to data at scale. These organizations also often rely on in-cache memory to improve data-connection latency and the use of a shared data layer that is continually refreshed as the data associated with distributed systems of records is updated (a process also described as change data capture (CDC)).
Expanded Access Control
Confluent’s RBAC control for individual Kafka resources now extends to Kafka topics (which describe the different data systems of record), consumer groups and transactional IDs. While this tighter RBAC control for onboarding can always be done manually, the process of ensuring the right users have access to the right data “all the way down to individual Apache Kafka topics requires time and resources that most enterprises don’t have,” Dan Rosanova, head of product for Confluent Cloud, told The New Stack. Enterprises can set clear roles for those accessing data giving them visibility into only those areas specifically required for their work with the new RBAC capability.
Beyond helping to solve the challenges associated with access control when scaling databases, RBAC also enables organizations to quickly onboard development teams and hand over the responsibility of managing specific Kafka resources to them, Rosanova said. “This ensures that Kafka resources are managed by the true owners, which was very complex to manage in the past due to the complicated scripts that needed to be written,” Rosanova said.
Other streaming capabilities on offer include:
Fully managed Oracle CDC Source Connector. This feature enables the source-connector support for Oracle CDC Source Connector, which previously required investments in proprietary solutions, licensing fees or in-house resources to design, build and test a customized connector for Oracle CDC. The release also provides support for Amazon S3 Source, Azure Cosmos DB Source, InfluxDB 2 Source, InfluxDB 2 Sink and Jira Source connectors.
Metrics API expansion and Grafana Cloud integration. For better metrics observability, metrics access is improved through direct access to and support for Grafana Cloud integrations for those organizations that rely on Grafana Cloud’s popular visualizations and observability tools and want to use Grafana for data-streaming performance monitoring. Confluent’s Metrics API also offers Principal ID breakdown for organizational chargebacks and consumer lag metric for performance monitoring.
Truly Cloud Native
These and other enhancements Confluent is developing for the future are part of its mission to rebuild its offerings in the cloud to make them “truly cloud native,” Rosanova said. “This has resulted in many enterprises starting their data streaming journey in Confluent Cloud, and existing customers adopting Confluent to manage hybrid and multicloud environments,” Rosanova said. “Having a consistent experience across cloud provides with all the security and manageability that enterprises need makes Confluent the base option around.”
Confluent also has not changed its bet that streaming and, especially, services built for streaming that are based on Kafka, will play a key role in the development of CDC in the future, as organizations seek to scale their databases and operations dependent on them, often across disparate geographical zones.
“We believe data streaming is at the center of the next major wave of technology innovation companies are undergoing,” Rosanova said. “The ability to harness it will be a key factor in determining the competitiveness of every company, especially as real-time experiences and operations become table stakes. Open source Apache Kafka started with movement, and saw tremendous adoption with over 80% of the Fortune 100.”
For organizations that are just getting started with streaming, adoption usually begins with a single-use case that harnesses streams of data using open source, Rosanova said.However, a key aspect of this platform is that these data streams are meant to be shared, and these streams attract new applications.
“This is a virtuous cycle that drives the adoption of the platform: apps bring streams, which, in turn, attract more apps. As use cases grow, the cost and time associated with managing Kafka increases too,” Rosanova said. “For organizations that want to offload the operational burden, we offer a fully managed Kafka service. We provide streaming ‘as a service’ that doesn’t require you to learn about the underlying infrastructure. Instead, you can focus on the data and the value you derive from it.”
Confluent, AWS, Oracle and InfluxData are sponsors of The New Stack.
