Data contains information and information is actionable. Data-driven enterprises are increasingly looking for ways to understand data and therefore the movement of data should be seamless across the organization. Therefore how data is moved is as important as data itself. More speed simply equates to more value for data and that's what Confluent is striving to achieve.
When Apache Kafka was created it revolutionized more than half of the existing Fortune 500 companies. From travel companies to banks to streaming services, Kafka has its roots in a number of applications. There are companies where data pours indefinitely and they are the biggest fans of Kafka. For instance LinkedIn, where Kafka originated uses it in many activities like tracking activity data and operational mechanics.
Kafka is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system, and can work with Flume/Flafka, Spark Streaming, for real-time ingesting, analysis and processing of streaming data. Kafka is normally used as a central repository of streams, where events are stored in Kafka for an intermediate period of time before they are routed elsewhere in a data cluster for further processing and analysis.
Story of Confluent
In 2014, creators of the Apache Kafka founded Confluent after leaving LinkedIn. Primarily the company focused on the open-source Apache Kafka: real-time messaging technology that was created by the founders. Over time they become major players n the Stream Analytics that acts as a doorway to key business intelligence insights through real-time data exploration.
The company started by providing commercial support for building tools based on Apache Kafka and the company has raised more than $31 million in venture capital funding. Confluent's business is based on the open-source software but the company provides premium features with added capabilities for a fee. Confluent also offers technical consulting and training as a service to its customers.
A number of big names like Netflix, Airbnb, Hotels.com, Uber, Spotify, Cisco, Goldman Sachs, Wal-Mart now use Kafka for real-time analysis. Also, the company is used across financial services firms to reduce credit card frauds.
Confluent essentially provides advanced management software by subscription to help large companies run Kafka for their production systems.
Confluent Platform enables all the interfaces to the world to be connected to decide with all the internal systems in real-time. Kafka helps in moving data across the company and accessing it as a continuously free-flowing stream within seconds, Kafka’s plug-ins also let enterprises connect their existing database systems with these tools, which allows them to leverage these new technologies without having to switch to a new database platform.
In 2015 the company launched Confluent Platform 2.0 with an updated Apache Kafka 0.9 core that had several verticals like multi-tenant operation, simplified development and maintenance of applications that produce or consume data in Kafka, and provide high-output, scalable data integration with a wide array of data sources. Recently Confluent launched its latest version Confluent Platform 3.0 which has a comprehensive management system for Apache Kafka with a Confluent control center to help data engineering teams operationalize Kafka throughout their organization.
Confluent is now battle-ready to go head-to-head with other cloud computing giants with its release of a cloud-native and fully managed service based upon the Apache Kafka streaming platform.
Confluent's latest cloud-native version is more scalable and introduces variable pricing based upon consumption, which drastically helps in reducing costs for most cloud customers. The company assures that the Confluent cloud can scale dynamically between zero and 100 megabytes per second of throughput in seconds and support gigabyte-per-second deployments with planned provisioning.
By understanding the most common demand of customers, the company is also launching three managed services which include a managed central registry to define and share standard schemas for events, a streaming SQL query engine and automated connections to most commonly used data source which is usually Amazon Web services (AWS) S3 storage.
Working at Confluent
There’s always something new happening each week
Confluent is much more than just work. The team enjoys homemade treats for bake-off competitions and regularly heads out of the office to lunch together every week. Confluent's vision is radically different from the rest of the companies and that maybe one of the reason employees stick around the company, they also have different types of spaces in which to work and collaborate. They also host Kafka meetups in the UK.
“I feel comfortable, encouraged, and challenged. To me, that’s the definition of a good culture.” — Guozhang Wang (Engineering Lead)
The IT wizard
Jay Kreps is the CEO of Confluent Inc, a company that is based on the popular Apache Kafka Messaging system, He has also authored several open source projects like Apache Kafka, Apache Samza (a stream processing system), Project Voldemort (a key-value store) and Azkaban. Before founding Confluent he was formerly the lead architect for data infrastructure at LinkedIn.
“Confluent offers a streaming platform based on Apache Kafka that enables companies to easily access data as real-time streams.”