Kafka Confluent Platform

About Confluent

Confluent is a company founded by the team that built Apache Kafka. It builds a platform around Kafka that enables companies to easily access data as real-time streams.

Confluent offers three different ways to get started with Kafka.

  1. Confluent Open Source
  2. Confluent Enterprise
  3. Confluent Cloud

While we in this series of Kafka Tutorial discuss much about Confluent Open Source, you may check the other two ways based on your requirement and interest.

While comparing Confluent Open Source with Apache Kafka, we get the following capabilities or tools in addition to standard Apache Kafka :

  • Additional Clients : Supports C, C++, Python, .NET and several other non-Java Clients.
  • REST Proxy – Provides universal access to Kafka from any network connected device via HTTP
  • Schema Registry – Central registry for the format of Kafka data – guarantees all data is always consumable
  • Pre-Built Connectors – HDFS, JDBC, Elasticsearch, Amazon S3 and other connectors fully certified and supported by Confluent

Install Kafka Confluent Open Source on Ubuntu

To install Confluent Open Source on Ubuntu, following is a step by step guide :

1. Install Confluent public key

$ wget -qO – http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add –
root@tutorialkart:/home/arjun# wget -qO - http://packages.confluent.io/deb/3.3/archive.key | sudo apt-key add -
OK

2. Add the repository to sources list

$ sudo add-apt-repository “deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main”
root@tutorialkart:/home/arjun# sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.3 stable main"

3. Update Packages

$ sudo apt-get update

4. Install Confluent Open Source Platform

$ sudo apt-get install confluent-platform-oss-2.11

2.11 at the end refers to the scala version that is currently supported.

root@tutorialkart:/home/arjun# sudo apt-get install confluent-platform-oss-2.11
Reading package lists... Done
Building dependency tree      
Reading state information... Done
...

Following would be the locations

LocationDescription
/usr/bin/Confluent CLI and individual driver scripts for starting/stopping services, prefixed with <package> names
/etc/<package>/Configuration files. <package> : [confluent-common/ confluent-control-center/ confluent-control-center-fe/ confluent-rebalancer/]
/usr/share/java/<package>/Jars <package> : [kafka/ kafka-connect-jdbc/ kafka-connect-storage-common/ kafka-connect-elasticsearch/ kafka-connect-replicator/ kafka-rest/ kafka-connect-hdfs/ kafka-connect-s3/ kafka-serde-tools/ ]

5. Start Confluent

You may start all or some of the services using confluent command line interface with start command.

root@tutorialkart:~# confluent start
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
connect is [UP]

Conclusion

In this Kafka Tutorial, we have learnt about Confluent Platform and how to install Confluent Open Source Platform.