1.
Intro
Kafka Cluster
incoming message
- without kafka client library support
(e.g. Android or IOS app, IOT devices, …)
- send message using Rest APIs

2.
Intro
source : https://docs.confluent.io/platform/current/kafka-rest/index.html#confluent-rest-apis

3.
License
source : https://www.confluent.io/confluent-community-license-faq/

4.
Installation
• wget https://packages.confluent.io/archive/7.3/confluent-community-7.3.0.tar.gz
• tar -zxvf confluent-community-7.3.0.tar.gz
[root@master ~]# cd confluent-7.3.0/
[root@master confluent-7.3.0]# vi etc/kafka-rest/kafka-rest.properties
#id=kafka-rest-test-server
#schema.registry.url=http://localhost:8081
zookeeper.connect=master:2181,slave1:2181,slave2:2181
bootstrap.servers=master:9092,slave1:9092,slave2:9092
By default the server starts bound to port 8082,
does not specify a unique instance ID (required to safely run multiple proxies concurrently)
configuration settings : https://docs.confluent.io/platform/current/kafka-rest/production-deployment/rest-proxy/config.html#configuration-options-for-https

5.
Run & Test
• 1. Zookeeper Ensemble, Kafka Cluster Running
• 2. execute shell at kafka rest home directory
sh bin/kafka-rest-start etc/kafka-rest/kafka-rest.properties
default port : 8082

6.
1. List all the topics in a Kafka cluster
• curl -X GET "http://localhost:8082/topics/"

7.
2. Query Kafka Topic
• curl -X GET -H "Accept: application/vnd.kafka.v2+json"
"http://localhost:8082/topics/kopo-topic"

8.
3. Sending data to Kafka Topic
• curl -X POST -H "Content-Type: application/vnd.kafka.json.v2+json" --
data '{"records":[{"value":{"name": "testUser"}}]}'
"http://localhost:8082/topics/kopo-topic"
Producer
Consumer

9.
4. Prepare Consumer
• consumer id and group registration
• curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"name":
"my_consumer_instance", "format": "json", "auto.offset.reset": "earliest"}'
http://localhost:8082/consumers/my_consumer_group

10.
4-1. Mapping topic with consumer
• curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data
'{"topics":["kopo-topic"]}'
http://localhost:8082/consumers/my_consumer_group/instances/my_consumer_instance
/subscription

11.
4-2. Receiving data
• curl -X GET -H "Accept: application/vnd.kafka.json.v2+json"
http://localhost:8082/consumers/my_consumer_group/instances/my_consumer_instance/records

12.
etc.
• Idle consumer will be destroyed after 300 seconds by default
• Delete
curl -X DELETE -H "Content-Type: application/vnd.kafka.protobuf.v2+json" http://localhost:8082/consumers/[consumer group]/instances/[consumer
instance id]
source : https://docs.confluent.io/platform/current/kafka-rest/production-deployment/rest-proxy/config.html#configuration-options-for-https
for more examples : https://docs.confluent.io/platform/current/kafka-rest/quickstart.html#produce-and-consume-json-schema-messages

13.
I Think,
• In environments where the Kafka-client is not available, Rest
Proxy is a good option to consider.
• Detailed options are not available compared to implementing
directly with the Kafka-client.
• The Rest Proxy shall be used after the performance test.
• I think it is more appropriate to apply a reverse proxy using
nginx, etc., and link it with a back-end application using the
kafka-client.