Nginx Reverse Proxy - Install on Linux CentOS7 - using Kafka-Python as backend app. - use reverse proxy as a load balancer

2. Intro
Kafka Cluster
incoming message
- without kafka client library support
(e.g. Android or IOS app, IOT devices, …)
- send message using Rest APIs

3. why we use nginx?
source : https://www.nginx.com/resources/wiki/community/why_use_it/

4. Sample architecture
Kafka Cluster
upstream testme{
server backendAddr:8090
server backendAddr:8091
server backendAddr:8092
}
incoming message
http://[nignx server address]:8070/api
backendAddr:8090 backendAddr:8091 backendAddr:8092
Load Balancing
… (can be added)

5. Install Nginx
• vi /etc/yum.repos.d/nginx.repo
• yum install nginx
• firewall-cmd --permanent --zone=public --add-port=80/tcp
• firewall-cmd --reload
[nginx]
name=nginx repo
baseurl=https://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

6. Install Nginx
• Configuration Directory : /etc/nginx
• Configuration Reload : nginx -s reload
• Service Stop : systemctl stop nginx
• Service Start : systemctl start nginx http://your server address

7. [Port] Setting Permission and Open
• semanage port -l | grep http_port_t
• semanage port -a -t http_port_t -p tcp 8090
• semanage port -a -t http_port_t -p tcp 8091
• semanage port -a -t http_port_t -p tcp 8092
• semanage port -a -t http_port_t -p tcp 8070

8. Log
• access log
tail -f /var/log/nginx/access.log
• error log
tail -f /var/log/nginx/error.log

9. Configuration
reference : https://www.sobyte.net/post/2021-09/nginx-proxy-remove-prefixes/
upstream testme {
server 127.0.0.1:8090;
server 127.0.0.1:8091;
server 127.0.0.1:8092;
}
server {
listen 8070;
listen [::]:8070;
location /api {
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 3000;
proxy_pass http://testme/api;
}
}
# pwd
/etc/nginx/conf.d
# vi test.conf
firewall-cmd --permanent --zone=public --add-port=8070/tcp
firewall-cmd --reload
nginx -s reload
systemctl stop nginx
systemctl start nginx

10. Backend Application
• python3 testBackend.py 8090 &
• python3 testBackend.py 8091 &
• python3 testBackend.py 8092 &
testBackend.py
source : https://github.com/wonyongHwang/KopoBlockchain/tree/master/ref

11. Test
• curl --location --request POST 'http://192.168.56.30:8070/api' --header 'Content-
Type: application/json;encode=utf-8' --data-raw '{"key":"value"}'
request
response
application log
nginx access log

12. Backend Application as a Producer (1/2)
prerequisite : kafka-python (https://kafka-python.readthedocs.io/en/master/install.html)
testKafka.py
source : https://github.com/wonyongHwang/KopoBlockchain/tree/master/ref

13. Backend Application as a Producer (2/2)
request
response
application log
kafka-console-consumer.sh --bootstrap-server
master:9092,slave1:9092,slave2:9092 --topic kopo-
topic --from-beginning
kafka consumer

14. Reference
• https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/