Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

TechEvent 2019: Wie sichere ich eigentlich Kafka ab?; Markus Bente - Trivadis

24 visualizaciones

Publicado el

Wie sichere ich eigentlich Kafka ab?; Markus Bente - Trivadis TechEvent 2019

Publicado en: Tecnología
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Sé el primero en recomendar esto

TechEvent 2019: Wie sichere ich eigentlich Kafka ab?; Markus Bente - Trivadis

  1. 1. blog.trivadis.com@trivadis Wie sichere ich eigentlich Kafka ab? Markus Bente
  2. 2. Markus Persönliche Interessen: D&D Abende Modellbau Golf @trivadis blog.trivadis.com
  3. 3. Agenda 1. Architektur 2. Allgemeine Möglichkeiten 3. Zookeeper Security 4. Kafka Security 5. Demo
  4. 4. Architektur
  5. 5. Architektur Kafka Broker Kafka Broker Zookeeper Zookeeper Zookeeper Client Kafka Broker
  6. 6. Architektur Kafka Broker Kafka Broker Zookeeper Zookeeper Zookeeper Client Kafka Broker
  7. 7. Architektur Kafka Broker Kafka Broker Zookeeper Zookeeper Zookeeper Client Kafka Broker
  8. 8. Allgemeine Möglichkeiten
  9. 9. Allgemeine Möglichkeiten • Netzwerk Security • Netzseperieriung • Host Firewall • Linux • Benutzerkonzept • ACLs
  10. 10. firewall-cmd --new-zone=kafka-cluster --permanent firewall-cmd --zone=kafka-cluster --add-port=2181/tcp --permanent firewall-cmd --zone=kafka-cluster --add-port=2888/tcp --permanent firewall-cmd --zone=kafka-cluster --add-port=3888/tcp --permanent firewall-cmd --zone=kafka-cluster --add-port=9092/tcp --permanent firewall-cmd --zone=kafka-cluster --add-source=Broker –permanent firewall-cmd --new-zone=kafka-clients --permanent firewall-cmd --zone=kafka-clients --add-port=9092/tcp --permanent firewall-cmd --zone=kafka-clients --add-source=Client --permanent Allgemeine Möglichkeiten • Beispiel für eine Host Firewall
  11. 11. Allgemeine Möglichkeiten • Netzseperieriung Kafka Broker Kafka Broker Zookeeper Zookeeper Zookeeper Client Kafka Broker
  12. 12. Zookeeper Security
  13. 13. Zookeeper Security - Authentication • Kerberos • Zentrales Principle Management • DigistMD5 • Secret ist im JAAS file im Klartext Zookeeper Kafka Broker Zookeeper Kafka Broker jaas Config File jaas Config File jaas Config Filejaas Config File SASL Absicherung mittels Zwischen Zookeeper – Zookeeper & Zookeeper – Broker
  14. 14. Zookeeper Security - Authorization • ACLs • Muss enabled werden auf Broker Seite • Wenn nachträglich eingeschaltet Migration erforderlich • World Read ist Standard
  15. 15. #Zookeeper Konfiguration authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme=sasl #Broker Konfiguration zookeeper.set.acl=true #Zookeeper Jaas Server { org.apache.zookeeper.server.auth.DigestLoginModule required user_super="super" user_kafka="kafka"; }; #Broker Jaas Client { org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="kafka"; }; export KAFKA_OPTS="- Djava.security.auth.login.config=/apps/config/kafka/zookeeper_jaas.conf" Zookeeper Security - Konfigurationen
  16. 16. Kafka Security
  17. 17. Kafka Security - Encryption • Wire Encryption • SSL • Nachteil kein zero-copy mehr möglich • Message Encryption • Applikationseitige Encyprion
  18. 18. Kafka Security - Authentication SASL Non production usage • Plain • Passwort ist im JAAS file im Klartext • Nur für Test Zwecke • OAUTH • Unterstützt default nur ungesicherte JSON Web Tokens
  19. 19. Kafka Security - Authentication SASL Production usage • Kerberos • Zentrales Principle Management • Ausgereifter Standard • SCRAM (Salted Challenge Response Authentication Mechanism) • Credentials werden standardmässig im Zookeeper gespeichert • Kann angepasst werden • Netzseperierung Zookeeper • SHA-256 & SHA-512 mit min. 4096 Wiederholungen
  20. 20. Kafka Security - 2 Way SSL Kafka Broker CA Server 1 Keystore Truststore Kafka Broker CA Server 2 Keystore Truststore Kafka Broker CA Server 3 Keystore Truststore InterBrokerAuthmittelsSSL CA Client A Keystore Truststore CA Client B Keystore Truststore CA Client B Keystore Truststore Applikation mit mehreren Servern ClientAuthmittelsSSL
  21. 21. Kafka Security - Authentication 2 Way SSL Production usage • 2 Way SSL • Zentrales CA Management • Server / Client Auth Extentions möglich • Key & Trust Store für jeden Prozess • Authentication über Abgleich des Trusts • Passwörter für Keys & Stores im Broker Konfigurations Datei • Seit Kafka 2.0 Dynamische Konfiguration möglich • Passwörter veschlüsselt im Zookeeper
  22. 22. security.inter.broker.protocol=SSL listeners=SSL://:9092 ssl.client.auth=required ssl.truststore.location=/cert/broker.server.truststore.jks ssl.truststore.password=broker ssl.keystore.location=/cert/broker.server.keystore.jks ssl.keystore.password=broker ssl.key.password=broker super.users=User:CN=broker1,OU=Applikation,O=FastData,L=BE,ST=BE,C=CH; broker2,OU=Applikation,O=FastData,L=BE,ST=BE,C=CH Kafka Security - Konfigurationen 2 Way SSL
  23. 23. #Broker [req_cert_extensions] subjectAltName=@subject_alt_name keyUsage=critical,digitalSignature,keyEncipherment extendedKeyUsage= serverAuth,clientAuth #Client [req_cert_extensions] keyUsage=critical,digitalSignature,keyEncipherment extendedKeyUsage= clientAuth Kafka Security - Zertifikate SSL Extentions
  24. 24. Kafka Security - Authorization • ACLs • ACLs werden im Zookeeper abgelegt • Default SSL User Name = DN • Rule Matching möglich • Kerberos primäre Teil des principles • Super User Eintrag in der Broker Konfiguration
  25. 25. allow.everyone.if.no.acl.found=false Kafka Security - Konfigurationen ACLs Broker Konfiguration kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=ApplikationA,OU=Applikation,O=FastData,L=BE,ST=BE,C=CH --operation read --topic 'testSSLZugriff' CLI für neue ACLs
  26. 26. DEMO
  27. 27. # Broker A ./gen_broker.sh brokera.test.com broker DNS.1=brokera.test.com openssl req -new -config brokera.test.com.conf -keyout brokera.test.com.key -out brokera.test.com.csr -passout pass:broker openssl x509 -req -CA ca-cert -CAkey ca-key -in brokera.test.com.csr -out brokera.test.com.cert -days 356 -extensions req_cert_extensions -extfile brokera.test.com.conf -passin pass:test cat brokera.test.com.key > brokera.test.com.pem cat brokera.test.com.cert >> brokera.test.com.pem openssl pkcs12 -export -in brokera.test.com.pem -out brokera.test.com.pkcs12 -passin pass:broker -passout pass:broker keytool -list -v -keystore brokera.test.com.pkcs12 -storetype PKCS12 #test123 keytool -keystore broker.truststore.jks -alias CA -import -file ca-cert keytool -importkeystore -destkeystore brokera.test.com.keystore.jks -deststoretype JKS -srcstoretype PKCS12 -srckeystore brokera.test.com.pkcs12 -srcstorepass broker - deststorepass broker Kafka Security - Konfigurationen
  28. 28. DEMO
  29. 29. DEMO
  30. 30. # server.property security.inter.broker.protocol=SSL ssl.client.auth=required ssl.truststore.location=/apps/certs/broker.truststore.jks ssl.truststore.password=test123 ssl.keystore.location=/apps/certs/brokera.test.com.keystore.jks ssl.keystore.password=broker #ssl.key.password=test ssl.client.auth=required zookeeper.set.acl=true authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer allow.everyone.if.no.acl.found=false super.users=User:CN=brokera.test.com,OU=Broker,O=Cluster2,L=BE,ST=BE,C=CH;User:CN=brok erb.test.com,OU=Broker,O=Cluster2,L=BE,ST=BE,C=CH;User:CN=brokerc.test.com,OU=Broker,O =Cluster2,L=BE,ST=BE,C=CH listeners=SSL://0.0.0.0:9092 advertised.listeners=SSL://brokera.test.com:9092 DEMO
  31. 31. DEMO
  32. 32. # zookeeper.prop dataDir=/tmp/zookeeper # the port at which the clients will connect clientPort=2181 # disable the per-ip limit on the number of connections since this is a non-production config maxClientCnxns=0 authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme=sasl DEMO
  33. 33. DEMO
  34. 34. export KAFKA_OPTS=-Djava.security.auth.login.config=/apps/sasl/kafka/kafka.jaas #kafka /apps/confluent/bin/kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=client,OU=Broker,O=Cluster1,L=BE,ST=BE,C=CH --operation read --topic 'testSSLZugriff' --group='*’ #kafka /apps/confluent/bin/kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=MM,OU=Broker,O=Cluster1,L=BE,ST=BE,C=CH --operation write --topic 'testSSLZugriff’ #kafka2 /apps/confluent/bin/kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=MM,OU=Broker,O=Cluster1,L=BE,ST=BE,C=CH --operation read --topic 'testSSLZugriff' --group='*’ #kafka2 /apps/confluent/bin/kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=client,OU=Broker,O=Cluster1,L=BE,ST=BE,C=CH --operation write --topic 'testSSLZugriff’ #kafka1+2 /apps/confluent/bin/kafka-topics --zookeeper=localhost:2181 --create --topic testSSLZugriff --replication-factor 3 --partitions 3 DEMO
  35. 35. DEMO
  36. 36. [root@kafka mirror]# cat consumer.conf bootstrap.servers=brokera.test.com:9092,brokerb.test.com:9093,brokerc.test.com:9094 exclude.internal.topics=true isolation.level=read_committed auto.offset.reset=earliest group.id=Replication_Consumer security.protocol=SSL ssl.truststore.location=/apps/certs/broker.truststore.jks ssl.truststore.password=test123 ssl.keystore.location=/apps/certs/MM.keystore.jks ssl.keystore.password=mirror [root@kafka mirror]# cat producer.conf bootstrap.servers=broker1.test.com:9092,broker2.test.com:9093,broker3.test.com:9094 acks=all security.protocol=SSL ssl.truststore.location=/apps/certs/broker.truststore.jks ssl.truststore.password=test123 ssl.keystore.location=/apps/certs/MM.keystore.jks ssl.keystore.password=mirror DEMO
  37. 37. /apps/confluent/bin/kafka-mirror-maker --consumer.config /apps/mirror/consumer.conf -- producer.config /apps/mirror/producer.conf --whitelist testSSLZugriff DEMO [root@kafka2 ~]# cat /apps/certs/client.prop security.protocol=SSL ssl.truststore.location=/apps/certs/broker.truststore.jks ssl.truststore.password=test123 ssl.keystore.location=/apps/certs/client.keystore.jks ssl.keystore.password=test123 /apps/confluent/bin/kafka-console-producer --topic testSSLZugriff --broker-list brokera.test.com:9092 --producer.config /apps/certs/client.prop /apps/confluent/bin/kafka-console-consumer --topic testSSLZugriff --bootstrap-server broker1.test.com:9092 --consumer.config /apps/certs/client.prop
  38. 38. DEMO
  39. 39. DEMO

×