Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Level-up your gaming telemetry using Kafka Streams | DevNation Tech Talk

Level-up your gaming telemetry using Kafka Streams | DevNation Tech Talk

Descargar para leer sin conexión

Many modern video games are constantly evolving post-release. New maps, game modes, and game balancing adjustments are rolled out, often on a weekly basis. This continuous iteration to improve player engagement and satisfaction requires data-driven decision making based on events and telemetry captured during gameplay, and from community forums and discussions.

In this session you will learn how OpenShift Streams for Apache Kafka and Kafka Streams can be used to analyze real-time events and telemetry reported by a game server, using a practical example that encourages audience participation. Specifically you’ll learn how to:

Provision Kafka clusters on OpenShift Streams for Apache Kafka.
Develop a Java application that uses Kafka Streams and Quarkus to process event data.
Deploy the application locally, or on OpenShift and connect it to your OpenShift Streams for Apache Kafka Cluster.

Many modern video games are constantly evolving post-release. New maps, game modes, and game balancing adjustments are rolled out, often on a weekly basis. This continuous iteration to improve player engagement and satisfaction requires data-driven decision making based on events and telemetry captured during gameplay, and from community forums and discussions.

In this session you will learn how OpenShift Streams for Apache Kafka and Kafka Streams can be used to analyze real-time events and telemetry reported by a game server, using a practical example that encourages audience participation. Specifically you’ll learn how to:

Provision Kafka clusters on OpenShift Streams for Apache Kafka.
Develop a Java application that uses Kafka Streams and Quarkus to process event data.
Deploy the application locally, or on OpenShift and connect it to your OpenShift Streams for Apache Kafka Cluster.

Más Contenido Relacionado

Similares a Level-up your gaming telemetry using Kafka Streams | DevNation Tech Talk

Más de Red Hat Developers

Audiolibros relacionados

Gratis con una prueba de 30 días de Scribd

Ver todo

Level-up your gaming telemetry using Kafka Streams | DevNation Tech Talk

  1. 1. CONFIDENTIAL designator An Exploration of the Kafka Streams API Processing Game Telemetry with OpenShift Streams for Apache Kafka Evan Shortiss Technical Marketing Manager 1 @evanshortiss
  2. 2. CONFIDENTIAL designator Welcome! 2 Source: Spider-Man (2018) by Insomniac Games and Sony Interactive Entertainment
  3. 3. CONFIDENTIAL designator 3 Apache Kafka is a distributed system designed for streams. It is built to be an high-availability, horizontally-scalable, fault-tolerant, commit log, and allows distributed data streams and stream processing applications. Known use cases are: ▸ High-performance data pipelines ▸ Streaming analytics ▸ Data integration What is Kafka? What is Apache Kafka? Open-source distributed event streaming platform Learn more on the Apache Kafka community page: https://kafka.apache.org/
  4. 4. CONFIDENTIAL designator Source: https://kafka.apache.org/ Brokers, topics, partitions, producers, and consumers! What is Kafka? 4 What is Apache Kafka? Kafka Cluster Broker A Broker B Broker C Consumer Group Topic: Clicks Topic: Purchases P1 P2 P3 P1 P2 P3 Producer Group
  5. 5. CONFIDENTIAL designator Source: https://kafka.apache.org/intro What is Kafka? 5 APIs What is Apache Kafka? ▸ The Admin API to manage and inspect topics, brokers, and other Kafka objects. ▸ The Producer API to publish (write) a stream of events to one or more Kafka topics. ▸ The Consumer API to subscribe to (read) one or more topics and to process the stream of events produced to them. ▸ The Kafka Streams API to implement stream processing applications and microservices. ▸ The Kafka Connect API to build and run reusable data import/export connectors that consume (read) or produce (write) streams of events from and to external systems and applications so they can integrate with Kafka.
  6. 6. CONFIDENTIAL designator Kafka & Video Game Telemtry 6 Use Cases Kafka and Video Game Telemetry ▸ Modern video games are constantly evolving post-release. ▸ Developers release patches with new features and game balance fixes. ▸ Data-driven decision making can increase the value of each iteration. ▸ Popular video games have millions of daily active users. ▸ Generate millions/billions of telemetry events and data. ▸ Kafka, Kafka Streams, and the broader ecosystem can be used for ingestion and processing data! Source: https://steamcharts.com (July 14th 2021) 600,000 Counter-Strike players firing just 20 shots could generate 12,000,000 unique events!
  7. 7. CONFIDENTIAL designator Shipwars, a Battleship style game where players face-off against AI opponents. First to sink opponent ships is the winner. Bonus round provides an opportunity to boost high-scores. 7 Kafka at Red Hat Summit 2021 Red Hat Summit 2021
  8. 8. CONFIDENTIAL designator Kafka at Red Hat Summit 2021 8 3 Clusters, Open Hybrid Cloud AWS, Azure, and GCP
  9. 9. CONFIDENTIAL designator Scoring API Kafka at Red Hat Summit 2021 9 Game Server WebSocket Game State Players Matches Scores Binary Cloud Event Broker HTTP Cloud Events Scoring/Bonus APIs HTTP Cloud Events Binary Strimzi Kafka 3 Clusters, Open Hybrid Cloud AWS, Azure, and GCP AI Move REST API HTTP Request Next Move Bot State Machines HTTP Request Create AI Bot Player WebSocket
  10. 10. CONFIDENTIAL designator Kafka at Red Hat Summit 2021 10 3 Clusters, Open Hybrid Cloud AWS, Azure, and GCP Search for Katia Aresti’s content about the leaderboard and Infinispan replication on Red Hat Developers YouTube and Blog!
  11. 11. CONFIDENTIAL designator Scoring API Kafka at Red Hat Summit 2021 11 Game Server WebSocket Game State Players Matches Scores Binary Cloud Event Broker Scoring/Bonus APIs HTTP Cloud Events Binary Kafka Leaderboard Binary WebSocket 3 Clusters, Open Hybrid Cloud AWS, Azure, and GCP HTTP Cloud Events AI Move REST API HTTP Request Next Move Bot State Machines HTTP Request Create AI Bot Player WebSocket
  12. 12. CONFIDENTIAL designator Human: Connected AI: Connected 12 Introduction Event Stream
  13. 13. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit 13 Introduction Event Stream
  14. 14. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss 14 Introduction Event Stream
  15. 15. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit 15 Introduction Event Stream
  16. 16. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss 16 Introduction Event Stream
  17. 17. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss Human: [3,1] / Hit 17 Introduction Event Stream
  18. 18. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss Human: [3,1] / Hit AI: [1,3] / Hit 18 Introduction Event Stream
  19. 19. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss Human: [3,1] / Hit AI: [1,3] / Hit Human: [3,2] / Hit 19 Introduction Event Stream
  20. 20. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss Human: [3,1] / Hit AI: [1,3] / Hit Human: [3,2] / Hit AI: [2,3] / Miss 20 Introduction Event Stream
  21. 21. CONFIDENTIAL designator Human: Connected AI: Connected Human: [4,0] / Hit AI: [3,1] / Miss Human: [4,1] / Hit AI: [1,2] / Miss Human: [3,1] / Hit AI: [1,3] / Hit Human: [3,2] / Hit AI: [2,3] / Miss Human: [1,1] / Miss 21 Introduction Event Stream
  22. 22. CONFIDENTIAL designator Product Overview 22 https://red.ht/TryKafka Development Preview trial available for free at:
  23. 23. CONFIDENTIAL designator Red Hat OpenShift Streams for Apache Kafka Complete solution for stream-based applications Product Overview 23 Metrics & monitoring Configuration mgmt Hosted & managed (99.95% SLA) Kafka cluster Broker(s) Topic(s) Streamlined developer experience: a curated solution with a developer-first, consistent experience Delivered as a service, managed by Red Hat SRE - 24x7 global support and a 99.95% service-level agreement (SLA) Real-time, streaming data broker - Dedicated Apache Kafka cluster deliver as a service in the cloud and location of choice ▸ Access to Kafka brokers, topics, and partitions ▸ Configuration management ▸ Metrics and monitoring ▸ UI / CLI / API / service bindings ▸ Integrated identity & access management STREAMLINED DEVELOPER EXPERIENCE UI API CLI Service Binding https://red.ht/TryKafka
  24. 24. CONFIDENTIAL designator 24 ▸ Based on the Strimzi community project that provides an optimized version of Apache Kafka on Kubernetes ▸ Access to customer dedicated Kafka instances. Each instance provides Kafka core capabilities ▸ Kafka instances are configured based on opinionated recommendations made by Red Hat experts Product Overview A dedicated managed Apache Kafka cluster An opinionated experience for your Kafka service Apache Kafka implementation optimized for OpenShift https://red.ht/TryKafka
  25. 25. CONFIDENTIAL designator Kafka Streams Topology 25
  26. 26. CONFIDENTIAL designator Kafka Streams Topology 26
  27. 27. CONFIDENTIAL designator End the Keynote demo with a twist. Prove that we can track all events throughout the game! 27 Introduction Red Hat Summit 2021 ALL YOUR BASE EVENT ARE BELONG TO US. Source: Zero Wing by Namco (1989) and Williams Electronics (1990)
  28. 28. CONFIDENTIAL designator Record player “connect” events. Each player and match is unique. Capture each “attack” by each player. Join these events together to create a complete record of each match. 28 Introduction Aggregate Match Visualisation
  29. 29. CONFIDENTIAL designator Record player “connect” events and player type. Capture each “attack” by each player. Create a matrix using this data to understand where humans tend to shoot or place their ships. Subscribe the AI model to this real-time data so it continuously learns. 29 Introduction Real-time AI Adjustment https://github.com/redhat-gamedev/shipwars-move-server/blob/master/prediction.py
  30. 30. CONFIDENTIAL designator Game Server Game Server Kafka Streams Topology 30 Game Server WebSocket Attacks Topic Key Value (Player,Result, Cell) GameVer:MatchA PlayerA,Hit [0,1] GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerA,Hit [1,2] GameVer:MatchB PlayerD,Miss [4,1] Players Topic Key Value (Name, Type, Region) PlayerA Slow Bow, Human, EMEA PlayerB Iron Ocelot, AI, EMEA PlayerC Plume Boar, Human, APAC PlayerD Rose Fang, AI, APAC
  31. 31. CONFIDENTIAL designator Game Server Game Server Kafka Streams Topology 31 Game Server WebSocket Attacks Topic Key Value (Player,Result, Cell) GameVer:MatchA PlayerA,Hit [0,1] GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerA,Hit [1,2] GameVer:MatchB PlayerD,Miss [4,1] Source: Super Mario 3D World via https://mario.fandom.com Players Topic Key Value (Name, Type, Region) PlayerA Slow Bow, Human, EMEA PlayerB Iron Ocelot, AI, EMEA PlayerC Plume Boar, Human, APAC PlayerD Rose Fang, AI, APAC
  32. 32. CONFIDENTIAL designator Game Server Game Server Kafka Streams Topology 32 Game Server WebSocket Attacks Topic Key Value (Player,Result, Cell) GameVer:MatchA PlayerA,Hit [0,1] GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerA,Hit [1,2] GameVer:MatchB PlayerD,Miss [4,1] Source: Super Mario 3D World via https://mario.fandom.com Players Topic Key Value (Name, Type, Region) PlayerA Slow Bow, Human, EMEA PlayerB Iron Ocelot, AI, EMEA PlayerC Plume Boar, Human, APAC PlayerD Rose Fang, AI, APAC Kafka Streams API
  33. 33. CONFIDENTIAL designator Stateless Processing Kafka Streams Topology 33 Kafka Streams Attacks Topic Key Value (Player,Result, Cell) GameVer:MatchA PlayerA,Miss [0,1] GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerA,Hit [1,2] GameVer:MatchB PlayerD,Miss [4,1] Filter out missed shots Only Hits Topic Key Value (Player,Result, Cell) GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerA,Hit [1,2] Kafka Streams API Filter (Stateless)
  34. 34. CONFIDENTIAL designator Kafka Streams Join Game Server Game Server Kafka Streams Topology 34 Game Server WebSocket Topic: Attacks Key Value (Result, Cell) GameVer:MatchA PlayerA,Hit [0,1] GameVer:MatchA PlayerB,Hit [2,3] GameVer:MatchA PlayerC,Hit [1,2] GameVer:MatchB PlayerD,Miss [4,1] Topic: Players Key Value (Name, Type, Region) PlayerA Slow Bow, Human, EMEA PlayerB Iron Ocelot, AI, EMEA PlayerC Plume Boar, Human, APAC PlayerD Rose Fang, AI, APAC Topic: Enriched Attacks Key Value GameVer:MatchA Human,Hit [0,1] GameVer:MatchA AI,Hit [2,3] GameVer:MatchA Human,Hit [1,2] GameVer:MatchB AI,Miss [4,1] Stateful Processing Kafka Streams
  35. 35. CONFIDENTIAL designator Kafka Streams Topology 35 Topic: Enriched Attacks Key Value GameVer:MatchA Human,Hit [0,1] GameVer:MatchA AI,Hit [2,3] GameVer:MatchA Human,Hit [1,2] GameVer:MatchB AI,Miss [4,1] Topic: Match Aggregates Key Value GameVer:MatchA [ { human,hit:0,1 } ] GameVer:MatchA [ { human,hit:0,1 }, { ai,hit:2,3 } ] GameVer:MatchA [ { human,hit:0,1 }, { ai,hit:2,3 }, { human,hit:1,2 } ] GameVer:MatchB [ { ai,miss:4,1 } ] Aggregation State: Match Aggregates Key Value(s) GameVer:MatchA [ { human,hit:0,1 }, { ai,hit:2,3 }, { human,hit:1,2 } ] GameVer:MatchB [ { ai,miss:4,1 } ] Kafka Streams Aggregation Commit to Topic on a fixed interval. Can restore Kafka Streams table state from Topic. Kafka Streams Join
  36. 36. CONFIDENTIAL designator Game Server Game Server Kafka Streams Topology 36 Game Server WebSocket players attacks enriched-attacks match-aggregates Produce Produce GlobalKTable KStream Consume Consume Join on Player ID Produce KStream Consume Update Aggregate Record Group By Key (Game Ver. & Match ID) Produce Kafka Streams: Join Kafka Streams: Aggregation Kafka Topics Consume
  37. 37. CONFIDENTIAL designator Replay Service Each Game Server instance produces shots to the Managed Kafka Cluster Kafka Streams Topology 37 Game Server WebSocket Produce Consume Records players & attacks match-aggregates Produce Match Aggregates Query RocksDB via HTTP API Render Match Replay in Web Browser OpenShift Dedicated Single Region Processing Cluster OpenShift Dedicated 3 Region Game Clusters Kafka Streams Single Region
  38. 38. CONFIDENTIAL designator 1. Define Serdes. 2. Create shots KStream. 3. Create players GlobalKTable. 4. Join stream and table using composite key. 5. Return new record (String) that contains player type, shot result, and shot cell. 38 Kafka Streams Topology Kafka Streams: Join
  39. 39. CONFIDENTIAL designator 1. Define Serde. 2. Enriched shots KStream. 3. Group by key. 4. Aggregate grouped records. 5. Materialise record into store. 6. Write record to aggregate topic. 39 Kafka Streams Topology Kafka Streams: Aggregate
  40. 40. CONFIDENTIAL designator Kafka Streams Topology 40
  41. 41. CONFIDENTIAL designator Horizontally Scalable Kafka Streams Topology 41 Kafka Streams Scaling
  42. 42. CONFIDENTIAL designator Demo 42 Source: https://crimson-ceremony.net
  43. 43. CONFIDENTIAL designator linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 43 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you

×