Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Introduction to Akka Cluster Sharding

Explore how to make your next application stateful, providing a better understanding of the technology landscape, challenges and pitfalls, and successful methods with Akka Cluster Sharding

  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Introduction to Akka Cluster Sharding

  1. 1. Presented By: Anjali Sharma @anjalihere3 Software Consultant Knoldus Inc
  2. 2. Lack of etiquette and manners is a huge turn off. KnolX Etiquettes Punctuality Respect Knolx session timings, you are requested not to join sessions after a 5 minutes threshold post the session start time. Feedback Make sure to submit a constructive feedback for all sessions as it is very helpful for the presenter. Silent Mode Keep your screens on mute unless you have query. Avoid Disturbance Avoid unwanted chit chat during the session.
  3. 3. Agenda
  4. 4. Sharding ● The term Sharding means Partitioning. ● It's a technique that mostly databases use to improve their elasticity and resiliency.
  5. 5. Database Sharding ● Records are distributed across nodes, using a shard key or a partition key. ● A router which directs requests to the appropriate shard or partition. ● Even after sharding, it may lead to bottleneck.
  6. 6. Akka Cluster Sharding ● The Akka toolkit provides cluster sharding as a way to introduce sharding into your application. ● Instead of distributing database records across a cluster, we are going to distribute actors across the cluster. ● Each actor is then going to act as a consistency boundary, for the data that it manages.
  7. 7. Entities ● The basic unit in akka cluster sharding is an actor called an entity. ● There is only one entity per entity ID in the cluster. ● Messages are addressed to the entity ID and processed by the entity. This allows the entity to act as a single source of truth, acting as a consistency boundary for the data that it manages.
  8. 8. Shards ● Entities are distributed in shards. ● Each shard manages a number of entities and creates entity actors on demand. ● And each shard has a unique ID mapping entities to a shard ID is how we control the distribution.
  9. 9. Shard Region ● Shards gets distribute into different shard regions. Each shard region contains a number of shards. ● For a type of entity, there is usually one shard region per JVM. ● A shard region will look up the location of the shard for the entity the first time when it doesn’t already know its location, and then forwards the messages to the appropriate node region, and the entity.
  10. 10. Shard Coordinator ● The shard coordinator is responsible to manage shards, it’s a cluster singleton. ● It’s responsible for ensuring that the system knows where to send messages addressed to a specific entity. ● And it decides which shard gets to live in which region, which is to stay on which node.
  11. 11. ● To uniquely identify each entity, entityIDs are used. ● They are used to create name of the actor and hence must be unique across the entire cluster. ● Entity Id Extractors are used to process each incoming message and separate it into an entity id and a message to be passed to the entity actor. case class MyMessage(entityID: String, message: String) val idExtractor: ExtractorEntityId = { case MyMessage(id, message) => (id, message) }
  12. 12. ● To identify shards, Shard Ids are used. ● Entities are mapped to a Shard Id. ● An Extractor function is used to process each incoming message and produce Shard Id. ● Best practice is to aim for roughly 10 shards per node. ● When selecting a ShardId and producing an extractor it is important to consider how the Shards will be balanced. ● Poor sharding strategy will produce hotspots which result in uneven workload. val shardIdExtractor: ShardRegion.ExtractShardId = { case MyMessage(id, _) => (Math.abs(id.hashCode % totalShards)).toString }
  13. 13. ● ClusterSharding.start is called on each node that will be hosting shards. ● The role of the above block of code is to provide an actor ref which is the reference for the local shard region. ● For sending messages we have to take the shard region actor ref and we send it the message we’re expecting. ● Messages are first sent to the entities, through the local shard region. val shards = ClusterSharding(myActorSystem).start( “shardedActors”, MyShardedActor.props(), ClusterShardingSettings(myActorSystem), idExtractor, shardIdExtractor ) shards ! MyMessage(entityId, someMessage)
  14. 14. ● What would happen if a node fails ? ● All the entities on that node will become unavailable.
  15. 15. ● The size of the cluster changes due to failure, deployment, scaling. ● This is when rebalancing come into picture. ● The shard coordinator initiates the rebalancing process. ● Rebalancing can occur only on a healthy cluster.
  16. 16. ● The coordinator informs regions that a rebalance has started. ● Messages to an entity on a moving shard are buffered. ● Once the rebalancing is done, the messages will be delivered.
  17. 17. ● The shard allocation and rebalancing is controlled by Shard Allocation Strategy. ● LeastShardAllocationStartegy is the default implementation. ● rebalance-threshold must be exceeded before it will rebalance. ● max-simultaneous-rebalance it will rebalance only this number of shards at a time. least-shard-allocation-strategy { rebalance-threshold = 1 max-simultaneous-rebalance = 3 }
  18. 18. ● Akka cluster sharding provides Passivation as a way to remove idle actors from memory. ● It works on configurable time span. ● Each actor tracks the last time it processed any message. ● If it hasn't processed any message since the configured time period, the actor will passivate from the memory.
  19. 19. ● Once the actor is removed from memory, its state is also lost. ● The state of the actor will be buffered in database. ● This helps in recovery of the state of the actor in times when the actor starts again.
  20. 20. ● This configuration allows us to configure when the entities will passivate. ● Generally we want actors to stay in-memory as long as they are active. ● We can also manually passivate an actor. ● Sending the parent actor Passivate message will start the passivation process. akka.cluster.sharding.passivate-idle-entity-after = 120 seconds import ShardRegion.Passivate context.parent ! Passivate(stopMessage = Stop)
  21. 21. References Introduction to Akka Cluster Sharding