Part 2 of a 3 part introduction to apache geode.

1. Day 2
Swapnil Bawaskar
@sbawaskar
(incubating)

2. • Clients - in depth
• Function Execution
• Serialization
• Transactions
• Data Colocation
Agenda
2

3. • Client
• A process (Your Application)
connected to the Geode server(s)
• Can maintain near cache
• Run OQL queries on local data
• Can be notified about events on the
servers
3
Concepts
Application
GemFire Server
Region
Region
RegionClient Cache

4. • Client Notifications
• Register Interest
• Individual Keys OR RegEx for Keys
• Updates Local Copy
• Examples:
• region.registerInterest(“key-1”);
• region1.registerInterestRegex(“[a-z]+“);
• Continuous Query
• Receive Notification when Query condition met
on server
• Example:
• SELECT * FROM /tradeOrder t WHERE t.price > 100.00
• Can be DURABLE
Concepts
4

5. • Functions
• Used for distributed concurrent processing
(Map/Reduce, stored procedure)
• Push code vs pull data
• Run in Server’s process
• Highly available
• Member oriented
5
Concepts
Submit (f1)
f1 , f2 , … fn
Execute
Functions

6. 6
Concepts
Server Server
FunctionService.onRegion.withFilter.execute
ResultCollector.getResult
Server Distributed System
execute
Server
Server
6
1
result
execute
execute
result
result
2
5
3
4
3 4
Server
Partitioned Region
Data Store - X
Partitioned Region
Data Store - Y
Partitioned Region
Data Store - Z
Partitioned Region
Data Accessor
Partitioned Region
Data Accessor
filter = Keys X, Y
Client Region
• Functions

7. • Listeners
• CacheWriter / CacheListener
• AsyncEventListener (queue / batch)
• Parallel or Serial
• Conflation
7
Concepts

8. Fixed or flexible schema?
id name age pet_id
or
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}

9. C#, C++, Java, JSON
No IDL, no schemas, no hand-coding
Schema evolution (Forward and Backward Compatible)
* domain object classes not required
| header | data |
| pdx | length | dsid | typeid | fields | offsets |
Portable Data eXchange

10. Efficient for queries
{
id : 1,
name : “Fred”,
age : 42,
pet : {
name : “Barney”,
type : “dino”
}
}
SELECT p.name FROM /Person p WHERE p.pet.type = “dino”
single field
deserialization

11. But how fast is it?
Benchmark: https://github.com/eishay/jvm-serializers

12. Schema evolution
Member A Member B
Distributed Type Definitions
v2v1
Application #1
Application #2
v2 objects preserve data
from missing fields
v1 objects use default values to
fill in new fields
PDX provides forwards and backwards
compatibility, no code required

13. • Persistent Regions
• Durability
• WAL for efficient writing
• Consistent recovery
• Value always one disk seek away
• Compaction
13
Concepts
Modify
k1->v5
Create
k6->v6
Create
k2->v2
Create
k4->v4
Oplog2.crf
Member
1
Modify
k4->v7Oplog3.crf
Put k4->v7
Region
Cache
java.util.Map
JVM
Key Value
K01 May
K02 Tim
Region
Cache
java.util.Map
JVM
Key Value
K01 May
K02 Tim
Server 1 Server N

14. Persistence - Shared Nothing
14
Server 3Server 2Server 1

15. Persistence - Shared Nothing
15
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary

16. Persistence - Shared Nothing
16
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary

17. Persistence - Shared Nothing
17
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary

18. Persistence - Shared Nothing
18
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary

19. Persistence - Shared Nothing
19
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary
B3
B2
Server 1 waits for others when it starts

20. Persistence - Shared Nothing
20
Server 3Server 2Server 1
B1
B3
B2
B1
B3
B2
Primary
Secondary
Fetches missed operations on restart

21. Concepts - HA
21

22. CacheTransactionManager mgr = cache.getCacheTransactionManager();
mgr.begin();
region1.put("K1", "V1");
region1.put("K2", "V2");
region2.put("K2", "V2");
mgr.commit();
Transaction - API
22
TransactionId txId = mgr.suspend();
… other non-transactional work
mgr.resume(txId);
mgr.tryResume(…);
region.putIfAbsent(K, V);
region.replace(K, V, V);
region.remove(K, V);
Single Entry

23. • Repeatable Read
• Thread sees own changes
• Other threads do not until commit is called
• Optimistic
• No Entry level locks, Readers not blocked
• Conflict Detection
• NOT Persistent (yet)
• Not a problem if you have at-least one member up
• Data must be colocated
Transaction - Semantics
23

24. • Inspiration
“For scalability, applications should
manipulate single collection of data that lives on
one JVM”
— Pat Helland (Life Beyond Distributed Transactions)
• Custom Partitioning
• Within one Partitioned Region
• E.g. All trades in January
• Data Colocation
• Between Two or more Partitioned Regions
• All Orders of a Customer
• Implement a PartitionResolver
Concepts - Data Colocation
24

25. Hands On
25

26. • Serialize Teeny using PDX
• Trigger a CQ when popularity > 5
• Implement function to find the most popular
domain
Hands On
26