New for 3.0, Materialized Views deliver automated server-side denormalization. This is the presentation given at Cassandra Summit 2015.

1. Materialized Views
Carl Yeksigian

2. What are Materialized Views?
• Two copies of the data using different partitioning and placed
on different replicas
• Automated, server-side denormalization of data
• Native Cassandra read performance
• Write penalty, but acceptable performance

3. Basic Rules of Data Modeling, Refresher
• Best practice: Denormalization
• Start by understanding the queries you need
• Create a table for each query

4. Why is Denormalization Hard?
• Implemented by every application
• No guarantees on performance or consistency
• Updates to existing rows require cleanup, read-before-write

5. Denormalization Example: User Playlists
Queries
• All Songs for a given
playlist
• Track Users who like
the same Artist
• Find most recently
played song

6. Denormalization in Practice
CREATE TABLE playlists
(user_name text,
playlist_name text,
song_id text,
artist_name text,
last_played timestamp)
SELECT song_id
FROM playlists
WHERE user_name=‘carl’
AND playlist_name=‘jams’
SELECT COUNT(song_id)
FROM playlists
WHERE artist_name=‘Weezer’
SELECT last_played, song_id
FROM playlists
WHERE user_name=‘carl’
AND playlist_name=‘jams’
ORDER BY last_played DESC

7. Denormalization in Practice
CREATE TABLE playlists
(user_name text,
playlist_name text,
song_id text,
artist_name text,
last_played timestamp,
PRIMARY KEY (user_name, playlist_name, song_id))
CREATE TABLE artists_to_playlists
(artist_name text,
user_name text,
playlist_name text,
song_id text,
PRIMARY KEY (artist_name, user_name, playlist_name, song_id))

8. Denormalization in Practice
CREATE TABLE last_played
(user_name text,
playlist_name text,
last_played timestamp,
song_id text,
PRIMARY KEY (user_name, playlist_name, last_played, song_id))
CLUSTERING ORDER BY (last_played DESC)

9. Denormalization in Practice: Inserts
BEGIN BATCH
INSERT INTO playlists
(user_name, playlist_name, song_id, artist_name, last_played)
VALUES (‘carl’, ‘jams’, ‘Undone’, ‘Weezer’, ‘2015-09-24 09:00’);
INSERT INTO artists_by_playlist
(artist_name, user_name, playlist_name, song_id)
VALUES (‘Weezer’, ‘carl’, ‘jams’, ‘Undone’);
INSERT INTO last_played
(user_name, playlist_name, last_played, song_id)
VALUES (‘carl’, ‘jams’, ‘Undone’, ‘2015-09-24 09:00’);
APPLY BATCH;

10. Denormalization in Concept: Updates
UPDATE playlists
SET last_updated=now()
WHERE user_name=‘carl’
AND playlist_name=‘jams’
AND song_id=‘Undone’
DELETE FROM playlists
WHERE user=‘carl’

11. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator

12. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator
Query Existing Data

13. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator
Query Existing Data
Query Existing Data

14. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator
Query Existing Data
Query Existing Data Return Existing Data
Return Existing Data

15. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator
Write New Values

16. Manual Denormalization with updates
Client Batchlog
Base Table
View Table
Coordinator
Write New Values Write New Values

17. Manual Denormalization with updates
Client Batchlog
View Table
Coordinator
Write New Values Write New Values
Base Table
Write New Values

18. Manual Denormalization with updates

19. Manual Denormalization Limitations
• Updates need to keep view in sync, including tombstoning
previous values
• How to keep the view and base in sync on failure?
• Contentious updates can potentially cause extra values
• Your application doesn’t always know what is a update or an
insert (i.e. upsert)

20. Manual Denormalization: Contentious Updates
Client 1
Cassandra
Client 2
playlists:(‘carl’, ‘jams’, ‘Undone’, 2015-09-24 9:00)
last_played:(‘carl’, ‘jams’, 2015-09-24 9:00, ‘Undone’)
Query existing last_played
Query existing last_played

21. Manual Denormalization: Contentious Updates
Client 1
Cassandra
Client 2
playlists:(‘carl’, ‘jams’, ‘Undone’, 2015-09-24 9:02)
last_played:(‘carl’, ‘jams’, 2015-09-24 9:01, ‘Undone’)
last_played:(‘carl’, ‘jams’, 2015-09-24 9:02, ‘Undone’)
Update last played time to 9:02
last_played: 2015-09-24 9:00
last_played: 2015-09-24 9:00
Update last played time to 9:01

22. Manual Denormalization Limitations

23. Materialized Views
• Provide automated server-side denormalization
• No need for read-before-write on the application side
• Simplify application code
• Provide safer guarantees

24. Materialized Views: Guarantees
• If a write is acknowledged, at least CL number of base and
view replicas will receive the write
• If a write is actually an update, the previous value will be
cleaned up in the view
• Even with contentious updates, view synchronized with base
for each update
• Takes care of deletions properly
• When a base table is repaired, the data will also be inserted
into the view
• TTL’d base data will remain TTL’d in view

25. Why Not Just Use Secondary Indexes?
• We can get most of the same functionality by using
secondary indexes
• Secondary indexes query each node, not being able to use
the ring
• On a node, not a single access

26. Secondary Indexes: Query Pattern
Client

27. Secondary Indexes: Query Pattern
Client

28. Secondary Indexes: Query Pattern
Client

29. Materialized Views: Query Pattern
Client

30. Materialized Views in Practice
CREATE TABLE playlists
(user_name text,
playlist_name text,
song_id text,
artist_name text,
last_played timestamp,
PRIMARY KEY (user_name, playlist_name, song_id))

31. Materialized Views in Practice
CREATE MATERIALIZED VIEW artist_to_user AS
SELECT song_id, user_name
FROM playlists
WHERE song_id IS NOT NULL
AND playlist_name IS NOT NULL
AND user_name IS NOT NULL
AND artist_name IS NOT NULL
PRIMARY KEY (artist_name, user_name, playlist_name, song_id)

32. Replica Placement
user_name:carl

33. Replica Placement
artist_name:Weezer
user_name:carl

34. Materialized Views in Practice
• On creation, a new materialized view will be populated with
existing base data
• Each node tracks the completion of the build independently
SELECT *
FROM system.built_views
WHERE keyspace=‘ks’
AND view_name=‘view’

35. Materialized Views in Practice
CREATE MATERIALIZED VIEW last_played AS
SELECT last_played, song_id
FROM playlists
WHERE user_name IS NOT NULL
AND last_played IS NOT NULL
AND song_id IS NOT NULL
PRIMARY KEY (user_name, playlist_name, last_played, song_id)
CLUSTERING ORDER BY (last_played DESC)

36. Materialized Views: Performance
• https://github.com/tjake/mvbench
• Uses java-driver to simulate MV and manual denormalization

37. Materialized Views: Performance (ops/s)
Manual Denormalization
Materialized Views

38. Materialized Views: Performance (p95 latency)
Manual Denormalization
Materialized Views

39. • Adding WHERE clause support (#9664)
CREATE MATERIALIZED VIEW carls_last_played AS
SELECT last_played, song_id
FROM plays
WHERE user_name=‘carl’
PRIMARY KEY (last_played, song_id)
• Knowing when a view is completely finished building without
querying each node (#9967)
• Insert only tables can skip read-before-write, lock acquisition
(#9779)
Materialized Views: Future Features

40. write<p1, p2, c1, c2, v1>
Node A Node B Node C
Coordinator
Base
View
Node D
write<p1, p2, c1, c2, v1>
del<v0, p1, p2, c1, c2>
Client
write<v1, p1, p2, c1, c2>
BL

41. Materialized Views: Under the Hood
• If update is partial, we will reinsert data from the read when
generating a new row
• If no tombstone generated, only new columns are written
to view

42. Materialized Views: The Edge
• Materialized Views have different failure properties than the
rest of the system
• Data from a single base replica can be on many view replicas

43. Materialized Views: The Edge
• When data is lost on all replicas of the base table, it can not
be cleaned up in the view (#10346)
• No read repair between the base and the view table
• Repair of base table will clean up view
• Requires local read-before-write.
• If you will never ever update/delete use manual MVs

44. Materialized Views: The Edge
• An update from a base table is asynchronously applied to the
view, so it is possible there will be a delay
• A MV on a low-cardinality table can cause hotspots in the
ring, overloading some nodes

45. Thanks