2. About.Me()
• Veritabanı Yöneticisi / Doğan Online
• Eğitimen/Danışman / Btakademi
• Kitap Yazarı
• Microsoft Certified Trainer
ismailadar@hotmail.com
http://www.ismailadar.com
İsmail
Adar
@ismailadars
3. Content.GetAgenda()
• SQL Server 2014 In-Memory OLTP Basics
• Create Database For In-Memory OLTP
• Creating Memory Optimized Tables and Indexes
• Accessing Memory Optimized Tables
• Memory Optimized Tables Limitations
• Memory management on memory-optimized tables
• Tools for analysis and migration
9. Create Table DDL
Hash Index
CREATE TABLE [Customer](
BUCKET_COUNT 1-2X nr of
unique index key values
[CustomerID] INT NOT NULL
PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000),
[Name] NVARCHAR(250) NOT NULL,
[CreatedDate] DATETIME NULL
INDEX [IX_CreatedDate] NONCLUSTERED
)
Indexes are specified inline
NONCLUSTERED indexes are
supported
WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);
This table is memory
optimized
This table is durable
10. Memory Optimized Table Creation
CREATE TABLE DDL
Code generation and compilation
Table DLL produced
Table DLL loaded
12. Memory-Optimized Indexes
•
Exist only in memory
– Rebuilt on database startup
•
Do not contain data rows
– Indexes contain memory pointers to the data rows
– No duplication of data
– All indexes are covering
13. Memory-optimized Table: Row
Format
Row header
Payload (table columns)
8 bytes * (IdxLinkCount)
Begin Ts
End Ts
StmtId
IdxLinkCount
8 bytes
8 bytes
4 bytes
2 bytes
14. Memory Optimized Tables and Hash Indexes
Timestamps
Hash index on
Name
50, ∞
Chain ptrs
Name
City
Hash index on
City
Jane
Prague
100, ∞
John
Prague
90, ∞
90, 150
Susan
Bogota
15. Nonclustered (range) index
•
No latch for page updates
•
No in-place updates on index pages
•
1
PAGE
Physical
Page size- up to 8K. Sized to the row
•
Page Mapping Table
0
Handled thru delta pages or building new pages
Sibling pages linked one direction
Root
PAGE
2
1
0
3
2
0
•
2
8
Require two indexes for ASC/DSC
No covering columns (only the key is stored)
Page-pid-0
PageID-3
4
8
1
0
1
1
1
5
1
8
2
1
2
4
2
7
Non-leaf pages
Logical
1
14
15
2
200,
4
∞
5
1
Key
6
2
5
7
50, 300
2
Key
2
6
2
7
Data rows
leaf pages
18. Create Procedure DDL
CREATE PROCEDURE [dbo].[InsertOrder] @id INT, @date DATETIME
WITH
This proc is natively
compiled
NATIVE_COMPILATION,
SCHEMABINDING,
Native procs must be
schema-bound
EXECUTE AS OWNER
Execution context is
required
AS
BEGIN ATOMIC
WITH
Atomic blocks
• Create a transaction if
there is none
• Otherwise, create a
savepoint
(TRANSACTION
ISOLATION LEVEL = SNAPSHOT,
LANGUAGE = N'us_english')
-- insert T-SQL here
END
Session settings are fixed at
create time
19. Procedure Creation
CREATE PROC DDL
Query optimization
Code generation and compilation
Procedure DLL produced
Procedure DLL loaded
20. Accessing Memory Optimized Tables
Interpreted T-SQL Access
• Access both memory- and disk-based
tables
• Less performance
Natively Compiled Stored
Procs
• Access only memory optimized tables
• Maximum performance
• Limited T-SQL surface area
• Virtually full T-SQL surface
• When to use
– Ad hoc queries
– Reporting-style queries
– Speeding up app migration
• When to use
– OLTP-style operations
– Optimize performance critical business logic
24. Limitations on In-Memory OLTP in SQL 2014
• Tables
–
–
–
–
–
–
Triggers: no DDL/DML triggers
No Identity columns
Data types: no LOBs, no XML and no CLR data types
Constraints: no FOREIGN KEY and no CHECK constraints
No schema changes (ALTER TABLE) – need to drop/recreate table
No add/remove index – need to drop/recreate table
• Natively Compiled Stored Procedures
– No outer join, no OR, no subqueries, no CASE
– Limited built-in functions [core math, date/time, and string functions are
available]
25.
26. Memory Management For In-Memory OLTP
Data resides in memory at all times
• Must configure SQL server with sufficient memory to store memoryoptimized tables
• Failure to allocate memory will fail transactional workload at run-time
• Integrated with SQL Server memory manager and reacts to memory
pressure for GC (Garbage Collection)
27. Memory Management For In-Memory
OLTP
Integrated with Resource Governor
• To ensure performance stability for disk-based table workloads
• “Bind” a database to a resource pool
• Mem-opt tables in a database cannot exceed the limit of the resource
pool
• Hard top limit (80% of phys. memory) to ensure system remains stable
under memory pressure
28. Memory Management For In-Memory OLTP
Memory optimized max
memory limit
CREATE RESOURCE POOL InMemResourcePool
WITH (MAX_MEMORY_PERCENT=40);
ALTER RESOURCE GOVERNOR RECONFIGURE;
GO
EXEC sp_xtp_bind_db_resource_pool 'InMemoryDemoDB','InMemResourcePool';
GO
ALTER DATABASE InMemoryDemoDB SET OFFLINE;
ALTER DATABASE InMemoryDemoDB SET ONLINE;
Filegroup for memory optimized
table storage
Talking points:* Main target scenario is high throughput OLTP; other scenarios include ETL, read scale-up (many concurrent transactions reading same data), and burst insert/update (i.e., batches of updates, interfering with read workload)Memory optimized data structuresNovel data structures – no Btree and no PINTABLEUse new structures such as hash indexesTables reside in memoryIndexes exist only in memoryTables are still fully durable – there is a copy of the data on disk for crash recovery purposesNative compilationT-SQL Stored procedures are compiled to native code as DLLsAllows more efficient query execution by reducing the CPU cycles required for each operationLatch- and lock-free data structuresTable structures are multi-versioned. This is built right into the data structures, no tempdb usageData structures are latch free – row updates are realized by inserting new row versionsNo locks, thus no blocking. Transactional consistency is guaranteed through multi-versioning and conflict detectionIntegration in SQL ServerUse the same familiar T-SQL syntax for creating memory optimized tables and natively compiled stored procedures
Talking points:The code page of a (var)char value is determined by the collation associated with the value. For example, the collation SQL_Latin1_General_CP1_CI_AS has the associated code page 1252.-- all supported collations for (var)char columns in memory-optimized tables select * from sys.fn_helpcollations() where collationproperty(name, 'codepage') = 1252;Indexes can only be created on string columns if they use a BIN2 collation. The LastName variable uses BIN2 collation. FirstName uses the database default, which is CI_AS (case-insensitive, accent-sensitive).select * from sys.fn_helpcollations() where name like '%BIN2'
Talking points:* Hash indexes are good for point lookups; need bucket countRange indexes are in the plan; you can think of a range index as a kind of nonclustered index, except that it is memory optimized and latch-free – no bucket-count requiredLike hash indexes, range indexes will be inherently coveringAlter table is not pertimmited
Talking points:This diagram shows what happens when a table is createdTables are compiled for efficient access to table data – knowledge about indexes, columns, datatypes, etc, is baked into the DLLDLL is loaded immediately after compilationDLL is recompiled at server startup, so no risk of anyone tampering with the DLLs while the server is down
Talking points:* Hash indexes are good for point lookups; need bucket countRange indexes are in the plan; you can think of a range index as a kind of nonclustered index, except that it is memory optimized and latch-free – no bucket-count requiredLike hash indexes, range indexes will be inherently covering
Talking points:This diagram shows what happens when a table is createdTables are compiled for more efficient access to table data – knowledge about indexes, columns, datatypes, etc, is baked into the DLLDLL is loaded immediately after compilationDLL is recompiled at server startup, so no risk of anyone tampering with the DLLs while the server is down
Starting SQL “14”, SQL Server has both memory optimized and disk-based tablesThe familiar interpreted T-SQL can be used to access bothHekaton introduces natively compiled stored procedures for efficient access to memory optimized tables, and efficient business logic executionCan access only memory optimized tablesMachine code in DLLs can access the table and index structures directly; machine code requires fewer CPU cycles to execute the queries than the SQL Server query execution engine, interpreting the query execution plan Because most query operators and functions need to be re-implemented for native procs, we have limitations in the query surface area in native procs. For example, no OR, no subqueries, and we will have a limited number of built-in functionsWhile interpreted T-SQL is less performant, there is more flexibility:1. can use ad hoc batches2. no restrictions on T-SQL surface area3. can access both memory optimized and disk-based tables in the same queryWith interop there is still the benefit of latch-free data structuresInterop limitations – No Cursors (except for static readonly cursors, API cursors), table sample, MERGE with memory optimized table as targetYou can start with interop, and over time move to natively compiled stored procedures.
Talking points:The code page of a (var)char value is determined by the collation associated with the value. For example, the collation SQL_Latin1_General_CP1_CI_AS has the associated code page 1252.-- all supported collations for (var)char columns in memory-optimized tables select * from sys.fn_helpcollations() where collationproperty(name, 'codepage') = 1252;Indexes can only be created on string columns if they use a BIN2 collation. The LastName variable uses BIN2 collation. FirstName uses the database default, which is CI_AS (case-insensitive, accent-sensitive).select * from sys.fn_helpcollations() where name like '%BIN2'