Modernizing your database with SQL Server 2019

1. Modernizing Your Database
with SQL Server 2019
Dec 17, 2019
Athens, Greece

3. Antonios Chatzipavlis
Data Solutions Consultant & Trainer
60+Certifications
6.0
SQL Server
Since 1999
30+Years in a Field Founder

4. A community for Greek professionals who use the Microsoft Data Platform
Connect / Explore / Learn
@antoniosch - @sqlschool
./sqlschoolgr -
./groups/sqlschool
yt/c/SqlschoolGr
SQLschool.gr Group
help@sqlschool.gr
Join us
Articles
SQL Server in Greek
SQL Nights
Webcasts
SQL Server News
Resources

5. Explore
everything
PASS has
to offer Get involved
Free online
webinar
events
Local user groups
around the world
Free 1-day local
training events
Online special
interest user
groups
Business analytics
training

6. Slide TitleSQLServer1.0-Filipi
SQLServer1.1-Pietro
SQLServer4.2a
SQLServer4.2BSQLServer4.21-SQLNT
SQLServer6.0–SQL95
SQLServer6.5–Hydra
SQLServer7.0-Sphinx
SQLServer7.0OLAP-Plato
SQLServer2000-Shiloh
SQLServer2000x64-Liberty
SQLServer2005-Yukon
SQLServer2008-Katmai
SQLServer2008R2-Kilimanjaro
SQLServer2012-Denali
SQLServer2014–SQL14
SQLServer2016–SQL16
SQLServer2017–Helsinki
SQLServer2019–Seattle
SQL Server Evolution
My experience with SQL Server

7. This is not your Grandpa’s
SQL Server
(by Travis Wright)

8. This is not your Grandpa’s
SQL Server
(by Travis Wright)

9. Slide Title
• Database Engine
• Analysis Services
• Reporting Services
• Integration Services
• Master Data Services
• Data Quality Services
• Machine Learning Services (In-database)
• Machine Learning Services (Standalone)
SQL Server 2019 Components

10. Slide TitleSQL Server Editions Overview
Edition Overview
Enterprise Fully featured edition of SQL Server for Enterprise systems and critical apps
Standard Core database and BI functionality, aimed at departmental level systems and noncritical apps
Web Is only available for service providers hosting public web sites that use SQL Server
Developer
A fully featured edition, to the level of Enterprise edition, but meant for development use
and not allowed for use on production systems
Express
A free, entry-level version of SQL Server geared toward small applications with local data
requirements

11. Slide Title
• Database Engine http://bit.ly/sql2019features
• Analysis Services http://bit.ly/AS2019Features
• Reporting Services http://bit.ly/Rs2019Features
• Integration Services http://bit.ly/IS2019Features
• Master Data Services and Data Quality Services
http://bit.ly/MDS2019Features
Editions and supported features

12. Slide TitleEditions and License Models
Edition License Models Comments
Enterprise Per-core
Standard
Per-core
Server + CAL
Web Third-part hosting only
Developer Free for noncommercial use
Express Free edition of SQL Server Limited functionality and small capacity limits

13. Slide Title
• CPU types
• Memory
• Storage
Hardware Considerations

14. Slide Title
• Setting the Power Plan
• Optimizing for Background Services
• Disk Block Size
• Locking Page in Memory
• SQL Audit the Event Log
Operating System Considerations

15. Slide Title
• Windows
- Windows Server 2016 or greater
- Windows Server Core support
• Linux
- Red Hat Enterprise 7.3-7.6
- SUSE Enterprise Server V12 SP2
- Ubuntu 16.4
• Docker Container
- Kubernetes Support
Installation on Heterogeneous OS

16. Demo
Installation process
On Windows OS

17. Slide Title
• SQL Server 2019 introduces the Hybrid Buffer Pool, offering support for PMEM
(persistent memory)
• PMEM devices are supported by Windows Server 2016 and higher
• Persistent memory is in fact memory and SQL Server can access any data stored on a
persistent memory device like it is really memory.
• If you place your database data files on a persistent memory device, SQL Server can
simply access pages on the data file from this device without having to copy data from
the data file into a buffer pool page.
• Performance results vary on the benefits of using hybrid buffer pool, but you can
typically expect some boost from this technology, especially on read-heavy workloads.
Hybrid Buffer Pool
ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED HYBRID_BUFFER_POOL = ON;
ALTER DATABASE <dbname> SET MEMORY_OPTIMIZED = ON;

18. Slide Title
• Consider the number of files that you will require
• The current, general recommendation is that you should have one
TempDB file for every core available to the instance, with a minimum of
two files and a maximum of eight files.
• You should only add more than eight files if you specifically witness
GAM/SGAM contention.
• This will manifest itself as PAGELATCH waits occurring against TempDB.
Optimizing TempDB database

19. Slide Title
• Tempdb metadata does not use memory-optimized tables by default when you install SQL Server
• What is the catch?
- Error 41317 “A user transaction that accesses memory optimized tables or natively compiled modules cannot access more than
one user database or databases model and msdb, and it cannot write to master” when you have transactions that involve in-
memory tables in user databases
- Columnstore indexes are not supported on #temp tables when this feature is enabled (error 11442).
- CREATE TABLE #t (c1 int, INDEX CCI_T CLUSTERED COLUMNSTORE );
- A related issue is that sys.sp_estimate_data_compression_savings can't be used to estimate COLUMNSTORE or
COLUMNSTORE_ARCHIVE
- Locking hints against any system views that reference these system tables will be ignored; effectively, they will use READ
COMMITTED SNAPSHOT.
- This feature requires a service restart to take effect; you can't just decide in the middle of a business cycle that you're going to turn
it on (or off).
Memory-Optimized TempDB Metadata in SQL 2019
ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED TEMPDB_METADATA = ON

20. Demo
Memory-Optimized TempDB

21. Slide TitleSQL Server 2019 Intelligent Performance
Adaptive QP
Adaptive Joins
Batch Mode
Interleaved
Execution
Memory
Grant
Feedback
Batch Mode Row Mode
Table Variable
Deferred
Compilation
Batch Mode
on Row store
Scalar UDF
Inlining
Approximate
QP
Approximate
Count Distinct
• You can enable the capabilities of Intelligent Query Processing by changing the database compatibility level of the database to 150.
• Approximate Count Distinct is a T-SQL function that is new to SQL Server 2019 and does not require a database compatibility level of 150.

22. Slide Title
• What is a memory grant?
• Two types of problems can occur:
- The memory grant can be too small for what is really needed, resulting in the infamous
and painful “tempdb spill.”
- The memory grant is too large for what is really needed, resulting a wait type
called RESOURCE_SEMAPHORE.
• Memory grant feedback solves this problem by storing information in the
cached query plan for what the correct memory grant should be for
future executions.
Memory Grant Feedback Row Mode

23. Demo
Memory Grant Feedback
Row Mode

24. Slide Title
• The famous problem of cardinality estimation for table variables, that the
cardinality estimation by the SQL Server optimizer is always one row, no
matter how many rows are populated into the table variable.
https://blogs.msdn.microsoft.com/psssql/2014/08/11/having-performance-issues-with-table-variables-sql-server-2012-sp2-can-help/
• Table variable deferred compilation improves plan quality and overall
performance for queries referencing table variables because during
optimization and initial plan compilation, this feature will propagate
cardinality estimates that are based on actual table variable row counts
Table Variable Deferred Compilation

25. Demo
Table Variable Deferred
Compilation

26. Slide Title
• Any table or index that is not organized with a
columnstore index is a rowstore.
• A batch is a structure of 64 KBs, allocated for a bunch of
rows, that contains column vectors and qualifying rows
vector.
• Depending on the number of columns it may contain
from 64 to 900 rows.
• The main advantages of Batch Mode are:
- Algorithms optimized for the multi-core modern CPUs;
- Better CPU cache utilization and increased memory throughput;
- Reduced number of CPU instructions per processed row.
Batch Mode on Rowstore
http://www.queryprocessor.com/batch-mode-on-row-store/

27. Demo
Batch Mode on Rowstore

28. Slide Title
• Scalar UDF inlining automatically transforms scalar UDFs into relational
expressions.
• It embeds them in the calling SQL query.
• This transformation improves the performance of workloads that take
advantage of scalar UDFs.
• Scalar UDF inlining facilitates cost-based optimization of operations
inside UDFs.
• The results are efficient, set-oriented, and parallel instead of inefficient,
iterative, serial execution plans.
• This feature is enabled by default under database compatibility level 150.
Scalar UDF Inlining

29. Demo
Scalar UDF Inlining

30. Slide Title
• There are scenarios where you need to count the number of rows in any table.
- SELECT COUNT(∗) FROM <table>
• But there are also situations where you need to know the number of distinct values of
a column across all rows of a table.
- SELECT COUNT(DISTINCT <col>) FROM <table>.
• The only problem is how the query processor must do work to figure out what are all
the distinct values.
- This often requires the use of a Hash Match operator.
• APPROX_COUNT_DISTINCT() uses a concept called HyperLogLog
(https://en.wikipedia.org/wiki/HyperLogLog)
- Using an approximation of the count of distinct values comes with a 2% error rate on a 97% probability.
Approximate Count Distinct

31. Demo
Approximate Count Distinct

32. Slide Title
• To solve a complex performance problem, you need details of the query plan.
• What is the problem?
You can see what is running, but you can’t dive deep into a query plan for an active query.
• Lightweight profiling is enabled by default on SQL Server 2019
• Lightweight profiling can be disabled at the database level using the
- ALTER DATABASE SCOPED CONFIGURATION SET LIGHTWEIGHT_QUERY_PROFILING = OFF;.
• A new DMF sys.dm_exec_query_plan_stats is introduced to return the equivalent of the last
known actual execution plan for most queries, and is called last query plan statistics.
• The last query plan statistics can be enabled at the database level using the
- ALTER DATABASE SCOPED CONFIGURATION SET LAST_QUERY_PLAN_STATS = ON;
Lightweight Query Profiling

33. Demo
Lightweight Query Profiling

34. Slide Title
• It improves throughput for high-concurrency inserts into an ascending
clustered index.
• By adding OPTIMIZE_FOR_SEQUENTIAL_KEY option to your index or
primary key constraint, you are telling SQL Server to enable new code to
try and avoid the convoy problem.
• This option doesn’t eliminate latches or prevent a latch contention
problem. What it does is try to avoid the dreaded convoy problem so
that your workload throughput is consistent.
Last-Page Insert Contention

35. Slide Title
Do you know what one of the most voted customer
feedback requests of all time with SQL Server is?
String or binary data would be truncated
Verbose truncation warnings

36. Demo
Verbose truncation warnings

37. Slide TitleAlways Encrypted in SQL Server 2016

38. Slide TitleAlways Encrypted with Secure Enclaves

39. Slide TitleData Classification
Prior SQL Server 2019 SQL Server 2019
The idea of the tool was to analyze column names in your database
and make recommendations on how to classify columns via
a label and an information_type.
The information_type used to tell you what kind of data exists in the
column (e.g., Contact Info, Name, Financial)
The label could be used to classify the sensitivity of the data stored
in that column (Confidential, Confidential-GDPR, HIPAA, etc.).
The tool was nice, but there were two limitations:
1. The tool used a concept in SQL Server called extended
properties.
2. There is no built-in auditing for access to the columns that are
marked for classification.
New set of T-SQL statements, catalog views, and auditing.
ADD SENSITIVITY CLASSIFICATION
DROP SENSITIVITY CLASSIFICATION
These T-SQL statements result in metadata stored directly into
system tables (exposed by catalog views) that are specific to labels
and information_types associated with columns in a table.
A new catalog view is supported to view this metadata
called sys.sensitivity_classifications
Furthermore, SQL Server Auditing now supports a new property
called data_sensitivity_information which can be used to audit
who, what, and when users are trying to view classified data.

40. Demo
Data Classification

41. Slide Title
• Transparent Data Encryption (TDE) is all about data encryption at rest.
• Available at Enterprise and now in Standard edition
• SQL Server 2019 introduces the concept of pause and resume for TDE encryptions.
• sys.dm_database_encryption_keys has three new columns to see the state of the TDE scan:
- encryption_scan_state – A number indicating if the TDE scan is in progress, suspended, or completed
- encryption_scan_state_desc – A string description of the scan state such as RUNNING, SUSPENDED, COMPLETE
- encryption_scan_modify_date – A date/time for the last time the scan state changed
TDE Pause and Resume
ALTER DATABASE <db_name> SET ENCRYPTION SUSPEND/RESUME

42. Slide TitleCertificate Management

43. Slide Title
SQL Server 2017 introduces the concept of resumable index rebuild
operation.
SQL Server 2019 introduces the concepts
• of resumable indexes when creating the index with CREATE INDEX.
• of a default database scoped setting for online and resumable index
operations.
These new options are called ELEVATE_ONLINE and ELEVATE_RESUMABLE.
Resumable Index Operations

44. Demo
Resumable Index Operations

45. Slide Title
• SQL Server 2017 introduced the ability to build and rebuild nonclustered
columnstore indexes online.
• SQL Server 2019, clustered columnstore indexes can now be built and
rebuilt online.
• The syntax to rebuild a clustered columnstore index online will be the
same as with a standard index using the ALTER INDEX syntax
• Resumable indexes are not yet supported for online clustered or
nonclustered columnstore indexes.
Online Index Maintenance for Columnstore

46. Slide Title
• Support for More Synchronous Replicas
- Support up to five synchronous replicas in an Availability Group, and a total of nine
overall replicas.
• Secondary to Primary Replica Read/Write Connection Redirection
- A new capability to solve the challenge for your application to always be directed to the
primary replica no matter what SQL instance is hosting the primary replica for the
Availability Group.
https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/secondary-replica-connection-redirection-always-on-availability-groups?view=sql-server-ver15
Enhancing Always On Availability Groups

47. Slide Title
Phase 1: Analysis Determines the state of each transaction in the
system at the time SQL Server stopped.
Phase 2: Redo
Returns the database to the state it was in at the
time the SQL Server stopped.
Phase 3: Undo
Rolls back any uncommitted transaction(s)
individually.
Log
Start
Oldest
uncommitted Tx
(XACT_BEGIN_LSN)
Checkpoint
(or oldest dirty page LSN)
Log
End
Recovery Phase / Transaction Log
[DB is PARTIALLY available after Redo]
[DB is FULLY available after Undo]
Current SQL Server Database Recovery process

48. Slide Title
Checkpoint
(or oldest dirty page LSN)
Log
End
Phase 1: Analysis
Regular Analysis + Reconstructs sLog
Phase 2b: Redo Phase 1: Redo from sLog
Phase 2: Redo from Transaction Log
Phase 2a: Redo from sLog
Phase 3: Undo from sLog Instant Undo by using sLog instead of Transaction
Log
Transaction Log
sLog (in memory)
Log Record for non-versioned operation
Database Recovery (with ADR)
sLog Records
[DB is FULLY available]
Log
Start
Oldest
uncommitted Tx
(XACT_BEGIN_LSN)
Recovery Phase / Transaction Log / sLog
Accelerated Database Recovery

49. Demo
Accelerated Database
Recovery

50. Slide TitleLanguages and Drivers
http://aka.ms/sqldev

51. Slide Title
• Traversing a graph path using the new SHORTEST_PATH() syntax
• Support for derived tables and views in a graph database
• Edge constraints to enforce proper graph relationships
• Using the T-SQL MERGE statement with edge tables
- In SQL Server 2017, you can use the MERGE statement to consolidate DML operations on
node tables, but not on edge tables.
Graph Enhancements for SQL Server 2019

52. Slide Title
• Platform and deployment enhancements
• I/O Accelerated Performance with Persistent Memory Support
• SQL Server Replication
• Change Data Capture (CDC)
• Distributed Transactions are supported
• Simplified Active Directory deployment using OpenLDAP providers.
• Support SQL Server Machine Learning Services and Extensibility
• Bring Data Virtualization to SQL Server on Linux by supporting Polybase queries to external data sources,
with no data movement, such as Hadoop, SQL Server, Oracle, Teradata, and MongoDB.
What Is New for SQL Server 2019 on Linux

53. Slide Title
• A distributed computing engine
• Query data where it lives with T-
SQL
• Distributed, scalable query
performance
• Manual/deploy with SQL Server
• Auto deploy/optimize with Big
Data Clusters
Data Virtualization / SQL Server Polybase

54. Slide TitleHow Polybase Works

55. Slide TitleBig Data Cluster Architecture

56. Slide Title
• Reduced compilations for temporary tables
• Indirect checkpoint scalability
• Concurrent PFS updates
• Worker stealing
• Diagnostics on statistics
• Query Store enhancements
• Plan cache granular control
• DBCC CLONEDATABASE enhancements
Performance Enhancements

57. Slide Title
• Calculation groups in tabular models
• Query interleaving
• Many-to-many relationships in tabular models
• Property settings for resource governance
• Governance setting for Power BI cache refreshes
New features - SQL Server 2019 Analysis Services

58. Slide Title
• Flexible file task
• Flexible file source and destination
New features - SQL Server Integration Services

59. Slide Title
• Azure SQL Managed Instance support
• Power BI Premium dataset support
• AltText (alternative text) support for report elements
• Azure Active Directory Application Proxy support
• Custom headers
• Transparent Database Encryption
• Microsoft Report Builder update
New features - SQL Server 2019 Reporting Services

60. Slide Title
• Support for Azure SQL Database managed instance databases
• New HTML controls
New features - SQL Server Master Data Services

61. Slide Title
• In-place upgrade
• Restore a database
• SSIS - Bulk Import/Export
• Live Migration
- Rolling upgrade
- Log Shipping
- Replication
Upgrading to SQL Server 2019

62. Slide TitleMigration Tools
Data Migration
Assistant
Readiness assessment: blocking
issues breaking changes, behavior
changes
Moves schema, data and
uncontained objects (like logins) To
Azure SQL Database
Backup / Restore to another SQL
Server (Keeps source DB
Compatibility Level)
New feature recommendation
Database
Experimentation
Assistant
A/B Testing
Capture and Replay workload for
performance testing and reporting
Also reports on migration blockers
because of failed T-SQL syntax
Query Tuning
Assistant
Upgrade Database Compatibility
Model to desired state
Detects workload regressions, and
tests CE model variations (subsets)
Provides tangible
recommendations for tuning
queries without reverting DB
compat

63. Slide Title
• Database Compatibility Level sets certain database
behaviors to be compatible with the specified version of
SQL Server.
• Compatibility level affects behaviors only for the
specified database, not for the entire server.
• Deprecated functionality is protected by database
compatibility level.
• Discontinued functionality means we have removed a
feature or behavior from a release of SQL Server, so
database compatibility will not help.
• Breaking changes is defined as behavior changes that
can result in a different outcome.
- In some cases, a breaking change is protected by using a
database compatibility level on a new version of SQL Server.
- In other cases, a breaking change is not protected by database
compatibility level.
Database Compatibility
Product
Compatibility
Level
Supported
Compatibility
SQL Server 2019 150 150, 140, 130, 120, 110, 100
SQL Server 2017 140 140, 130, 120, 110, 100
Azure SQL
Database
130 150, 140, 130, 120, 110, 100
SQL Server 2016 130 130, 120, 110, 100
SQL Server 2014 120 120, 110, 100
SQL Server 2012 110 110, 100, 90
SQL Server 2008
R2
100 100, 90, 80
SQL Server 2008 100 100, 90, 80
SQL Server 2005 90 90, 80
SQL Server 2000 80 80

64. Thank you!
@antoniosch - @sqlschool
./sqlschoolgr - ./groups/sqlschool
yt/c/SqlschoolGr
SQLschool.gr Group
Antonios Chatzipavlis
Data Solutions Consultant & Trainer

65. A community for Greek professionals who use the Microsoft Data Platform
Copyright © 2019 SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS