Db2: Write once, run anywhere, from any source

1. Db2 family and 11.1.4.4 update
Jessica Rockwood
IBM
Monday October 1, 2018

2. Please note :
• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice
and at IBM’s sole discretion.
• Information regarding potential future products is intended to outline our general product direction and it
should not be relied on in making a purchasing decision.
• The information mentioned regarding potential future products is not a commitment, promise, or legal
obligation to deliver any material, code or functionality. Information about potential future products may not be
incorporated into any contract.
• The development, release, and timing of any future features or functionality described for our products remains
at our sole discretion.
• Performance is based on measurements and projections using standard IBM benchmarks in a controlled
environment. The actual throughput or performance that any user will experience will vary depending upon
many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O
configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that
an individual user will achieve results similar to those stated here.
2

3. Db2: Write once, run anywhere, from any source
3
Db2 Warehouse on Cloud
Db2 Warehouse
Db2 Hosted
Db2 on Cloud
IBM Integrated
Analytics SystemDb2 Big SQL
Db2 Common SQL Engine

4. Db2 Common SQL Engine = Investment protection
• The Db2 Common SQL Engine lets you leverage your existing investment in skills and
applications across the different deployments
• Application portability
• Common programming model
• Common client interfaces
• “Inclusive” compatibility: Oracle, Netezza
• Eco-system compatibility
• One ISV product certification for all platforms – test once, certify all
4

5. DB2 Common SQL Engine = Support for different workload types
• The Db2 Common SQL Engine brings with it the proven ability to support different
types of workloads
• Analytics
• OLTP
• Hybrid of Transactional and Analytic Processing (HTAP)
OR +OR
5

6. Db2 Common SQL Engine = Consistent Technical Capabilities
• All of Db2’s strong underlying capabilities are available
• High capacity
• High concurrency levels
• High performance levels
• Continuous/High availability
• Consistent operations
• Inherent security (including encryption)
• Integrated workload management
• Common monitoring interfaces
6

7. Db2: More intelligent analytics and insight
• Db2 Common SQL Engine provides the strong
foundation needed to take the next steps up the AI
ladder
• High performance, high capacity
• Integrated analytic capabilities
• Data virtualization technologies to provide a single point of
access regardless of where data resides
7
AI
Machine Learning
Analytics
Data
Db2 Common SQL Engine
The “AI Ladder”

8. Db2: Flexibility of deployment
8
On Premise
On Cloud
SOFTWARE docker APPLIANCE
PRIVATE
CLOUD
HOSTED
DBaaS
MANAGED
DBaaS
CONTROL SIMPLICITY
Db2 Warehouse
Db2 Warehouse
IBM Integrated
Analytics System Db2 Hosted
Db2 on Cloud
Db2 Warehouse on Cloud

9. Agenda
• A quick overview of Db2 11.1
• Looking at the latest:
• Highlights from Db2 11.1.2.2 and 11.1.3.3
• A look into the Future
9

10. A quick overview
10

11. Db2 11.1: A new beginning
• Starting with DB2 11.1, we have begun to move towards a “continuous delivery”
model
• Explicitly plan to deliver function throughout the active development lifetime of a version
• New versions will still happen occasionally for various reasons
• What does this mean to you?
• Introduction of Modification levels to the product signature
• Function delivered continuously during a version’s lifecycle
• No more “big gulps”, just a series of “little bites”
11

12. What is a Modification level?
• Db2 product signature consists of 4 parts: VV.RR.MM.F (Version.Release.Modification.Fix)
• Traditionally, updates for a Db2 version were fixpacks and would increment the last part
• E.g. The ninth update for Db2 10.5 was a fixpack with the signature of Db2 10.5.0.9
• These contained fixes and sometimes new function
• Starting with Db2 11.1, each update with planned function will be a new modification level
• Also decided to increase fixpack level at same time to keep reference terms familiar (e.g. “Fixpack 2” is
the same as “Mod 2”)
• Db2 11.1.1.1 was the first update provided after Db2 11.1 general availability (GA) level
• It is an update just like past fixpacks and is applied in the same way with the same expectations*
12
* Except that mod 1 was not pureScale rolling update compatible!! All subsequent 11.1 updates are!

13. V11.1.0.0 V11.1.1.1 V11.1.2.2 V11.1.4.mV11.1.4.4
Fixes +
Function
Fixes +
Function
Fixes +
Function FixesGA
V11.1.3.3
Fixes +
Function
4Q ‘16Mid ‘16 2Q ‘17 1Q ‘18 2H ‘18
Regular fix/mod pack deliveries (every 6-9 months)
! With overriding focus on stability for immediate production deployment
! Modifications will contain select functional enhancements (often off by default)
The Db2 11.1 lifecycle
13

14. 14
pureScale “for the Masses”
• Up and running in hours
• Zero data loss DR with HADR
• Multi-switch GDPC (no SPOF)
• Seamless HADR pureScale
upgrades
• Restore REBUILD
• Online Index Create
• Add/Drop CF
Increased Serviceability
• 16x increase in active log space capability
• Enhanced monitoring for lock and logging events
Additional Core Function
• Workload Manager (WLM) multi-tenancy extensions
• Federation simplification and integration
Comprehensive Enterprise Security
Mainframe Class Availability
Significant Core Database Advances
Enterprise Encryption
GA, FP1, FP2, FP3 Highlights: Mission Critical Workloads
• Centralized Key Managers (KMIP)
• PKCS#11 HSM support
• SSL Encryption for non-pS HADR (Linux x86, ALL platforms)
Even Greater Availability
• Online Crash Recovery
• Faster Rollback of Multiple and Large Txn
• ADMIN_MOVE_TABLE advances
• Better tablespace state and replay window
monitoring with HADR
• Per partition OLR
• Avoid Lock Escalation
• Timeout support for vendor Archive
Simpler, Faster, More Online Upgrades
• Faster, no need for offline backup
• Streamlined HADR upgrade
• DB2 Version 9.7 direct to 11.1
Very Large Database Performance
• Higher user throughput
• Faster MCR by default
• Improved sockets performance
• Multiple hosts in maintenance mode
• Recover HADR Standby w/ tbsp
restore (now for pS)
• Improved XA performance
HADR ease of use:
• Export HADR TSAMP configuration to
XML file

15. 15
Massive Scale Warehousing at In-Memory Performance
MPP BLU Scalability
• PB scale in-memory warehousing
Next Gen In-Memory Performance, Function and Workloads
Continued Performance Improvements
• Synopsis table enhancements
• Additional SIMD exploitation
• Aggregation enhancements, sort elimination
• INSERT from sub-select – multi-core parallelism improvements
• BLU Acceleration indexes
SQL and Compatibility
PDF manuals for DB2 LUW v11
Multi-Lingual SQL Advances
• BINARY, EXTENDED ROWSIZE
• CHAR(255)
BLU Restriction Removal
• Automatic Dictionary Creation improvements
• ALTER VARCHAR/VARGRAPHIC length support
• Codepage 819
GA, FP1, FP2, FP3 Highlights: SQL & Warehousing Workloads
Faster ELT/ETL performance
More Query Workloads Optimised
More Function supported
• Generated Columns
• RCAC
• User maintained temporal tables
Richer and Even More Compatible SQL
• JSON support
• BOOLEAN data type
• Common table expression (Db2 for z/OS compatibility)
• WITH and SELECT INTO support
• Alias for XML

16. What about getting critical fixes in between official updates?
• To ensure that critical fixes are made available as soon as possible, we have
introduced the concept of an interim fix (aka “iFix”)
• An iFix is an update based on the last shipped update + a select set of critical updates
that have occurred since the update was released
• Critical updates = Hiper, Security, and “high impact” APARs
• Cumulative (i.e. Db2 11.1.2.2 iFix002 contains Db2 11.1.2.2 iFix001)
• Reset after each new official update (i.e. the first iFix after Db2 11.1.3.3 was Db2 11.1.3.3 iFix001)
• To be released every 2-3 months until the next update occurs
16

17. How to stay current with Db2
• Passive
• Browse the list of available fix packs
• http://www-01.ibm.com/support/docview.wss?rs=71&uid=swg27007053
• Security Vulnerabilities, HIPER and Special Attention APARs fixed in DB2 for Linux, UNIX, and Windows
Version 11.1
• https://www-01.ibm.com/support/docview.wss?uid=swg21994955
• Proactive
• Go to the IBM Support page and sign up for “My notifications”
• http://www-01.ibm.com/software/support/einfo.html
17

18. Getting to Db2 11.1
• Upgrade to Db2 11.1 is supported from Db2 9.7, Db2 10.1, and Db2 10.5
v New option to upgrade from Db2 10.5 without the need for an offline backup
• For non-pureScale & non-DPF configurations, minimum origin level is Db2 10.5.0.7
• For pureScale configurations, minimum origin level is Db2 10.5.0.9
• Not available for DPF configurations at this time
v New option for “seamless” upgrade for HADR configurations
• For serial configurations, minimum origin level is Db2 10.5.0.7
• For pureScale configurations, minimum origin level is Db2 10.5.0.9 moving to at least Db2 11.1.1.1
18

19. UPGRADE without Offline Backup
• No more need to take an offline backup to ensure recoverability across upgrade!
• Example scenario :
• Recovery Procedure Overview:
• Revert back to original Db2 10.5 level
• Restore online backup (A)
• Begin to roll-forward to a desired point-in-time just before the error
• Receive SQL2463N or SQL2464N indicating the start of upgrade encountered in the logs
• Upgrade to Db2 11.1 and continue roll-forward recovery
Online Backup (A) Online Backup …..UPGRADE
Transactions Transactions ……
Db2 10.5 FP7/9 Db2 11.1
19

20. Primary
Replays
through
UPGRADE
UPGRADE
Primary
Standby
Backup
Image
UPGRADE +
BACKUP
Backup
Image
RESTORE
COPY BACKUP IMAGE
Standby
UPGRADE Log Records
Db2 11.1
pre-Db2 11.1
Seamless HADR Upgrades
• HADR environments can now be upgraded without re-
initializing the standby.
• Standby database will ‘replay’ the upgrade done on the
primary by rolling forward through the update log records
• For non-pureScale: Db2 10.5.0.7 => Db2 11.1.0.0
• For pureScale: Db2 10.5.0.9 => Db2 11.1.1.1
• More details in the DB2 11.1 documentation in “Upgrading
DB2 servers in HADR pureScale environments (without
standby reinitialization)”
• https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11
.1.0/com.ibm.db2.luw.qb.upgrade.doc/doc/t0070029.html
20

21. Looking at the latest
21

22. Where to go to find out what is in each Db2 11.1 update?
• “Mod Pack and Fix Pack Updates” section in the Db2 documents
• https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11.1.0/com.ibm.db2.luw.wn.doc/do
c/c0061179.html
• The following section describes only some of the highlights in Db2 11.1.2.2 and Db2
11.1.3.3
22

23. Just an FYI: RFEs that have been shipped so far in DB2 11.1
23
RFE

24. Db2 11.1.3.3
24

25. Online Index Create (concurrent write access) now on pureScale!
• Prior to Db2 11.1.3.3, a CREATE INDEX on a pureScale system would prevent
concurrent writes to the table
• As of Db2 11.1.3.3, full online index creation is supported on pureScale!
• Concurrent read and write access to the table now available
• One exception is expression-based indexes
• Off by default on pureScale
db2set DB2_CREATE_INDEX_ALLOW_WRITE=YES
db2 connect to MYDB
db2 create index ….
25

26. Online ADD/DROP of a cluster caching facility (CF)
• While Db2 already supports the ability to add a member online, the same was not
possible for a cluster caching facility (CF)
• As of Db2 11.1.3.3 you can now add or drop a CF online
• Expected use cases:
• Move a CF to a new host
• Remove a failed or maintenance-mode CF
• Improve HA of a single-CF cluster by adding a second CF
26
RFE

27. Greater flexibility in pureScale maintenance mode options
• When you are applying updates to a host in a Db2 pureScale configuration, or to its
underlying platform, you have to put it into maintenance mode to isolate it
• Prior to Db2 11.1.3.3, your options were:
• Single host in maintenance mode
• All hosts (cluster outage) in maintenance mode
• This can be a problem when you have a member in maintenance mode for a long time (e.g.
hardware issue) as it prevents any other host from being updated until the first issue is
resolved
• As of Db2 11.1.3.3 you can now put multiple hosts into maintenance mode!
• You can enter maintenance mode on each host one at a time as long until there is only one host (with at
least a CF, and preferably a member) left in the cluster
27
RFE

28. V11.1.2.2 V11.1.3.3
Rollback time
Seconds
Faster Rollback Of Very Large Transactions
• Internal changes made to use buffered I/O on filled
log files which significantly reduces rollback time for
very large transactions
• Example from internal testing:
• Single threaded rollback of a very large insert transaction is
up to 3x faster!
28

29. HADR Read on Standby (RoS): Improved monitoring
• Improve identification for cause of HADR ROS 'replay-only-windows’
• Additional diagnostic messages in db2diag.log on standby to report the operation that caused the
replay-only window
• When the LOG_DDL_STMTS database configuration parameter is set to YES on the primary, the
DDL statement text is included
• The frequency of these diagnostic messages can be configured using the
DB2_HADR_REPLAY_ONLY_WINDOW_DIAGLEVEL registry variable.
• db2fmtlog tool can also be used to display all log records (if any) that will cause a
replay-only-window
29

30. User Maintained Temporal Tables for BLU
• As of Db2 11.1.3.3, you can now:
• Create column-organized user-maintained system-period temporal tables (STT)
• Create column-organized user-maintained application-period temporal tables (ATT)
• Create column-organized and user-maintained bi-temporal table (BTT)
• Execute read-only temporal queries on column-organized temporal tables.
• For more information, see “Temporal Table Support for Column-Organized Tables in
DB2” technote
• http://www-01.ibm.com/support/docview.wss?uid=swg22016422
30
RFE

31. Why BLU Acceleration indexes are needed
• Db2 BLU acceleration provides superior performance for analytic workloads and is
designed for querying very large amounts of data … but some queries only access a
very small number of rows
Ø An index is the most efficient way to do this!
31

32. BLU Acceleration indexes
• DB2 11.1. currently implicitly creates unique indexes to support PRIMARY or UNIQUE KEY
constraints
• A unique index can be used to access data if at most 1 row qualifies and every key in the index must
have an equality predicate OR the FETCH FIRST 1 ROW ONLY clause is specified
• As of Db2 11.1.3.3, Db2 now supports CREATE INDEX for column-organized tables:
• Unique and non-unique indexes
• DROP and ALTER also supported
• ALTER INDEX only supports COMPRESS [YES|NO]
• These new indexes can be used to access column-organized data using most of the same
methods as row-organized data and without restrictions on the number of qualifying rows
32

33. Who will benefit from BLU Acceleration indexes?
• This new index support greatly improves Db2’s ability to execute queries that access
only a ‘small’ amount of column-organized data
• Extends Db2 BLU capabilities in the HTAP world
• Will improve performance for a range of applications:
• Data warehouse ETL with OLTP characteristics
• Operational analytics
• “Light” OLTP
• For existing BLU analytics workloads, will only be used in limited situations
• Analytic queries with transactional characteristics
33

34. Other notable content in Db2 11.1.3.3
• SSL Encryption for HADR communications now supported on all non-pureScale platforms
• Improved performance for moderate to heavy write workloads on pureScale configurations using TCPIP
sockets
• Native encryption enhancement to significantly reduce keystore access for transaction logs
• New value available for DB2AUTH registry variable to enable a warning diagnostic message to be
output when SERVER_ENCRYPT is not being respected
• Introduction of timeout support for Log-Archive/Retrieve Requests through Vendor API
• New “-o” option for db2haiku to export the current TSAMP configuration to an XML output file
34
RFE

35. A look into the future
35

36. Sign up for early access: ibm.biz/DB2-EAP
36

37. Db2 11.1.4.4
37

38. Db2 11.1.4.4
• This is most likely the final modification level for Db2 11.1
• I.e. future Db2 11.1 updates after this will just contain fixes
• Next updates would be DB2 11.1.4.5 followed by Db2 11.1.4.6
• Key content:
• Extent reclaim support for pureScale
• ISO JSON SQL functions
• Preview of HADR Read on Standby (RoS) enhancements
• Preview of 4K sector size support
38

39. Extent reclaim support for pureScale
• When table objects are dropped, the storage may not be immediately available to be
released from the tablespace
• Only space above the tablespace high water mark (HWM) can be released
• The ALTER TABLESPACE … REDUCE statement attempts to reduce the high water mark
for the table space by moving live extents to unused ones lower in the tablespace
• Supported with reclaimable storage DMS table spaces (created in Db2 9.7 or newer)
• Not yet supported on pureScale….. but we’re working on it for Db2 11.1.4.4!
39

40. Extent reclaim
40
DROP TABLE 2
DROP TABLE 3
Internal table space metadata extents
Table 1
Table 2
Table 3
Extent that is allocated to a table space, but not to a table
ALTER
TABLESPACE …
REDUCE MAX

41. A brief history of Db2 & JSON
• Db2 10.5 FP1 introduced JSON NoSQL support
• Focused on allowing Db2 to participate in the NoSQL
paradigm
• Our customers began to ask for native SQL
support of JSON in Db2
• Legacy applications accessing new JSON data
• New JSON applications wanting to access legacy data
Ø As a tactical response, we began to informally
reveal the existence of our proprietary Db2 JSON
SQL functions in 11.1.2.2
41

42. First wave of new built-in JSON SQL functions coming in Db2 11.1.4.4!
• New implementations of JSON SQL functions based on recent ISO report on SQL support for
JavaScript Object Notation (JSON)
§ The (proprietary) SYSTOOLS functions will be de-emphasized but will continue to be
supported
Schema Name Comments
SYSIBM BSON_TO_JSON Convert BSON formatted document into JSON strings
SYSIBM JSON_TO_BSON Convert JSON strings into a BSON document format
SYSIBM JSON_ARRAY Creates JSON array from input key value pairs
SYSIBM JSON_OBJECT Creates JSON object from input key value pairs
SYSIBM JSON_VALUE Extract an SQL scalar value from a JSON object
SYSIBM JSON_QUERY Extract a JSON object from a JSON object
SYSIBM JSON_TABLE Creates relational output from a JSON object
SYSIBM JSON_EXISTS Determine whether a JSON object contains the desired JSON value
42

43. What makes these JSON SQL functions different/better?
• Following a public “standard” (ISO technical report)
• Easier to use
• No need to qualify or add SYSTOOLS to function path
• No need to grant EXECUTE privilege
• Simpler and more flexible storage options
• You choose the stored format: JSON or BSON
• You choose the table organization: row or column
• You choose the column data type
• BLOB, CHAR, CLOB, VARBINARY, VARCHAR
• Normal Db2 mechanisms can be used to load JSON (or BSON) data into tables (e.g. INSERT, Load, etc..)
• Conversion functions are now complimentary not mandatory
43

44. Preview: HADR Read on Standby (RoS) as it exists today…
44
T1
T2
Primary server
T1
T2
Standby server
HADR log flow
• CREATE TABLE T1
• CREATE INDEX ON T1
• RUNSTATS ON T1
• Etc.

45. Preview: HADR Read on Standby (RoS) as it exists tomorrow…
45
T1
T2
Primary server
T1
T2
Standby server
HADR log flow
• CREATE TABLE T1
• CREATE INDEX ON T1
• RUNSTATS ON T1
• Etc.

46. Preview: Support for storage devices using 4K sector size
• This support can be enabled by setting the DB2_4K_DEVICE_SUPPORT registry
variable to ON
• This feature is not supported for production use at this time
Ø Try it out and let us know!
• Restrictions and limitations:
• The use of DMS Raw containers is not supported
• Backup and load copy files will be slightly larger
• There may be a performance penalty accessing:
• LOB data stored on 512-byte sector storage
• Backup or load copy files created prior to the enablement of 4K device support
46

47. Db2 “VNext”
47

48. Db2 “VNext”
• We plan to release a new version of Db2 in 2019
• This will get all our Db2 and Db2 Warehouse offerings on the same level of the Db2 common SQL
engine
• Db2 11.1 will be 3 years old in 2019!!
• We will continue to follow the “continuous delivery” paradigm started with Db2 11.1
• Steady release of function through GA and subsequent modification updates
• Db2 11.1 shipped 4 modification updates since GAing in 2016
• Version/Release number for Db2 VNext not decided yet
48

49. Examples of the enhancements planned for Db2 Vnext lifetime
• Available in the Db2 Warehouse family today
• External tables
• Grooming for columnar tables
• Reduced logging
• Schema level authorization
• Adaptive WLM
• Used as the active default for the DB2 Warehouse family today
• Advanced log space management
• Enhanced support for Currently Committed (CC) isolation on pureScale
49

50. Continuous evolution: New technologies
• Adaptation and integration of new technologies into the Db2 common SQL engine
• E.g. Machine Learning (ML)
• Exploration of new approaches to the challenges of data growth and real-time
analytics
• Db2 Event Store
• IBM Queryplex (http://www.queryplex.com/tag/db2/)
50

51. Further out?
• We revisit development priorities frequently (e.g. every quarter) in response to
customer and market demand/feedback
• As a result: some items move up, some down, some in, and some out.
• Public roadmap:
• http://ibm.biz/AnalyticsRoadmaps
51

52. How to learn more
52

53. Come hear more from Db2 community experts!
53
Db2 development lab speakers include:

54. Try out IBM Db2 Developer Community Edition
• Free, fully-functional version of Db2
• Includes all Db2 features such as compression, BLU Acceleration
• Supports all Db2 configurations including pureScale and DPF
• No restrictions on where you can use it (e.g. production, development, test)
• Find out more: http://Ibm.biz/Db2devc
• Restrictions
• Unsupported edition (non-warranted)
• Environment limited to:
• 4 cores with 16GB of memory
• 100GB per database
• Excludes some “Supporting Programs”:
• Cognos, IDA, WebSphere AS / MQ, DSM Enterprise
• Provided as a docker-based install with Data Server Manager (DSM) and Data Studio
• Also offered: Db2 Developer-C Community Edition which is a native Db2 (only) installation
54

55. Jessica Rockwood
IBM
jessicae@ca.ibm.com
Db2 family and 11.1.4.4 update