ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
Optim test data management for IMS 2011
1. InfoSphere Optim Test Data Management Solution– IMS Focus Peter Costigan – Product Line Manager, Optim Solutions 9/28/2011
2.
3. Mastering information across the Information Supply Chain Transactional & Collaborative Applications Business Analytics Applications External Information Sources Trusted Relevant Governed Analyze Integrate Manage Cubes Streams Big Data Master Data Content Data Streaming Information Information Governance Data Warehouses Content Analytics Govern Quality Security & Privacy Lifecycle Standards Integrate & Cleanse
4. Requirements to manage data across its lifecycle Validate test results Define policies Report & retrieve archived data Enable compliance with retention & e-discovery Move only the needed information Integrate into single data source Create & refresh test data Manage data growth Classify & define data and relationships Develop database structures & code Enhance performance Discover where data resides Develop & Test Discover & Define Optimize, Archive & Access Consolidate & Retire Information Governance Core Disciplines Lifecycle Management
5.
6.
7. Optim Captures Complete Business Objects Business data is related across a wide variety of data sources
11. Sensitive Production Data: What’s the risk? Hundreds of thousands of secret reports regarding US wars in Iraq and Afghanistan published on WikiLeaks. December 2010: A private in the US military, downloaded top secret military documents and passed them to journalist for publication. This puts US national security at risk as well as the lives of those named in reports. Unprotected test data sent to and used by test/development teams as well as third-party consultants. February 2009: An FAA server used for application development & testing was breached, exposing the personally identifiable information of 45,000+ employees. SQL injection is fast becoming one of the biggest and most high profile web security threats. April 2011 : A mass SQL injection attack that initially compromised 28,000 websites shows no sign of slowing down. Known as LizaMoon, this malicious code is after anything stored in a database. Hackers obtained personal information on 70 million subscribers. April 2011: Malicious outsiders stole name, address (city, state, zip), country, email address, birth date, PlayStation Network/Qriocity password and login, and handle/PSN online ID, and possibly credit card numbers from 70 million Sony PlayStation users.
12.
13.
14.
15.
16.
17.
18.
19. Requirements to manage data across its lifecycle Validate test results Define policies Report & retrieve archived data Enable compliance with retention & e-discovery Move only the needed information Integrate into single data source Create & refresh test data Manage data growth Classify & define data and relationships Develop database structures & code Enhance performance Discover where data resides Develop & Test Discover & Define Optimize, Archive & Access Consolidate & Retire Information Governance Core Disciplines Lifecycle Management
20.
21. InfoSphere Discovery Speeds Understanding Data Table 1 Table 25 The Discovery Engine analyzes data values to automatically discover the columns that relate rows across data sources, and the columns which contain sensitive data . IBM InfoSphere Discovery Hit Rate: 98% X - Row Member SS # Age Phone Sex 1 595846226 123-45-6789 15 (123) 456-7890 M 2 567472596 138-27-1604 8 (138) 271-6037 F 3 540450092 154-86-4196 22 (154) 864-1961 M 4 514714372 173-44-7900 55 (173) 447-8996 F 5 490204164 194-26-1648 4 (194) 261-6476 F 6 466861109 217-57-3046 66 (217) 573-0453 M 987,623 444629628 243-68-1812 25 (243) 681-8107 F 987,624 423456789 272-92-3629 87 (272) 923-6280 M ID Demo1 595846226 0 567472596 1 540450091 2 514714372 3 490204164 1 466861109 0 444629628 3 423456789 2
22.
23.
24.
25.
26.
Editor's Notes
This presentation is the Essentials of Test Data Management part of the InfoSphere Information Lifecycle Management Solutions
We are going to cover the following: -Information Governance: Review -What is Test Data Management -Role of Test Data Management in the Testing Discipline -Risks and Challenges of Poor Test Data Management -Best Practices in Test Data Management -Data Privacy Concerns with Test Data -IBM InfoSphere Optim Test Data Management Solution -Conclusion
This slide you have seen in the Information Lifecycle Management presentation. There are typically hundreds or even thousands of different systems throughout an organization. Information can come in from many places (transaction systems, operational systems, document repositories, external information sources), and in many formats (data, content, streaming). Wherever it comes from, there are often meaningful relationships between various sources of data. We manage all this information in our systems, integrate to build warehouses and master the data to get single views and analyze it to make business decisions. This is a supply chain of information, flowing throughout the organization. Integration information, ensuring its quality and interpreting it correctly is crucial to using the information to make better decisions. Information must be turned into a trusted asset, and governed to maintain the quality over its lifecycle.
We went through the requirements for Information lifecycle management. We are focusing on Develop and Test. Specifically efficiently creating the test & development environments (and protecting sensitive data within), effectively validating test results and quickly & securely deploying the application
How our enterprises creating test data today…manually or just cloning their entire production to obtain their test database. The downside of cloning your entire production is that you now have a data growth problem and uses significant storage. In addition, you have a privacy issues because you have exposed sensitive data to developers and testers using production data for testing.
The business benefits of test data management: More time for testing In many organizations, 30-40% of test script execution is spent on manufacturing new test data…and much of this is done manually today. Automating Test Data Management will reduce the amount of time spent creating new data thereby allowing for the execution of more tests Reduce cost Maximize allocated disk space Catch errors earlier in the testing cycle because now you have realistic test data to test with. Shift errors from production to test Increase data quality Enforce data ownership Test Data Management offers role driven security to support level segmentation of the development and testing teams Reduce data dependencies across test sets Multiple test sets often use the same data, but different tests can negatively impact other tests using the same data. Test Data Management allows for the creation of an unlimited number of test data sets and can create unique IDs each time to ensue clean data is used when testing
Why is it important to mask sensitive information….some examples: -Hackers obtained personal information on 70 million subscribers to Sony PlayStation . See article: http://online.wsj.com/article/SB10001424052748704587004576245131531712342.html 'LizaMoon' Mass SQL Injection Attack Escalates Out of Control. See article: http://www.eweek.com/c/a/Security/LizaMoon-Mass-SQL-Injection-Attack-Escalates-Out-of-Control-378108/ -Federal Aviation Administration: Exposes unprotected test data to a third party http://fcw.com/articles/2009/02/10/faa-data-breach.aspx Release of thousands of classified documents by WikiLeaks founder Julian Assange jeopardizes U.S. national security. US Army launches investigation. http://www.mcclatchydc.com/2010/12/23/105763/army-wikileaks-probe-could-lead.html
Ever since the inception of Information Technology (aka Electronic Data Processing) it has become commonly accepted to allow a certain percentage of IT staff to have access to the production environment. These "trusted employees" were carefully screened and usually in close proximity to executive management due to the confidentiality of critical sensitive corporate data. Originally, this was a practical matter and was voluntarily implemented by the enterprise. Over the years, the onslaught of international Data Privacy Legislation has made this a compliance matter as well. Today's large, multi-national enterprise is faced with numerous cross-border data privacy exposures. Additionally with the deployment of third-party contractors, there is further separation from the traditional "trusted employee". Data Masking provides development teams with meaningful test data, without exposing sensitive private information. Static data masking is the most common and most tradition approach. Static data masking extracts rows from production databases, conceal data values that ultimately get stored in the columns in the test databases. The concealed values are physically stored in the target databases. Dynamic data masking (a term coined by Gartner), is an emerging technology that performs data obfuscation at the presentation layer in real time. Implemented at the SQL protocol layer, operating as a database listener, in-bound SQL from any application is inspected and then dynamically re-written to include the appropriate masking function. The result is data masking at the presentation layer without having to change the underlying database or the application source code.
We went through the requirements for Information lifecycle management. We are focusing on Develop and Test. Specifically efficiently creating the test & development environments (and protecting sensitive data within), effectively validating test results and quickly & securely deploying the application
Most companies are still struggling with the first step of understanding their complex heterogeneous data landscapes for test data management. – with the resulting impact on the overall quality of applications. Some of the challenges are knowing what data is needed for test cases, lack of understanding of where data is located and how the data is related, limited understanding of the confidential data elements. It’s cost prohibitive to conduct manual analysis and hand coding.
-Test Data Management allows development teams to accelerate testing activities on a project -Test Data Management exploits production data while ensuring security of confidential data -Providing testers and developers with access to test data can improve operational efficiency and optimize resources on a project -A comprehensive Test Data Management solution is needed to minimize cost and shorten development cycles
You want to point customers to the InfoSphere Optim ibm.com page, solution sheet, whitepaper and case study on test data management.