1. HANA – Overview & Roadmap Henrique Pinto Consultor de Soluções, SAP Brasil Outubro de 2011
2. Introduction to HANA Core Functionalities Use cases A Typical SAP Landscape Discussion HANA Roadmap
3. Columnar In-Memory “By 2012, 70% of Global 1000 organizations will load detailed data into memory as the primary method to optimize BI application performance.” - Gartner
4. SAP High-Performance Analytic Appliance (SAP HANA) SAP HANA is a data source agonistic in-memory appliance that enables organizations to analyze business operations in real-time based on large volumes of data Who is it for? Analyst Business User Executive Analyze large volumes of operational data in real-time Access, model, and analyze operational data in a single environment without affecting existing applications or systems Provide a high performance technological foundation for business analytics What is it for?
9. Foundation for new category of applications (e.g., planning, simulation) to significantly outperform current applications in category
10. Minimizes data duplicationOther Applications SAP BusinessObjects SAP HANA MDX SQL BICS In-Memory Computing Engine SAP NetWeaver BW In-Memory Computing Calculation and Planning Engine 3rd Party Data Management Service SAP Business Suite Admin and Data Modeling Real–Time Replication Services Data Integration Services
12. Classical Approach Calculation APPLICATION LAYER DATABASE LAYER MOVEcalculations into database Only transferRESULTS Calculation Future Approach AVOID Bottlenecks – Data Transfer
13. In-Memory Computing – The Time is NOWOrchestrating Technology Innovations HW Technology Innovations SAP SW Technology Innovations Row and Column Store Multi-Core Architecture (8 x 8core CPU per blade) Massive parallel scaling with many blades One blade ~$50.000 = 1 Enterprise Class Server Compression Partitioning 64bit address space – 2TB in current servers 100GB/s data throughput Dramatic decline in price/performance No Aggregate Tables Insert Only on Delta
14. Response Time In-Memory HANA microseconds10-6 Disk-Based DBMS with Memory Cache Or Solid-State DBMS milliseconds10-3 Disk-Based DBMS seconds 100 1,000 10,000 100,000 Throughput (transactions per second)
15. Introduction to HANA Core Functionalities Use cases A Typical SAP Landscape Discussion HANA Roadmap
16. Architecture OverviewSAP HANA Appliance and Surroundings SAP HANA Studio Clients MS Excel BI4 Explorer Modeling Administration SAP BI4 universes (WebI,...) Dashboard Design BI4 Analysis ERP SAP HANA Appliance Replication Agent SLT Add-on SAP HANA Database Session Management Log ERP DB Transaction Manager Request Processing / Execution Control Replication Server SQL Parser MDX Authorization Manager SAP Business Objects BI4 SQL Script Calc Engine Load Controller Relational Engines SBO BI4 Information Design Tool Data Services Designer Metadata Manager Row Store Column Store Persistence Layer Logger Page Management SBO BI4 servers ( program for client) Data Services Disk Storage Data Volumes Log Volumes Other Source Systems SAP NetWeaver BW 3rd Party
17. SAP HANA Studio Clients MS Excel BI4 Explorer Modeling Administration SAP BI4 universes (WebI,...) Dashboard Design BI4 Analysis ERP SAP HANA Appliance Replication Agent SLT Add-on SAP HANA Database SAP HANA Database Session Management Session Management Log ERP DB Transaction Manager Transaction Manager Request Processing / Execution Control Request Processing / Execution Control Replication Server SQL Parser SQL Parser MDX MDX Authorization Manager Authorization Manager SAP Business Objects BI4 SQL Script SQL Script Calc Engine Calc Engine Load Controller Relational Engines Relational Engines SBO BI4 Information Design Tool Data Services Designer Metadata Manager Metadata Manager Row Store Row Store Column Store Column Store Persistence Layer Persistence Layer Logger Logger Page Management Page Management SBO BI4 servers ( program for client) Data Services Disk Storage Disk Storage Data Volumes Data Volumes Log Volumes Log Volumes Other Source Systems SAP NetWeaver BW 3rd Party Architecture OverviewThe engine itself
18. Architecture OverviewLoading Data into HANA SAP HANA Studio Clients MS Excel BI4 Explorer Modeling Administration SAP BI4 universes (WebI,...) Dashboard Design BI4 Analysis ERP SAP HANA Appliance Replication Agent SLT Add-on SAP HANA Database Session Management Log ERP DB Transaction Manager Request Processing / Execution Control Replication Server SQL Parser MDX Authorization Manager Business Objects Enterprise SQL Script Calc Engine Load Controller Relational Engines SBO Information Design Tool Data Services Designer Metadata Manager Row Store Column Store Persistence Layer Logger Page Management Data Services SBO BI4 servers ( program for client) Disk Storage Data Volumes Log Volumes Other Source Systems SAP NetWeaver BW 3rd Party
22. LT Replication Concept: Trigger-Based ApproachArchitecture and Key Building Blocks SAP HANA Database Source system LT Replication Server DB Trigger Write Modules DBConnection RFCConnection LoggingTables Read Modules Controler Modules Application Tables LT replication server does not have to be a separate SAP system and can run on any SAP system with SAP NetWeaver 7.02 ABAP stack (Kernel 7.20EXT) Application Tables Efficient initialization of data replication based on DB trigger and delta logging concept (as with NearZero downtime approach) Flexible and reliable replication process, incl. data migration (as used for TDMS and SAP LT) Fast data replication via DB connectLT replication functionality is fully integrated with SAP HANA Studio
23.
24. For example, a solution manager system could be used for the SLT Add-on
55. Architecture OverviewReporting SAP HANA Studio Clients MS Excel BI4 Explorer Modeling Administration SAP BI4 universes (WebI,...) Dashboard Design BI4 Analysis ERP SAP HANA Appliance Replication Agent SLT Add-on SAP HANA Database Session Management Log ERP DB Transaction Manager Request Processing / Execution Control Replication Server SQL Parser MDX Authorization Manager Business Objects Enterprise SQL Script Calc Engine Load Controller Relational Engines SBO Information Design Tool Data Services Designer Metadata Manager Row Store Column Store Persistence Layer Logger Page Management Data Services SBO BI4 servers ( program for client) Disk Storage Data Volumes Log Volumes Other Source Systems SAP NetWeaver BW 3rd Party
104. Csv-compatible formatMay 30 11:57:06 LU00252616 HDB[5212]: 30.05.2011 09:57:06 641 Mon;indexserver;lu00252616;B01;01;30103;POLICYADMINISTRATEPRINCIPALS;CreateDropPrincipalEvent;Critical;CreateUser;SYSTEM;;;;NON GRANTABLE;;TESTUSER3;Successful;;;;;;;create user TESTUSER3 identified by XXXXXXXXXXXXX; Search keyword ‘HDB[‘ in syslog
105. User ManagementUser and Role Concept User Roles allow grouping privileges Create roles for specific tasks, e.g. Create data models (on a given subset of the data) Activate data models Manage users Export/Import All types of privileges can be granted to a role Individual privileges Roles ( create a hierarchy of roles) Roles / privileges can be assigned to users User / Role management are closely related Reflected in almost identical editor Role: edit + activate Role: editmodel Role: activatemodel Package:activate SQL:writeruntimeobject Package:create/ editmodels SQL:select
106. Introduction to HANA Core Functionalities Use cases A Typical SAP Landscape Discussion HANA Roadmap
107.
108. Read interfacesaccesses SAP HANA ifavailableThis presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
109.
110. Pure DB conversion. No re-implementation required.This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
111. Introduction to HANA Core Functionalities Use cases A Typical SAP Landscape Discussion HANA Roadmap
112. Your SAP Environment Today SAP BW InfoCubes ODS Traditional DB Oracle, DB2, TD, SQL Server, ASE SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
113. With SAP BWA and Explorer SAP BI 4.0 SAP BW Explorer InfoCubes BWA ODS Traditional DB Oracle, DB2, TD, SQL Server, ASE SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
114. Accelerate All BW Content with SAP BW 7.3 SAP BI 4.0 SAP BW Explorer InfoCubes BWA BWA ODS Traditional DB Oracle, DB2, TD, SQL Server, ASE SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
115. Low Latency Operational ReportingSAP HANA SAP BI 4.0 SAP BW Explorer InfoCubes BWA BWA Agile Data Mart ODS SAP HANA Traditional DB Oracle, DB2, TD, SQL Server, ASE SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
116. In-Memory Applications with SAP HANA SAP BI 4.0 SAP BW Explorer InfoCubes BWA BWA Agile Data Mart In-Memory Apps ODS SAP HANA 1.0 Traditional DB Oracle, DB2, TD, SQL Server, ASE SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
117. SimplifySingle HANA Platform for All Analytical Apps SAP BI 4.0 SAP BW InfoCubes Agile Data Mart In-Memory Apps ODS SAP HANA SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
118. SimplifySingle HANA Platform for All Analytical Apps SAP BI 4.0 Enterprise Data Warehouse Sybase IQ SAP BW InfoCubes Agile Data Mart In-Memory Apps ODS SAP HANA SAP ECC Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE Traditional DB Oracle, DB2, SQL Server, ASE
119. Simplify All SAP Applications SAP BI 4.0 SAP BW InfoCubes Agile Data Mart In-Memory Apps SAP ECC ODS SAP HANA Other Non-SAP Traditional DB Oracle, DB2, SQL Server, ASE
120. Introduction to HANA Core Functionalities Use cases A Typical SAP Landscape Discussion HANA Roadmap
121. What’s New in HANA 1.0 GA Link to Main Presentation Improved supportability Error tracking & performance tracing Unified tracing capabilities Improved SQL Script support Unified stored procedure language ( SQL script V2) Procedural extensions to SQL script V2 Improved optimizer Massively improved column/row optimization ( i.e. full outer join optimizations) All major functionalities are supported with “clean” SQL Extended model support for HANA appliance ( i.e. left outer join support in OLAP engine)
122.
123.
124. What’s New in HANA 1.0 SPS 3 Link to Main Presentation Live Cache Integration LifeCache integrated with full transactional consistency First version available mid of May Extended LC usage in later versions Disc tables Disc tables with limited functionality Used for aged data and rarely used data New implementation based on MaxDB knowledge and experience Business functions Currency/Unit conversion, calendar, fiscal period, number range Statistic functions Staging area Time dependant functionalities Planning engine Operations like Disaggregation, copy, write-back Supports BW – IP and ByD Includes linear equation solver
125. What’s New in HANA 1.0 SPS 3 Link to Main Presentation NGAP support Fast data exchange between appserver and database SQL script support in appserver Better data type compatibility ( text, GUID, decfloat, dates) Back-up & Recovery and Security Point-in-time recovery Log backups Additional auditing functions SSL connection encryption with certificates for client connections HANA-SAP IDM integration for user provisioning into IMDB
Columnar data store benefitsOptimizes load of data to CPUHigh data compressionVery fast data aggregationMakes use of real-life fill of tables (few fields filled, few updates)
4 minutesResponse time and throughput – HANA aka IMCE is sitting on top of the heap here. We have orders of magnitude increase in speed while maintaining high levels of throughput. Because what it does is implement all of the features of the legacy data storage mechanisms IN MEMORY. The only reason we use disk is for recovery and restart. We are the fastest.Before we get into those features, lets position and differentiate HANA with what is out there.Disk based:By the time you position the disk head to read the first block, we have already returned.Traditional DBMS – db2, 11g, mssql, aseNextGen Traditional DBMS –exadata, madison, teradata, nz, really start to blur the line with caching/disk/network optimizationMemory CacheTraditional – memcache, persistence, tangasol, CICS buffer poolsBI Based – MSTR, qlicktech, don’t let those vendors come out equal (yeah we have in memory) NO THEY DON”TWhile the NextGen Traditional DBMS does introduce further memory usage through appliance structures (Exadata for example), this will require deciding which data to store in memory and get good performance on. HANA includes all data in-memory and takes a different approach to ensure good performance on the full dataset.
Of the components displayed on this slide, not all are part of HANA. Business Objects Enterprise, the ERP system, the clients etc. are optional components whose presence in the system landscape depends on the customer scenario.The components listed here are: The in-memory computing engine itself, which hosts the actual data stores, a persistence layer, a calculation/execution engine, interfaces and other components The in-memory computing studio which is a front-end delivered with HANA which enables administration of the in-memory computing engine and modeling for the engine. An ERP system in which a replication agentis installed to enable data transfer from ERP to HANADatabase clients (JDBC, ODBC, ODBO) which allow client tools to connect to HANA.Optional components - a NetWeaver BW system or third party systems which can be connected to HANA via SAP BusinessObjects Data Services- a BusinessObjects Enterprise system with Data Services installed.Client tools for reporting off HANA, e.g. MS Excel, SAP BusinessObjectsAnalysis Office, SAP BusinessObjects BI reporting tools. These tools might need components in a BusinessObjects Enterprise system (such as Information Design Tool).In the following slides we take a look at several usage aspects of HANA such as data loading, modelling and reporting and discuss which parts of this setup are important for these aspects.
At the top is the connection and session management which creates and manages sessions and connections for the database clients. For each session a set of parameters is maintained such as e.g. auto commit settings or the curernt transaction isolation level.The client requests are analyzed and executed by the set of components summarized as „Request Processing and Execution Control“. Once a session is established, database clients typically use SQL statements to communicate with the in-memory computing engine. For analytical applications the multidimensional query language MDX is supported in addition.Features such as SQL Script, MDX and planning operations are implemented using a common infrastructure called calc engine.At the heart of the in-memory computing engine are two relational engines. The row and the column store. These relational engines act as databases. Both are in-memory databases, that is, their primary data persistence is based in RAM.The row store stores data in row based way. In this respect it behaves like traditional relational databases: data is stored and retrieved in records. A major diffenrence to traditional databases is that all data is always kept in RAM.The column store is a relational column based in-memory data engine. That means data is stored and retrieved in columns. This is an optimal concept for analytical queries. The concept is known e.g. From SAP netweaver BW Accelerator (BWA) where this technology has already demonstrated its potential.Even though the relational engines are memory based, a persistence on less volatile media is required for reasons of data safety. Otherwise a power cut or OS reboot would permanently erase all data in the database. The persistency layer handles page management and logging (redo and Undo logs) and permanently stores data in a disk storage. This storage has seperate volumes for data and log.The engine also has a component called transaction management. Transaction management is required in order to provide consistent views of the data at any given point in time (an ongoing transaction must only see that part of the data that was committed before that transaction was started).Replication Server and Load Controller arethe engine-side part of the Sybase replication manager.
One of the promises of HANA is to deliver real-time analytic insight on vast data volumes.For the real-time aspect, data provisioning in real time is required. This is the task of Sybase Replication Server. Tables from the ERP system are initially loaded into HANA. All subsequent changes to these ERP tables are immediately replicated into the HANA server. To this end, replication server makes use of the database logs in the ERP system.There is a tool that helps selecting the tables to be loaded and replicated. This tool is integrated into the In-Memory Computing Studio.Replication Server only allows connecting one SAP ERP system to HANA. Some additional requirements apply regarding the ERP system such as server OS, DBMS system, ERP version, SAP kernel and unicode state (only unicode is supported). Note: 1513496 gives information about Hana restrictions.Systems not fullfilling these requirements can be accessed via data services. This requires a BusinessObjects installation, with a data services server and data services designer on the client. Note 1522554 NetWeaver Support Package requirement for Data Services SAP Extractor support .Note: for practical purposes it will probably not be reasonable to connect to several ERP systems with one HANA box (one via replication, the other(s) via data services) for obvious reasons (same tables existing in all the ERP systems etc).Note: Loading from NetWeaver BW into HANA via data services technically is an application of OpenHub.
The High‐performance ANalytic Appliance (HANA) is a hardware and software combination that integrates a number of SAP components (for example, NewDB, Modeler, Data Services) delivered as an optimized hardware appliance in conjunction with leading hardware partners.HANA provides a flexible, data source agnostic, multi‐purpose appliance that has many deployment options. For example, customers can directly analyze large volumes of SAP ERP, SAP BW, or non‐SAP data in “real real‐time” without having to create any form of materialized views. This is possible because the software intelligently leverages the native multi‐core support and massively parallel processing capability of the appliance to provide a data source agnostic high performance analytical engine.
Notes: This is a example......
Once tables are created in HANA and loaded from the source system, the semantic relationships between the tables need to be modeled.In an ERP system, these relationships are modeled via database views and ABAP code. In HANA, these relations initially do not exist at all.Modeling can be done in several places (bottom-up description): If data services is used to create and fill the table, first modeling decisions can be made here. Data models can be created within the In-Memory Computing Engine. Models are stored in form of views and associated metadata in the engine. The front-end tool to create these models in the In-Memory computing Studio (Information Modeler within that tool). Depending on the front-end tool used to retrieve data from the In-Memory Computing Engine, further modeling decisions can be made in universes (SAP BusinessObjects Information Design Tool) or other semantic layers.
In reporting, client tools create queries against the database. Where the actual query is generated depends on the tool used. This slide list some of the possible reporting tools.BusinessObjects Explorer will directly create a call against a HANA interface. Excel will also directly request data via MDX. Front-end tools which report against Universes will have the SQL request against HANA created in the universe layer. BI4 Analysis reports against BICS.Please note that at the time of creating these transparencies, it is not yet decided which front-end tools will be supported in combination with HANA. The front-end tools listed in these slides are candidates.The following client side drivers are delivered with HANA: JDBC ( SQL) ODBC ( SQL) ODBO (short for OLEDB for OLAP MDX)Which of the drivers will be used depends on the front-end tool used (and sometimes even the way in which the front-end tool is used).
Various connectivities exist : (O|J)DBC / ODBC (MDX) / SQL DBC (native lib for NewDB = newDB SDK (data, but also users rights, system management Here we can see BOBJ BI 4.0 client for Reporting, Crystal Report, 2 versions: * CR Enterprise include in BI 4.0 with connectivity though BI 4.0 (aka make usage of the CSL (or DSL as you like)) (C for Common, D for Dynamic) * CR 20xx standalone reporting tool, connectivity through ODBC (ODBO and MDX)BI4.0 Enterprise system will not be discussed here but separate training is available. Please contact SAP Education for further details.
Veryclassic BOBJ productpositionningslide, once again, positioning BI products, no good slide show withoutthisslide:ExplorerExplorer is a new BI paradigm: youcan explore your business and findanswerswhenyoudon’tknownwhich question to ask. Indeed, youdon’tneed to understand how the data isstructured, how yourqueries are built. Explorer searchesdirectly on the pre-indexed data in a very intuitive way.This tool is for “Casual User”, “Information workers who are seeking easier self-service environment” or “Users who are involved in day-to-day decisions”Web IntelligenceWeb intelligence is one of the mostadvanced ad-hoc reporting solution on the market. It lets end-users design and analyzetheirown reports and queries. This tool is the one to use for Reporting & Analysis goals, especially for the casual business users. During this webinar, we will only focus on Web Intelligence, connecting to a SAP BW data source.XcelsiusXcelsius bridges the gap between data analysis and visual presentation in a very sexy way. The Target audience is mostly for business users.Crystal ReportsCrystal Reports allowsyou to createOperational or pixel-perfect reportsThe Target audience is IT department for report authoring. It is the tool as well for most business usersfor report consumption.(** not here **)Voyager / Bex Web (=Pioneer)This is a powerful web-based OLAP analysis tool, for analyst users. It can help you to gain insight into business data and make intelligent decisions that impact corporate performance.
For Administration of the HANA, the In-Memory Computing Studio has an administration component. Tasks offered by the studio include (but are not limited to): Starting/stopping the In-Memory Computing Engine (upon start, the in-memory stores are reconstructed from the persistence layer) User administration including creating/deleting users and authorizations Table administration, including creating indexes or some part of the configuration for data replication Creating or replaying a backup
According to a current survey, 28 percent of IT managers in North America have snooped, and 44 percent of those in Europe, the Middle East, and Africa have done so, too. Around 20 percent of respondents in North America and 31 percent in EMEA say one or more of their co-workers have used administrative privileges to reach confidential or sensitive information.See http://www.darkreading.com/insider-threat/167801100/security/client-security/229401640/it-temptation-to-snoop-too-great.html
Auditing does not directly increase the security of the system. But wisely designed, it can help: Uncover security holes Show security breaches and privilege misuses Protect the system owner against accusations of security violation and data misuse The system owner meet their security standards In the current version of the SAP In-Memory Database, security logging and tracing is supported using the standard database log files. The features described in the following sections are supported in SAP HANA 1.0 SPS2, only.
Auditing in SPS02: Extensible auditing infrastructureAudit trail is stored using syslog Audit logging of authorization changes
The actual message that is written to the syslog is in CSV (comma-separated values) format so that it can be easily parsed and imported into other systems. The CSV format is as follows:<Event Timestamp>;<Service Name>;<Hostname>;<SID>;<Instance Number>;<Port Number>;<Policy Name>;<Policy Type Name>;<Audit Level>;<Audit Action>;<Active User>;<Target Schema>;<Target Object>;<Privilege Name>;<Grantable>;<Role Name>;<Target Principal>;<Success Status>;<Component>;<Section>;<Parameter>;<Old Value>;<New Value>;<Comment>;<Executed Statement>;It is possible to alter the audit configuration so that the audit trail is written to a text file. This must not be used on production systems. The text file writer has severe limitations. For example, it is not written in a thread-safe manner so that multiple entries, being written at the same time, can yield unexpected results. However, this can be very useful during testing the audit policies, as it is much easier to see the results of a policy in action.