Finance Data Lake objective is to create a centralized enterprise data repository for all Finance and Supply Chain data. It serves as the single source of truth. It enables a self-service discovery Analytics platform for business users to answer adhoc business questions and derive critical insights. The data lake is based on open source Hadoop big data platform and a very cost effective solution in breaking the ERP data silos and simplifying the data architecture in the enterprise. POCs were conducted on in-house Hortonworks Hadoop data platform to validate the cluster performance for Production volumes. Based on business priorities, an initial roadmap was defined using 3 data sources including 2 SAP ERPs and Peoplesoft (OLTP systems). Development environment was established in AWS Cloud for agile delivery. The near real time data ingestion architecture for the data lake was defined using replication tools and custom SQOOP based micro-batching framework and data persisted in Apache Hive DB in ORC format. Data and user security is implemented using Apache Ranger and sensitive data stored at rest in encryption zones. Business data sets were developed in Hive scripts and scheduled using Oozie. Multiple reporting tools connectivity including SQL tools, Excel and Tableau were enabled for Self-service Analytics. Upon successful implementation of the initial phase, a full roadmap is established to extend the Finance data lake to over 25 data sources and enhance data ingestion to scale as well as enable OLAP tools on Hadoop.

1. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Verizon: Finance Data Lake
Implementation as a Self Service
Discovery Big Data Platform
Sreenath Akinepalli
Sandeep Katuku
June 2017
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

2. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
• Why Data Lake?
• Finance Data Lake Value
• Use Cases
• Architecture Overview
• Data Ingestion at Scale
• Data Validation
• Security
• Self Service Discovery
• Takeaways
2
Agenda
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

3. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Why Data Lake?
• 70% of time spent data gathering vs 30% analysis
• Data exists in multiple ERP systems and other silos
• Data Replication through point to point interfaces
• Lack of Normalization & Harmonization
• Data Latency
3
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

4. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Finance Data Lake Value
Centralized Enterprise Data Repository & Self-Service Discovery Platform
Simplifies
access to raw
ERP data
Eliminate data
replication – lower
TCO
Enable Data Share -
reduce # of point to
point integrations
Rationalize &
harmonize master
data
Centrally apply
common business
rules
Single set of
Reporting &
Analytical tools
Drive Data
Archiving
Strategy
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners. 4

5. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Use Cases
• Accounts Payable (AP)
Working Capital Analytics
• Historical DataMart
• Spend Analytics
• Labor Transformation
• Audit & Compliance
• Capital Reporting &
Analytics
5
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

6. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 6
Architecture Overview
Source Systems
Ingestion
Store
Semantic Model
Data Consumption
Transformation
Governance
3
2
1
Data Lake
Logical Model
Reporting & Analytical Tools
Other Data
Sources
External
Applications
Interfaces
Security
ETL
ERP1 ERP2 ERP3
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

7. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 7
Data Ingestion - Design Success Factors
• Hadoop as Target
• Multiple Data Sources
• Transactional Source Systems
(OLTP)
• ACID Limitations
• Different types of tables
• Ability to Scale to thousands of
tables
• End to End Traceability
• Identifying the Tools
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

8. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 8
Databases
Data Ingestion at Scale
FINANCE DATA LAKE
• Metadata Driven Design
• Dynamic Object Creation
• Supports File, Batch & Incremental Ingestion
• File & Batch Ingest Data directly to Hadoop
• Incremental data streams from Source to Staging
• Micro batch process moves data to Hadoop
• Data Merge using Lambda Architecture
Text Files
Landing
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.
Staging
Lambda Architecture
File
Ingestion
Incremental
Ingestion
Batch
Ingestion

9. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 9
Data Ingestion - Data Patterns
Challenge Solution
Handling Deletes Prior snapshots as deletes
Handling Updates Updates as deletes and inserts
Primary Key updates Prior snapshots as deletes
Concurrent Operations Using transaction id
Batch Operations Configuration to capture truncates
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

10. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 10
Data Ingestion - Enhancement
• Simplified Architecture
• Ingest Data directly to Hadoop
• Supports all ERP tables
• Dynamic DDL changes from Source to Target
ERP3
FINANCE DATA LAKE
ERP2
ERP1
ERP(SOURCE)
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.
Attunity
Replicate
Lambda Architecture

11. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 11
Data Validations
• 4000 Automated Validations
• Row & Column count Dashboards
• Source to Hadoop Comparisons
• Data Latency Dashboards
• Report Reconciliation
Row Count
Column
Count
Aggregation
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

12. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 12
Security
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.
Access: (To Hadoop Cluster)
• Authentication
• Kerberos (Direct Access)
Perimeter Level Security:
• Network Security (firewalls)
• Apache KNOX (Gateway – BI Tools)
Data: (Protecting data in Cluster)
• Authorization
• Ranger
• Role Based
• Row/Column Level
• Encryption
• Data Masking
Audit/Administration
• Access Review
• Log Monitoring
Finance Data Lake

13. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 13
Self-Service Discovery
Explore TransformModel Discover Prepare & Share
relevant data data to
understand
its potential
& enrich
data to make
it ready for
analysis
powerful
New
insights
insights for
enterprise
leverage
Discovering the True Potential of Big Data
Data Lake Platform
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

14. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Takeaways
• Data lake based on Hadoop big data platform is the right
choice for self service discovery & analytics
• Adopt an Agile mindset in the implementation
• Evolve the architecture with the right tools for the right job
14
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.

15. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Thank You
© Verizon 2017 All Rights Reserved. Information contained herein is provided AS IS and subject
to change without notice. All trademarks used herein are property of their respective owners.