This presentation gives insights, tips, and techniques on using the Oracle EPM Automate tool to automate movement of metadata and data for Hybrid Environments and between Cloud Pods (e.g. Enterprise Planning and Budgeting Cloud Service (EPBCS) and Profitability and Cost Management Cloud Service (PCMCS) applications).
2. 2
• Introduction
• EPM Cloud and Automation
• EPM Automate
• Windows Batch Automation Architecture
• EPM Automate Algorithms
• Used Cases
• Challenges, Workarounds and other possibilities
• Q&A
Agenda
3. 3
• Summary
– Over 12 years of Enterprise Performance Management
(EPM) experience across on-premises, cloud and hybrid
deployments
– Design, Development and Deployment roles across
approx. 30 projects over the years
– Speaker at Oracle / Hyperion conferences such
HUGMN, DCOUAG, OAUG and ODTUG
• EPM Cloud Summary
– EPBCS design and deployment
– PCMCS development
– EPM Cloud products integration using Data
Management
– EPM Automation design and deployment
Introduction
Vatsal Gaonkar
vgaonkar@ranzal.com
https://www.linkedin.com/in/vats
algaonkar/
312.405.6242
4. 4
Comprehensive Business Solutions
Ranzal’s solutions drive improved business performance
through better decision making, strong customer
engagement and optimized operations
Deep Oracle Partnership Drives Customer Value Adaptable Deployment Models
Diverse Client Portfolio & Industry Expertise
Bio Tech and
Pharma
Medical
Supplies
Team Highlights
Multiple
Oracle ACEs
Seasoned delivery
team with avg ~6
yrs serving Ranzal
clients
Experienced
mgmt team with
avg 12 yrs leading
Ranzal
Introduction
8. 8
• EPM Automation
– What and Why?
• Integrated Process Automation examples
– Actuals Processing
• Reconcile Financials (ARCS)
• Consolidate Financials (FCCS)
• Load to EPM Planning (EPBCS)
– Actuals Allocation
• Reconcile Financials (ARCS)
• Consolidate Financials (FCCS)
• Load to Profitability (PCMCS)
• Allocation Model (PCMCS)
• Extract and Load to source (Data Management)
– Forecast Processing
• Reconcile Financials (ARCS)
• Consolidate Financials (FCCS)
• Load Actuals to EPM Planning (EPBCS)
• Process Forecast (EPBCS)
• Extract and Load to (PCMCS)
• Allocation Model (PCMCS)
• Extract and Load to Planning (Data Management)
The need for Automation
9. 9
• (E)PBCS Automation (PBCS platform)
– Metadata Load
• File based through Cloud Data Management
• Dimension files
– Data Load
• File Based through Cloud Data Management
• Import using pre-defined file format
– Data Extraction
• Cloud Data Management
• Migrations between instances
EPBCS
10. 10
• PCMCS Automation (non-PBCS platform)
– Metadata Load
• On-premises Essbase Cube
• File based through Cloud Data Management
– Data Load
• On-premises Essbase Cube
• File based through Cloud Data Management
– Data Extraction
• Cloud Data Management
• Level 0 query extraction
– Integration between Pods – Calc. Vs Reporting Pods for PCMCS
PCMCS
11. 11
• EPM Automate
– Utility download
– Windows
• Set Environment Variables - Path
– Linux
• Set Proxy Settings for this command line utility
– proxyHost
– proxyPort
– https.proxyHost
– https.proxyPort
– Currently doesn’t work with Single Sign-On (SSO)
– Use Native Administrator ID
– Presentation will concentrate on Windows Automation
– Oracle Documentation - https://docs.oracle.com/cloud/latest/epm-
common/CEPMA/toc.htm
EPM Automate
12. 12
Ground and Cloud(e.g. PCMCS)
EPM Cloud EPM Cloud
Master Data and
Data Staging
Data Source 1 Data Source 2 Data Source 3
Jump Box
Data Stage
Data Stage
LCM Application Copy
Data
Master Data
Legend
Historical Data Unicode to Ascii Conversion via PowerShell
13. 13
Ground and Cloud Movements
Windows Jump Box
EPBCS
PCMCS
Essbase
Data
Store
File
Systems
1. Files to Cloud
2. Essbase to Cloud
3. Datastore to Cloud
4. Cloud to Essbase
5. Cloud to Data Store
6. Cloud to Files
7. Cloud to Cloud
1
1,2,3
1,2,3
6
7
4,5,6
4,5,6
2 34 5
14. 14
• Jump Box
– Central Traffic Hub for Scripts, Triggers and Files
• Scheduler
– Windows Task Scheduler
– 3rd Party Schedulers (anything that can call Windows Batch)
• Centralized Function Library
• Trigger Automation
– Industries with Stringent Information Control Processes
• Error Exit Coding
– Exit Job on Critical Errors
• Cloud Data Management
– Dimension Build
– Data load and extractions
– Cross POD integration
Design Recommendations
Centralized Batch Library
Job -
Function Library (Central) –
Batches Integrator –
15. 15
• Why build Algorithms?
– Basis of Batch coding
– Troubleshooting large chunks of Code
– Enhancement enablement
– Communication / Documentation
• Swim Lane Algorithms
– Flow by Application specific artifacts
– Ease of use
Swim Lane Algorithms
16. 16
• EPBCS
– Dimension Sources
• File
• Existing Essbase application
(and no metadata integrator)
• EBS
– If not through Data
Management
• Zip Metadata files
• Create Import Jobs
Metadata Automation (EPBCS)
17. 17
• Trigger Automation
– Look for available metadata file
:RUN_SCHEDULED_PROCESS
REM ### Copy the trigger file into the scripts directory ###
COPY "%trigger_dir%%trigger_file%" "%scripts_dir%%trigger_file%" /Y
CALL %scripts_dir:~1,-1%%App_Name%_Load_EPBCS_Metadata.bat
:: DEL "%trigger_dir%%trigger_file%"
EXIT
Metadata Automation (EPBCS)
18. 18
• Metadata Automation
Call MetadataBuild.ps1
IF %ERRORLEVEL% GTR 0 ( ECHO - ** ERROR ** A critical error was
encountered while building dimension %Dimension%. Further processing of
the job will be suspended...>>%log_file% SET /A returncode+=1)
ELSE ( ECHO - Successfully built dimension %Dimension%...
>>%log_file%)
• Error Handling
:: * SAMPLE USAGE:
*:: * call:ErrHandler "Script failed" FALSE *::
**********************************************************************************************
***:ErrhandlerSET tempmsg=%1ECHO %tempmsg:~1,-1% >> %log_file%IF
"%2"=="TRUE" (EXIT 1) ELSE (goto:eof)
Metadata Automation (EPBCS)
The need for Automation
20. 20
ECHO Backing up Data before Dimension Build...>>%log_file%
call epmautomate login %admin_username% E:OracleAutomationPWFile%pwd_file% %url:~1,-1% "%domainname%" >>%log_file% 2>&1
call epmautomate exportqueryresults %App_Name% fileName="%App_Name%_Level0_DataExtract.txt" exportOnlyLevel0Flg=true >>%log_file% 2>&1
call:RollFile "%profitoutbox_dir:~1,-1%%App_Name%_Level0_DataExtract.txt" %log_retention% >>%log_file% 2>&1
call epmautomate downloadfile "profitoutbox%App_Name%_Level0_DataExtract.txt" >>%log_file% 2>&1
call:RollFile "%data_dir:~1,-1%%App_Name%_Level0_DataExtract.txt" %log_retention% >>%log_file% 2>&1
call:FileCopy "%profitoutbox_dir:~1,-1%%App_Name%_Level0_DataExtract.txt" "%data_dir:~1,-1%%App_Name%_Level0_DataExtract.txt" FALSE >>%log_file% 2>&1
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Run the metadata build based on which file exists in trigger directory
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
IF EXIST "%trigger_dir:~1,-1%%metadata_file1:~1,-1%" (
SET metadata_file=%metadata_file1%
CALL:RUN_PROCESS )
IF EXIST "%trigger_dir:~1,-1%%metadata_file2:~1,-1%" (
SET metadata_file=%metadata_file2%
CALL:RUN_PROCESS )
IF EXIST "%trigger_dir:~1,-1%%metadata_file3:~1,-1%" (
SET metadata_file=%metadata_file3%
CALL:RUN_PROCESS )
call epmautomate enableapp %App_Name% >>%log_file% 2>&1
call epmautomate deploycube %App_Name% isReplaceCube=true isRunNow=true comment="Redeployment after Dimensions Update" >>%log_file% 2>&1
call epmautomate deletefile profitinbox/%App_Name%_Level0_DataExtract.txt >>%log_file% 2>&1
call:Wait 10
call epmautomate uploadfile "%profitoutbox_dir:~1,-1%/%App_Name%_Level0_DataExtract.txt" profitinbox >>%log_file% 2>&1
call epmautomate loaddata %App_Name% dataLoadValue="OVERWRITE_EXISTING_VALUES" dataFileName="%App_Name%_Level0_DataExtract.txt" >>%log_file% 2>&1
call epmautomate runcalc %App_Name% %current_povyear%_%current_povscenario%_%current_povperiod% isClearCalculated=false isExecuteCalculations=true isRunNow=true subsetStart=999 subsetEnd=999
ruleSetName="Reporting Aggregation" ruleName="Optimize for Reporting Aggregation" exeType="RULESET_SUBSET" comment="Cube Aggregation" stringDelimiter="_" >>%log_file% 2>&1
call epmautomate logout >>%log_file% 2>&1
IF EXIST %trigger_dir:~1,-1%%App_Name:~0,-4%MetadataBuildVariables.txt DEL %trigger_dir:~1,-1%%App_NamE:~0,-4%MetadataBuildVariables.txt >>%log_file% 2>&1
ECHO. >> %log_file%
Metadata Automation (PCMCS)
21. 21
:: *************************************************************************************************
:: ***** RUN PROCESS *****
:: *************************************************************************************************
:: * SAMPLE USAGE: *
:: * call:RUN_PROCESS *
:: *************************************************************************************************
:RUN_PROCESS
ECHO Archive and copy data file...
call:RollFile "%data_dir:~1,-1%%metadata_file:~1,-1%" %file_retention%
call:SilentCopy "%trigger_dir:~1,-1%%metadata_file:~1,-1%" "%data_dir:~1,-1%%metadata_file:~1,-1%" FALSE
ECHO File copy and archival completed.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Build Dimension
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
ECHO Building Dimensions in %url%...>>%log_file%
call epmautomate deletefile profitinbox/%metadata_file:~1,-1% >>%log_file% 2>&1
call:Wait 10
call epmautomate uploadfile "%data_dir:~1,-1%%metadata_file:~1,-1%" profitinbox >>%log_file% 2>&1
call:Wait 10
SET error_message="There were errors during the Metadata load process."
SET /A returncode=0
call:LoadDimensionCalcPod %App_Name% %metadata_file:~1,-5%
IF %returncode% GTR 0 (call:ErrHandler %error_message% TRUE) ELSE (ECHO Dimension %metadata_file:~1,-5% built. >>%log_file%)
call:Wait 10
DEL "%trigger_dir:~1,-1%%metadata_file:~1,-1%" >>%log_file% 2>&1
goto:eof
Metadata Automation (PCMCS)
22. 22
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Migrate from Source to Target
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
call:EchoTimestamp
ECHO Backing up %source_environment% before Importing Snapshot...>>%log_file%
call epmautomate login %source_admin_username% C:OracleAutomationPWFile%source_pwd_file% %source_environment:~1,-1% "%domainname%" >>%log_file% 2>&1
SET error_message="There were errors during exporting the snapshot from %source_environment%.“
SET /A returncode=0
call:EXPORTSOURCESNAPSHOT
IF %returncode% GTR 0 (call:ErrHandler %error_message% TRUE)
ELSE (ECHO Snapshot Exported. >>%log_file%)
call epmautomate logout >>%log_file% 2>&1
ECHO Copying Snapshot from %source_environment% to %target_environment%...>>%log_file%
call epmautomate login %target_admin_username% C:OracleAutomationPWFile%target_pwd_file% %target_environment:~1,-1% "%domainname%" >>%log_file% 2>&1
call epmautomate deletefile %App_Name%_Exp >>%log_file% 2>&1
SET error_message="There were errors during copying the snapshot from %source_environment% to %target_environment%.“
SET /A returncode=0
call:COPYSNAPSHOT %App_Name%_Exp
IF %returncode% GTR 0 (call:ErrHandler %error_message% TRUE) ELSE (ECHO Snapshot Copied. >>%log_file%)
ECHO Exporting Data Grants from %target_environment%...>>%log_file%
call epmautomate exportsnapshot %App_Name%_DataGrants >>%log_file% 2>&1
call epmautomate logout >>%log_file% 2>&1
ECHO Recreate %target_environment% from %source_environment%...>>%log_file%
call epmautomate login %target_admin_username% C:OracleAutomationPWFile%target_pwd_file% %target_environment:~1,-1% “%domainname%" >>%log_file% 2>&1
call epmautomate recreate -f >>%log_file% 2>&1
call epmautomate importsnapshot %App_Name%_Exp >>%log_file% 2>&1
call epmautomate importsnapshot %App_Name%_DataGrants >>%log_file% 2>&1
call epmautomate logout >>%log_file% 2>&1
ECHO. >> %log_file%
Migration between Environments
23. 23
• Use of Cloud Data Management
– File management anywhere on the cloud
• Delete
• Upload
• Process
– EBS Integration available
• Trigger file processing
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Run the data load based on which file exists in trigger directory
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::I
F EXIST "%trigger_dir%%data_file1%" ( SET data_file=%data_file1% CALL:RUN_PROCESS )
IF EXIST "%trigger_dir%%data_file2%" ( SET data_file=%data_file2% CALL:RUN_PROCESS )
IF EXIST "%trigger_dir%%data_file3%" ( SET data_file=%data_file3% CALL:RUN_PROCESS )
IF EXIST "%trigger_dir%%data_file4%" ( SET data_file=%data_file4% CALL:RUN_PROCESS )
IF EXIST "%trigger_dir%%data_file5%" ( SET data_file=%data_file5% CALL:RUN_PROCESS )
Data Load (PCMCS)
24. 24
• Load Process based on the file availability
– Points of View in PCMCS
– CDM Import Mode
– Target Export Model – REPLACE VS STORE_DATA
ECHO Archive and copy data file...
call:RollFile "%data_dir:~1,-1%%data_file:~1,-1%" %file_retention%
call:FileCopy "%trigger_dir:~1,-1%%data_file:~1,-1%" "%data_dir:~1,-1%%data_file:~1,-1%" FALSE
ECHO File copy and archival completed.
:: Delete original file from inbox
call epmautomate deletefile inbox/File_PCM/%data_file:~1,-1% >>%log_file% 2>&1
call:Wait 5
:: Upload the new file to inbox
call epmautomate uploadfile "%data_dir:~1,-1%%data_file:~1,-1%" inbox/File_PCM >>%log_file% 2>&1 ::
Load data
call epmautomate rundatarule FilePCMActuals %StartMonth%-%StartYear:~2% %EndMonth%-%EndYear:~2%
REPLACE STORE_DATA File_PCM/%data_file:~1,-1% >>%log_file% 2>&1
EPM Automate
25. 25
• EPBCS
– Metadata (without using Data Management)
• epmautomate importmetadata
• Required to Zip the file and create an Import Job
• Consumable file format
• Refresh Database
– Data
• Data Management Export (Loading to database) Mode
– epmautomate loaddata
– REPLACE VS STORE_DATA
– REPLACE in BSO VS ASO Plan Types
– STORE_DATA in BSO VS ASO Plan Types
• Data Extraction
– epmautomate exportqueryresults
– 5M cells queryresultlimit
Challenges
26. 26
• PCMCS
– Metadata (if Metadata management tool not available)
• Consumable file format
• On-ground essbase integration with PCMCS – data manipulation required
• Database Deployment – Incremental Vs. Replace Cube
– Data
• MDX extractions
• Data Management Export Mode (with data management rule)
– epmautomate rundatarule
– REPLACE VS STORE_DATA
– 5M cell queryresultlimit
• Data Extraction (with a query name)
– epmautomate exportqueryresults
– Level0 extract using exportqueryresults export 2GB files
– 5M cells queryresultlimit with PCMCS queries
Challenges
27. 27
• EPBCS
– Metadata Management – use of error exit coding
IF %returncode% GTR 0 (call:ErrHandler %error_message% TRUE) ELSE (ECHO Snapshot Copied. >>%log_file%)
:Errhandler
SET tempmsg=%1
ECHO %tempmsg:~1,-1% >> %log_file%
IF "%2"=="TRUE" (EXIT 1) ELSE (goto:eof)
– Data Management
• BSO Load - Always use STORE_DATA Export mode in conjunction with a clear business rule
• PCMCS
– Metadata Management
• Dimconvertor.pl
• Error exit coding
– Level 0 export with 2GB increments handling (Data handling as well)
SET /A Counter=n
:Loop
IF %counter%==0 (
call epmautomate uploadfile "%data_dir:~1,-1%/%App_Name%_Level0_DataExtract.txt" profitinbox
goto:eof)
ELSE (
SET /A tempcounter=%counter%
call epmautomate uploadfile "%data_dir:~1,-1%/%App_Name%_Level0_DataExtract_%counter%.txt" profitinbox
)
SET /A counter=%counter%-1
goto:Loop
Workarounds
29. 29
• Possibility 1
– Import to ARCS
– Reconcile Accounts
– Export and Import to FCCS
– Consolidate FCCS
– Export and Import to EPBCS
– Trend based planning in EPBCS using loaded Actuals from FCCS
• Possibility 2 (Used case for us already)
– Import to EPBCS Workforce
– Calculate Workforce
– Extract and Load to PCMCS
– Allocate Direct Labor and other workforce attributes
– Extract and Load to EPBCS for fully allocated Budget P&L
Other Automation
30. 30
• Data Management is the future
– EPM Automate for integrating cloud products
– Metadata management using CDM
• For Windows use PowerShell, if possible
• Ensure Error exit coding in automation design
• Get on EPM Automate as soon as possible
• Oracle Reference for EPM Automate Command library
https://docs.oracle.com/cloud/latest/epm-
common/CEPMA/epm_automate_command_ref_general.htm#CEPMA
-GUID-5400C490-264F-493A-9EAD-267AB2A320F1
Closing Thoughts
31. 31
8 Speaker Sessions
Monday, 6/11:
• 10:45am – 11:45am: Baha Mar's All In Bet on Red - The story of integrating data and master data with PBCS, FCCS and ARCS
• 2:30pm - 3:30pm: Visual Approach to Essbase Calcs: 2018
• 4:15pm - 5:15pm: Integrated Planning Using Enterprise Planning and Budgeting Cloud Service at Sims Metal Management
Tuesday, 6/12:
• 9:00am - 10:00am: FDMEE versus Cloud Data Management - The Real Story
• 10:15am - 11:15am: Edgewater Ranzal: Winning Strategies for Oracle Cloud Adoption: Should You Test Drive, Lease, or Buy?
• 2:15pm - 3:15pm: Why Should I Care About DVD? Blu-Ray is the New Thing, Right?
Wednesday, 6/13:
• 11:45am - 12:45pm: Putting Predictive Planning to Work
• 2:15pm - 3:15pm: EPM Automate - Automating Enterprise Performance Management Cloud Solutions
Visit us at Booth # 407