This describes a generalised and structured approach to defining a strategy for collecting (near or actual) real time, high volume data. The appproach can be applied to areas such as Telemetry, Big Data, Smart Metering and Internet of Things implementations and operations. This proposed structured approach is intended to ensure that complexity is understood and can be appropriately addressed at an early stage before problems become to embedded to be solved. Real time situational data gives rise to situational awareness and understanding which in turn presents opportunities for effective and rapid situational decisions. Real time situational data enables greater situational visibility which means increased operational intelligence.

1. Real Time Data Strategy And
Architecture
Alan McSweeney
http://ie.linkedin.com/in/alanmcsweeney

2. Real Time Data Collection Strategy And Architecture
Approach
• These notes are concerned with describing a generalised
approach to defining a strategy for collecting (near or
actual) real time, high volume data
• This is data is generated by sensors that is transmitted to a
central location for processing, reporting, analysis and
ultimately action
• Sensors can be regarded as logical or physical sources of
streams of measurement data
• (Near) real time data can be termed Telemetry
March 8, 2016 2

3. Real Time Data Collection Strategy And Architecture
Approach
• Approach adopted from TMForum Resource Domain
Frameworx eTOM (Enhanced Telecoms Operating Model)
Business Process Framework
• Approach has been generalised for real-time data and
telemetry
March 8, 2016 3

4. Real Time Data Collection Strategy And Architecture
• Collect data from range of data sources across the
organisation’s (internal and external) operating landscape
• Approach can be applied to collection of measurement
data from multiple sources and of multiple types through
“sensors”:
− Different entities interacting with the organisation and the
gathering of data on different actions and events
• Approach can be applied to areas such as Telemetry, Big
Data, Smart Metering and Internet of Things
implementations and operations
March 8, 2016 4

5. Real Time Data Collection Strategy And Architecture
March 8, 2016 5
Real Time Data Strategy
Telemetry
Strategy Big Data
Strategy
Internet Of
Things
Strategy
SmartX
Strategy
Digital
Strategy

6. Why Have A Real Time Data Collection Strategy And
Architecture?
• Real time situational data gives rise to situational
awareness and understanding which in turn presents
opportunities for effective and rapid situational decisions
− What is happening – usage, performance
− What can be improved
− What are the optimisation and productivity opportunities
• Real time situational data enables greater situational
visibility which means increased operational intelligence
March 8, 2016 6

7. Organisation Operating Landscape
March 8, 2016 7

8. Organisation Operating Landscape
• The operating landscape of the organisation defines the
number and type of interactions outside the organisation
• This operating landscape affects the decision on what and
how to measure
• Not all interactions with all entities are measured
• Need to be realistic about what can be collected and
processed
• Need to understand the need for sensors to collect data
March 8, 2016 8

9. Data Sensors – Combinations Of Options
• Sensor Type – Physical sensors are actual units such as RTUs
(Remote Telemetry Units) that measure and generate data
while Logical sensors are representations of data sources
• Sensor Ownership – Direct sensors are those installed and
maintained by the collecting organisation while Indirect
sensors are installed by a third-party
March 8, 2016 9
Sensor Type
Logical Physical
Sensor Ownership
Direct
• Web Site and App Activity
and Usage Data
• Internet of Things Devices
• Remote Real Time Unit
• Smart Devices
Indirect
• Third-Party Web Site and
App Activity and Usage
Data
• Third-Party Devices

10. Measurement Data Sensors
March 8, 2016 10




11. Measurement Data Sensors
• These can gather data for different measures using
multiple measurement techniques
• These can be regarded as collectors of any data – usage,
activity, performance, throughput
• Each measurement type will have a unit or dimension
• Logical representation of data collectors
• Need to decide on what can be measured, what to
measure and how to measure it
March 8, 2016 11

12. Measurement Data Sensors – Decide On What And
How To Measure
March 8, 2016 12
    
 










 

 

13. Real Time Data Architecture Complexity
March 8, 2016 13
Real Time Data Architectures
Tend To Focus On The Simplicity
Of A Possible Real Time Data
Collection Architecture …
… And Ignore The Complexity And
Difficulties Of Sensor Installation,
Operation, Maintenance, Errors,
Rework, Logistics, Service
Management, Data Volumes

14. Real Time Data Architecture Complexity
• Structured approach is intended to ensure that complexity
is understood and can be appropriately addressed at an
early stage before problems become to embedded to be
solved
March 8, 2016 14

15. Real Time Data Architecture Complexity
March 8, 2016 15
Don’t Let The Ignored But
Knowable And Addressable
Complexity Sink Your Real
Time Data
Programme/Initiative

16. Real Time Data Strategy And Architecture Issues
• What business benefits will a real time data strategy yield?
• How can the benefits be realised?
• What is the business case for investment in a real time data
strategy?
• What infrastructure, communications/connectivity, data and
application architectures are needed to support a real time data
strategy?
• What integration is required with existing applications?
• What skills, capabilities and changes does the organisation need to
adopt and exploit a real time data strategy?
• How is the real time data strategy managed and serviced?
• What solution and service providers and tools/platforms and
sourcing strategy should be selected?
• What are the privacy and security issues and requirements?
March 8, 2016 16

17. Evolution Of Real Time Data Architecture
March 8, 2016 17
Create Awareness
for Real Time Data
Investment
Undefined/unarticulated/uncertain real time
data strategy. Real time data processes are
ad hoc, focussed on individual solution and
outcomes vary widely.
CrawlWalkRunFly
Building Real Time
Data Investment
Foundation
Implement investment controls and develop
key foundational capabilities
Developing
Complete Real
Time Data
Portfolio
Comprehensive selection and control
processes with benefit and risk criteria
linked to strategy requirements
Improving Real
Time Data
Processes
Process evaluation techniques focus on
improvement of performance and
management
Leveraging
for Strategic
Outcomes
Real time data management, reporting and
analysis techniques are deployed for
strategic mission/business outcomes
Admitting
There must be a better way
Communicating
Establishing and
communicating business case
Governing
Making and implementing
effective investment
decisions
Managing
Processes, mechanisms and
metrics
Optimising
Sense and respond

18. Developing A Real Time Data Strategy – Generalised
High Level Steps
March 8, 2016 18
Real Time
Strategy And
Planning
Real Time
Capability
Delivery
Real Time
Development
And
Retirement
Real Time
Management
and
Operations
Support And
Readiness
Real Time
Provisioning
Real Time
Data
Collection
And
Distribution
Real Time
Problem
Management
Real Time
Performance
Management
Workforce
Management
Real Time
Data
Aggregation
and
Reporting
41 2 3 5
109876

19. Developing A Real Time Data Strategy – Generalised
High Level Steps
• Comprehensive set of steps from definition of what is required
from real time data to commissioning of real time data
collection facilities to effective use of collected data
• Not all steps are relevant to all real time data initiatives
− For example, if installation, commissioning, operation and maintenance
of physical sensors is not applicable then related steps will not be
required
• Represents an idealised organisation and process breakdown
across entire spectrum of real time data from strategy to
workforce management to device installation, data collection
and data usage and actioning
• Provides a basis for developing a work breakdown and an
implementation plan
• Represents a comprehensive structure that can be adapt to
meet its long-term real time data needs
March 8, 2016 19

20. Developing A Real Time Data Strategy – Generalised
High Level Steps 1 – 2
Step Scope
1. Real Time Strategy And
Planning
• Develop real time data strategy, policies and plans for the organisation governed by long-term
business, market, product and service needs directions
• Perform research and analysis to determine real time targets and strategies to reach the defined
targets
• Understand the real time data capabilities of the existing infrastructure
• Build real time data model
• Define approaches to real time data quality and real time data governance
• Define and agree the infrastructure needs based on market, product and service strategies of the
organisation
• Manage the capabilities of the suppliers and partners to develop and deliver new real time data
capabilities and detail the approach to the deployment of new and enhanced infrastructure
• Define the real time data implementation standards sought, key real time data capabilities
required, real time data support levels and approaches required, real time data design elements
to be developed, and real time data cost parameters and targets.
• Define the policies relating to technical real time data sensors and their implementation
2. Real Time Capability
Delivery
• Ensure that network, application and computing real time data facilities are deployed
• Provide the physical real time data capabilities necessary for the ongoing operations and long-
term well-being of the organisation and ensure the basis on which all real time data capabilities
and services will be constructed
• Plan real time data resource supply logistics
• Plan real time data sensor installation
• Verify the real time data sensor installation
• Handover real time data capabilities to operations
March 8, 2016 20

21. Developing A Real Time Data Strategy – Generalised
High Level Steps 3 – 6
Step Scope
3. Real Time
Development And
Retirement
• Develop new or enhance existing technologies and associated real time data types applying the
capability definition or requirements defined by the Real Time Strategy And Planning step
• Decide on acquisition of real time data resources from third parties
• Retire or remove technology and associated real time data resource types that are no longer
needed by the organisation
4. Real Time
Management and
Operations Support And
Readiness
• Manage the types of real time data resources and ensure that necessary application, computing
and network facilities are available and ready to implement and manage resource instances
5. Workforce
Management
• Manage the direct and indirect personnel who perform work assignments or work orders relating
to real time data resources installation, commissioning and maintenance as well as managing the
actual activity being performed
• Report and monitor activities
• Establish, manage and allocate work assignments to direct and indirect personnel
• Establish and manage priority and urgent assignment capabilities to allow for modification of
work assignments as required to meet urgent and high priority conditions
6. Real Time Provisioning • Allocate, install, configure, activate and test of real time data resources to meet the defined
requirements
• Resolve real time resource capacity issues, availability issues or failure conditions
• Configure and activate physical and/or logical real time resources
• Update of real time resource register database to reflect that the specific real time resource has
been allocated, modified or recovered
March 8, 2016 21

22. Developing A Real Time Data Strategy – Generalised
High Level Steps 7 – 8
Step Scope
7. Real Time Data
Collection And
Distribution
• Collect and distribute management information and real time data between data sources and
service instances and other organisation functions and processes
• Work with the real time data resource and service instances to collect usage, network and
technology events and other management information for distribution to other processes within
the organisation
• Handle and process command, query and other management information for distribution to
resource and service instances
• Process the data and management information through actions such as filtering, aggregation,
formatting, transformation and correlation of the information before presentation to other
processes, real time data instances or service instances
• Perform usage reporting, fault and performance analysis, service quality management analysis,
resource performance analysis of resources and services
8. Real Time Problem
Management
• Manage real time data resource problems including security events
• Detect, analyse, manage and report on resource alarm event notifications
• Initiate and manage real time data resource problems reports
• Perform real time data resource problem localization analysis and resolve problems
• Reporting progress on resource trouble reports to other processes
• Assign and track real time data resource problem testing and resolution activities
• Managing real time data resource problem urgent conditions
March 8, 2016 22

23. Developing A Real Time Data Strategy – Generalised
High Level Steps 9 – 10
Step Scope
9. Real Time Performance
Management
• Manage, track, monitor, analyse and report on the performance of real time data resources
• Identify real time data resource performance disruptions or a service performance disruptions
10. Real Time Data
Aggregation and
Reporting
• Manage real time data resource events by correlating and formatting them into a usable format
• Report of real time data resource data
March 8, 2016 23

24. Sample Expansion – Step 1 – Real Time Strategy And
Planning – Activities 1.1 – 1.7
March 8, 2016 24
Step Scope
1.1 Gather And Analyse Real
Time Data Information
Research and analyse customer, technology, competitor and marketing information to identify
new real time data requirements and industry real time data capabilities and availability.
1.2 Manage Real Time Data
Research
Manage internally driven research investigations and activities which are used to provide detailed
technical assessment or investigation of new and emerging real time data capabilities.
1.3 Establish Real Time Data
Strategy And Architecture
Establish the real time data strategies based on market trends, future needs, technical
capabilities and addressing shortcomings in existing real time data support.
1.4 Define Real Time Data
Support Strategies
Define the principles, policies and performance standards for the operational organisation
providing real time data support.
1.5 Produce Real Time Data
Business Plans
• Develop and deliver annual and multi-year real time data plans in support of services,
products and offers that include volume forecasts, negotiation for required levels of resources
and budgets.
• Obtain real time data development and management as well as supply chain commitment
and executive approval for the plans.
• Identify the impacts that new or modified real time data infrastructure will cause on the
installed infrastructure and workforce and establish the functions and benefits that new or
modified real time data will provide to users.
1.6 Develop Real Time Data
Partnership Requirements
Identify the requirements for real time data capabilities to be sourced from partners or suppliers,
and any real time data capabilities to be delivered internally to the organisation.
1.7 Gain Enterprise
Commitment To Real Time
Data Plans
Obtain organisation commitment to the resource strategy and business plans including all aspects
of identification of stakeholders and negotiation to gain stakeholder approval.

25. Sample Expansion – Step 1 - Real Time Strategy And
Planning – Activity 1.5 – Tasks 1.5.1 – 1.5.5
March 8, 2016 25
Step Scope
1.5.1 Develop And Deliver
Annual/Multi Year Real
Time Data Business Plans
Develop and deliver annual/multi year real time data business plans focus on developing and
delivering annual and multi-year real time data in support of services, products and offers that
include volume forecasts, negotiation for required levels of resources and budgets, gaining real time
data development and management as well as supply chain commitment and executive approval for
the plans.
1.5.2 Forecast High Level
Real Time Data Demand
And Capture New
Opportunities
Forecast real time data demand and capture new opportunities processes ensures that budgets are
assigned which allow the organisation to implement the real time data capabilities and capacity
necessary for the future needs of their customers and potential customers.
1.5.3 Assess Impact Of
Real Time Data Business
Plans
Asses impact of real time data business plan processes assess the impacts that new or modified real
time data infrastructure will cause on the installed infrastructure and workforce, and establish the
functions and benefits that new or modified real time data will provide to users
1.5.4 Identify Timetables
For New Real Time Data
Capability Introduction
Identify timetables for new real time data capability introduction
1.5.5 Identify Logistics For
New Real Time Data
Capability Introduction
Identify logistics for new real time data capability introduction

26. Real Time Data Quality And Data Governance
• Real-time data is inherently:
− High Volume – lots of sensors generating lots of data
− Noisy – lots of variation, statistical noise, inaccuracies, sensor drift, errors,
incorrect calibration
− Changing – data landscape subject to substantial change along the dimensions
of data sources, volumes, types
− Inconsistent – different sensor types measuring different values with different
units of measure and at different intervals
− Heterogeneous – very mixed data sources
− Non-standardised – multiple, emerging, overlapping standards and
approaches
• Sophisticated approach to data quality and data governance must be
embedded in any real-time architecture
• Traditional approach of data collection storage and analysis may
need to change to handle data volumes and quality – data filtering,
quality, summarisation and transformation component
March 8, 2016 26

27. Real Time Data Principles
March 8, 2016 27
To manage and utilise real time information as a strategic asset
To implement processes, policies, infrastructure and solutions to govern, protect,
maintain and use real time information
To make relevant and correct real time information available in all business
processes and IT systems for the right people in the right context at the right time
with the appropriate security and with the right quality
To exploit real time information in business decisions, processes and relations

28. Real Time Data And Data Governance And Data
Quality
• Data Quality - measure, assess, improve, and ensure the
fitness of data for use
• Data Governance - authority and control over the
management of data assets
March 8, 2016 28

29. Real Time Data Governance
• Core function of real time data management
• Interacts with and influences each of the surrounding ten data
management functions
• Data governance is the exercise of authority and control
(planning, monitoring, and enforcement) over the management
of data assets
• Data governance function guides how all other data
management functions are performed
• High-level, executive data stewardship
• Data governance is not the same thing as IT governance
• Data governance is focused exclusively on the management of
data assets
March 8, 2016 29

30. Real Time Data Governance – Definition and Goals
• Definition
− The exercise of authority and control (planning, monitoring, and
enforcement) over the management of data assets
• Goals
− To define, approve, and communicate data strategies, policies,
standards, architecture, procedures, and metrics
− To track and enforce regulatory compliance and conformance to
data policies, standards, architecture, and procedures
− To sponsor, track, and oversee the delivery of data management
projects and services
− To manage and resolve data related issues
− To understand and promote the value of data assets
March 8, 2016 30

31. Real Time Data Governance Structure
March 8, 2016 31
Real Time Data
Governance Framework
Real Time Data
Architecture to Implement
Data Governance
Real Time Data
Infrastructure to
Implement Data
Architecture
Real Time Data
Operations to Manage
Data Infrastructure

32. Real Time Data Governance Activities
March 8, 2016 32
Real Time Data Governance
Real Time Data Management Planning
Understand Strategic Enterprise Real Time Data
Needs
Develop and Maintain the Real Time Data Strategy
Establish Real Time Data Professional Roles and
Organisations
Identify and Appoint Real Time Data Stewards
Establish Real Time Data Governance and
Stewardship Organisations
Develop and Approve Real Time Data Policies,
Standards, and Procedures
Review and Approve Real Time Data Architecture
Plan and Sponsor Real Time Data Management
Projects and Services
Estimate Real Time Data Asset Value and Associated
Costs
Real Time Data Management Control
Supervise Real Time Data Professional Organisations
and Staff
Coordinate Real Time Data Governance Activities
Manage and Resolve Real Time Data Related Issues
Monitor and Ensure Regulatory Compliance
Monitor and Enforce Conformance withReal Time
Data Policies, Standards and Architecture
Oversee Real Time Data Management Projects and
Services
Communicate and Promote the Value of Real Time
Data Assets

33. Real Time Data Governance Inputs And Outputs
March 8, 2016 33
•Business Goals
•Business Strategies
•IT Objectives
•IT Strategies
•Data Needs
•Data Issues
•Regulatory Requirements
Inputs
•Business Executives
•IT Executives
•Data Stewards
•Regulatory Bodies
Suppliers
•Intranet Website
•E-Mail
•Metadata Tools
•Metadata Repository
•Issue Management Tools
•Data Governance KPI
•Dashboard
Tools
•Executive Data Stewards
•Coordinating Data Stewards
•Business Data Stewards
•Data Professionals
•DM Executive
•CIO
Participants
•Data Policies
•Data Standards
•Resolved Issues
•Data Management Projects and
Services
•Quality Data and Information
•Recognised Data Value
Primary Deliverables
•Data Producers
•Knowledge Workers
•Managers and Executives
•Data Professionals
•Customers
Consumers
•Data Value
•Data Management Cost
•Achievement of Objectives
•# of Decisions Made
•Steward Representation /
Coverage
•Data Professional Headcount
•Data Management Process
Maturity
Metrics
Real Time Data
Governance

34. Real Time Data Quality Management
• Critical support process in organisational change management
• Data quality is synonymous with information quality since poor
data quality results in inaccurate information and poor
business performance
• Data cleansing may result in short-term and costly
improvements that do not address the root causes of data
defects
• More rigorous data quality program is necessary to provide an
economic solution to improved data quality and integrity
• Institutionalising and operationalising processes for data
quality oversight, management, and improvement hinges on
identifying the business needs for quality data and determining
the best ways to measure, monitor, control, and report on the
quality of data
• Continuous process for defining the parameters for specifying
acceptable levels of data quality to meet business needs, and
for ensuring that data quality meets these levels
March 8, 2016 34

35. Real Time Data Quality Management – Definition
and Goals
• Definition
− Planning, implementation, and control activities that apply quality
management techniques to measure, assess, improve, and ensure
the fitness of data for use
• Goals
− To measurably improve the quality of data in relation to defined
business expectations
− To define requirements and specifications for integrating data
quality control into the system development lifecycle
− To provide defined processes for measuring, monitoring, and
reporting conformance to acceptable levels of data quality
March 8, 2016 35

36. Real Time Data Quality Management Inputs And
Outputs
March 8, 2016 36
•Business Requirements
•Data Requirements
•Data Quality Expectations
•Data Policies and Standards
•Business metadata
•Technical metadata
•Data Sources and Data Stores
Inputs
•External Sources
•Regulatory Bodies
•Business Subject Matter Experts
•Information Consumers
•Data Producers
•Data Architects
•Data Modelers
Suppliers
•Data Profiling Tools
•Statistical Analysis Tools
•Data Cleansing Tools
•Data Integration Tools
•Issue and Event Management
Tools
Tools
•Data Quality Analysts
•Data Analysts
•Database Administrators
•Data Stewards
•Other Data Professionals
•DRM Director
•Data Stewardship Council
Participants
•Improved Quality Data
•Data Management
•Operational Analysis
•Data Profiles
•Data Quality Certification
Reports
•Data Quality Service Level
Agreements
Primary Deliverables
•Data Value Statistics
•Errors / Requirement Violations
•Conformance to Expectations
•Conformance to Service Levels
Metrics
Real Time Data
Quality
Management
•Data Stewards
•Data Professionals
•Other IT Professionals
•Knowledge Workers
•Managers and Executives
Customers
Consumers

37. Real Time Data Quality Plan Definition Activities
March 8, 2016 37
Real Time Data Quality Plan Definition
1. Develop and Promote Data Quality
Awareness
2. Define Data Quality Requirements
3. Profile, Analyse and Assess Data
Quality
4. Define Data Quality Metrics
5. Define Data Quality Business Rules
6. Test and Validate Data Quality
Requirements
7. Set and Evaluate Data Quality Service
Levels
8. Continuously Measure and Monitor
Data Quality
9. Manage Data Quality Issues
10. Clean and Correct Data Quality
Defects
11. Design and Implement Operational
Data Quality Management Procedures
12. Monitor Operational Data Quality
Management Procedures and
Performance

38. Developing A Real Time Data Strategy – Generalised
High Level Steps
• 10 high level steps with activities and tasks
• Over 180 detailed tasks for a complete view of work required
• Comprehensive set of steps from definition of what is required from real time data to
commissioning of real time data collection facilities to effective use of collected data
March 8, 2016 38
Real Time Data
Strategy and
Implementation
1 Real Time
Data Strategy
And Planning
Activities
Tasks
2 Real Time
Data
Capability
Delivery
3 Real Time
Data
Development
And
Retirement
4 Real Time
Data
Management
and
Operations
Support And
Readiness
5 Workforce
Management
6 Real Time
Data
Provisioning
7 Real Time
Data
Collection
And
Distribution
8 Real Time
Data Trouble
Management
9 Real Time
Data
Performance
Management
10 Real
Time Data
Aggregation
and
Reporting

39. Real Time Data Strategy and Implementation –
Organisation, Function And Process Structure – Steps 1-10
March 8, 2016 39
Real Time Data
Strategy and
Implementation
1 Real Time Data
Strategy And
Planning
1.1 Gather And
Analyse Real Time
Data Information
1.2 Manage Real
Time Data
Research
1.3 Establish Real
Time Data
Strategy And
Architecture
1.4 Define Real
Time Data
Support Strategies
1.5 Produce Real
Time Data
Business Plans
1.6 Develop Real
Time Data
Partnership
Requirements
1.7 Gain
Enterprise
Commitment To
Real Time Data
Plans
2 Real Time Data
Capability Delivery
2.1 Map And
Analyse Real Time
Data
Requirements
2.2 Capture Real
Time Data
Capability
Shortfalls
2.3 Gain Real Time
Data Capability
Investment
Approval
2.4 Design Real
Time Data
Capabilities
2.5 Enable Real
Time Data
Support And
Operations
2.6 Manage Real
Time Data
Capability Delivery
2.7 Manage
Handover To Real
Time Data
Operations
3 Real Time Data
Development And
Retirement
3.1 Gather And
Analyse New Real
Time Data Ideas
3.2 Assess
Performance Of
Existing Real Time
Data
3.3 Develop New
Real Time Data
Business Proposal
3.4 Develop
Detailed Real
Time Data
Specifications
3.5 Manage Real
Time Data
Development
3.6 Manage Real
Time Data
Deployment
3.7 Manage Real
Time Data Exit
4 Real Time Data
Management and
Operations
Support And
Readiness
4.1 Enable Real
Time Data
Provisioning
4.2 Enable Real
Time Data
Performance
Management
4.3 Support Real
Time Data Trouble
Management
4.4 Enable Real
Time Data
Collection And
Distribution
4.5 Manage Real
Time Data
Inventory
4.6 Manage
Logistics
5 Workforce
Management
5.1 Manage
Schedules and
Appointments
5.2 Plan and
Forecast
Workforce
Management
5.3 Administer
and Configure
Workforce
Management
5.4 Report
Workforce
Management
5.5 Manage Work
Order Lifecycle
6 Real Time Data
Provisioning
6.1 Allocate And
Install Real Time
Data
6.2 Configure And
Activate Real Time
Data
6.3 Test Real Time
Data
6.4 Track And
Manage Real Time
Data Provisioning
6.5 Report Real
Time Data
Provisioning
6.6 Close Real
Time Data Order
6.7 Issue Real
Time Data Orders
6.8 Recover Real
Time Data
7 Real Time Data
Collection And
Distribution
7.1 Collect
Management
Information And
Data
7.2 Process
Management
Information And
Data
7.3 Distribute
Management
Information And
Data
7.4 Audit
Management And
Security Data
Collection And
Distribution
8 Real Time Data
Trouble
Management
8.1 Survey And
Analyse Real Time
Data Trouble
8.2 Localise Real
Time Data Trouble
8.3 Correct And
Resolve Real Time
Data Trouble
8.4 Track And
Manage Real Time
Data Trouble
8.5 Report Real
Time Data Trouble
8.6 Close Real
Time Data Trouble
Report Flow
8.7 Create Real
Time Data Trouble
Report
9 Real Time Data
Performance
Management
9.1 Monitor Real
Time Data
Performance
9.2 Analyse Real
Time Data
Performance
9.3 Control Real
Time Data
Performance
9.4 Report Real
Time Data
Performance
9.5 Create Real
Time Data
Performance
Degradation
Report
9.6 Track And
Manage Real Time
Data Performance
Resolution
9.7 Close Real
Time Data
Performance
Degradation
Report
10 Real Time Data
Aggregation and
Reporting
10.1 Aggregate
Real Time Data
Records
10.2 Report Real
Time Data
Records

40. Step 1 – Real Time Data Strategy And Planning –
Processes And Functions Details
March 8, 2016 40
1 Real Time Data
Strategy And
Planning
1.1 Gather And
Analyse Real Time
Data Information
1.1.1 Gather Real
Time Data
Information
1.1.2 Analyse New
Real Time Data
Requirements
1.1.3 Analyse To
Develop
New/Enhance Real
Time Data
Requirements
1.2 Manage Real
Time Data Research
1.2.1 Manage Real
Time Data Research
Investigations
1.2.2 Manage
Administration Of
Real Time Data
Research
1.2.3 Define Real
Time Data Research
Assessment
Methodologies
1.3 Establish Real
Time Data Strategy
And Architecture
1.3.1 Establish Real
Time Data Strategy
1.3.2 Develop Real
Time Data Strategy
1.3.3 Establish Real
Time Data Delivery
Goals
1.3.4 Establish Real
Time Data
Implementation
Policies
1.4 Define Real
Time Data Support
Strategies
1.4.1 Define Real
Time Data Support
Principles
1.4.2 Define Real
Time Data Support
Policies
1.4.3 Define Real
Time Data Support
Performance
Standards
1.5 Produce Real
Time Data Business
Plans
1.5.1 Develop And
Deliver
Annual/Multi Year
Real Time Data
Business Plans
1.5.2 Forecast High
Level Real Time
Data Demand And
Capture New
Opportunities
1.5.3 Assess Impact
Of Real Time Data
Business Plans
1.5.4 Identify
Timetables For New
Real Time Data
Capability
Introduction
1.5.5 Identify
Logistics For New
Real Time Data
Capability
Introduction
1.6 Develop Real
Time Data
Partnership
Requirements
1.6.1 Identify The
Requirements For
Real Time Data
Capabilities
1.6.2 Recommend
Real Time Data
Partnership
1.6.3 Determine
Extent Of Real Time
Data Capabilities
Sourcing
1.7 Gain Enterprise
Commitment To
Real Time Data
Plans
1.7.1 Identify
Stakeholders To
Real Time Data
Strategy And Real
Time Data Plans
1.7.2 Gain Real Time
Data Strategy And
Real Time Data
Plans Stakeholders
Approval
1.7.3 Gain
Enterprise
Commitment To
Real Time Data
Strategy And Real
Time Data Plans

41. Step 2 - Real Time Data Capability Delivery –
Processes And Functions Details
March 8, 2016 41
2 Real Time Data
Capability Delivery
2.1 Map And
Analyse Real Time
Data Requirements
2.1.1 Capture Real
Time Data Demand
And Performance
Requirements
2.1.2 Agree Real
Time Data
Infrastructure
Requirements
2.2 Capture Real
Time Data Capability
Shortfalls
2.2.1 Capture Real
Time Data Capacity
Shortfalls
2.2.2 Capture Real
Time Data
Performance
Shortfalls
2.2.3 Capture Real
Time Data
Operational Support
Shortfalls
2.3 Gain Real Time
Data Capability
Investment
Approval
2.3.1 Develop Real
Time Data Capability
Investment
Proposals
2.3.2 Approve Real
Time Data Capability
Investment
2.4 Design Real Time
Data Capabilities
2.4.1 Define Real
Time Data Capability
Requirements
2.4.2 Specify Real
Time Data Capability
Infrastructure
2.4.3 Select Real
Time Data Capability
At Other Parties
2.5 Enable Real Time
Data Support And
Operations
2.5.1 Design Real
Time Data
Operational Support
Process
Improvements
2.5.2 Identify Real
Time Data Support
Groups, Skills And
Training
2.5.3 Identify Real
Time Data Support
Requirements
2.6 Manage Real
Time Data Capability
Delivery
2.6.1 Co-Ordinate
Real Time Data
Capability Delivery
2.6.2 Ensure Real
Time Data Capability
Quality
2.6.3 Manage
Commissioning Of
New Real Time Data
Infrastructure
2.6.4 Establish Real
Time Data Capability
Sourcing
2.7 Manage
Handover To Real
Time Data
Operations
2.7.1 Co-Ordinate
Real Time Data
Operational
Handover
2.7.2 Validate Real
Time Data
Infrastructure
Design
2.7.3 Ensure Real
Time Data Handover
Support

42. Step 3 - Real Time Data Development And
Retirement – Processes And Functions Details
March 8, 2016 42
3 Real Time Data
Development And
Retirement
3.1 Gather And
Analyse New Real
Time Data Ideas
3.1.1 Gather Real
Time Data
Information
3.1.2 Analyse Real
Time Data Classes
3.1.3 Develop Real
Time Data Classes
3.2 Assess
Performance Of
Existing Real Time
Data
3.3 Develop New
Real Time Data
Business Proposal
3.3.1 Develop Real
Time Data Business
Proposal
3.3.2 Gain Real Time
Data Business
Proposal Approval
3.4 Develop
Detailed Real Time
Data Specifications
3.4.1 Develop
Detailed Real Time
Data Technical
Specifications
3.4.2 Develop
Detailed Real Time
Data Support
Specifications
3.4.3 Develop
Detailed Real Time
Data Operational
Specifications
3.4.4 Develop
Detailed Real Time
Data Manuals
3.5 Manage Real
Time Data
Development
3.5.1 Identify
Required Processes
And Procedures For
Real Time Data
3.5.2 Develop
Required Processes
And Procedures For
Real Time Data
3.5.3 Develop
Service And
Operational
Agreements For
Real Time Data
3.5.4 Gain Service
And Operational
Agreements
Approval For Real
Time Data
3.5.5 Produce
Supporting
Documentation And
Training Packages
For Real Time Data
3.6 Manage Real
Time Data
Deployment
3.6.1 Manage Real
Time Data Process
And Procedure
Implementation
3.6.2 Manage Real
Time Data
Operational Staff
Training
3.6.3 Develop Real
Time Data
Supplier/Partner
Operational Support
3.6.4 Manage Real
Time Data
Acceptance Testing
3.7 Manage Real
Time Data Exit
3.7.1 Identify
Unviable Real Time
Data
3.7.2 Identify
Impacted Real Time
Data Customers
3.7.3 Develop Real
Time Data Transition
Strategies
3.7.4 Manage Real
Time Data Exit
Process

43. Step 4 - Real Time Data Management and Operations
Support And Readiness – Processes And Functions Details
March 8, 2016 43
4 Real Time Data Management
and Operations Support And
Readiness
4.1 Enable Real Time Data
Provisioning
4.1.1 Plan And Forecast Real
Time Data Infrastructure
Requirements And Manage
Capacity Planning
4.1.2 Establish, Manage, And
Develop Organisetion, Tools And
Processes
4.1.3 Develop And Implement
Capacity And Operational Rules
And Procedures
4.1.4 Perform Acceptance Test
And Address And Monitor The
Change
4.1.5 Track And Supervise The
Rollout Of New And/Or
Modified Infrastructure
4.1.6 Monitor, Report And
Release Mgmt. Of Real Time
Data Infrastructure And Capacity
Utilisation
4.1.7 Optimise Existing Real
Time Data Infrastructure
Utilisation
4.1.8 Track, Monitor And Report
Real Time Data Provisioning
4.1.9 Update Inventory Record
4.2 Enable Real Time Data
Performance Management
4.2.1 Monitor And Manage
Regulatory Issues
4.2.2 Establish And Maintain
Performance Threshold
Standards
4.2.3 Undertake Performance
Trend Analysis
4.2.4 Monitor And Analyse Real
Time Data Performance Reports,
And Identify Issues
4.2.5 Correlate The Performance
Problem Reports And Manage
Inventory Repository
4.2.6 Manage Real Time Data
Performance Data Collection
4.2.7 Establish, Maintain And
Manage The Support Plans
4.2.8 Assess And Report Real
Time Data Performance
Management Processes
4.2.9 Provide Supporting
Procedures And Quality
Management Support
4.3 Support Real Time Data
Trouble Management
4.3.1 Manage Real Time Data
Trouble And Performance Data
Collection
4.3.2 Manage Real Time Data
Infrastructure, Provisioning And
Preventive Maintenance
Schedules
4.3.3 Create Report
4.3.4 Establish Warehouse And
Manage Spares Including Other
Parties
4.3.5 Track, Monitor And
Manage Real Time Data Trouble
Management Processes
Including Other Parties
4.3.6 Provide Support For Real
Time Data Trouble Management
And Support Service Problem
Management Processes
4.4 Enable Real Time Data
Collection And Distribution
4.4.1 Manage And Administer
Real Time Data Collection And
Distribution
4.4.2 Manage Real Time Data
Storage Facilities And Associated
Processes
4.4.3 Track, Monitor And Report
Real Time Data Collection
Processes And Capabilities
4.4.4 Identify Data Collection
Issues And Report
4.5 Manage Real Time Data
Inventory
4.5.1 Manage Real Time Data
Inventory Database And
Processes
4.5.2 Track And Monitor Real
Time Data Repository
Capabilities
4.5.3 Identify Repository Issues
And Provide Reports And
Warnings
4.6 Manage Logistics
4.6.1 Manage Warehousing
4.6.2 Manage Orders
4.6.3 Track And Monitor
Logistics And Manage Real Time
Data Inventory
4.6.4 Identify Logistic Issues And
Provide Reports

44. Step 5 - Workforce Management – Processes And
Functions Details
March 8, 2016 44
5 Workforce Management
5.1 Manage Schedules And
Appointments
5.1.1 Workforce
Management Schedule
5.1.2 Determine Work
Schedule
5.1.3 Manage Reservations
5.1.4 Manage Appointments
5.2 Plan And Forecast
Workforce Management
5.2.1 Forecast Demand
5.2.2 Forecast Workforce
Availability
5.2.3 Adjust Durations
5.3 Administer And
Configure Workforce
Management
5.3.1 Configure Work
Catalog
5.3.2 Administer Human
Real Time Data Catalog
5.3.3 Administer
Organisation's Catalog
5.3.4 Administer Tools And
Materials Catalog
5.3.5 Configure Skill Catalog
5.3.6 Configure Schedules
5.3.7 Administer
Registration And Access
5.3.8 Configure Logging And
Audit
5.4 Report Workforce
Management
5.5 Manage Work Order
Lifecycle
5.5.1 Issue Work Order
5.5.2 Analyse And
Decompose Work Order
5.5.3 Assign Task
5.5.4 Dispatch Task
5.5.5 Track And Manage
Work Order
5.5.6 Close Work Order
5.5.7 Report On Work Order

45. Step 6 - Real Time Data Provisioning – Processes And
Functions Details
March 8, 2016 45
6 Real Time Data
Provisioning
6.1 Allocate And
Install Real Time
Data
6.1.1 Determine
Real Time Data
Availability
6.1.2 Reserve
Real Time Data
6.1.3 Release
Real Time Data
6.1.4 Allocate
Real Time Data
6.1.5 Install And
Commission
Real Time Data
6.2 Configure
And Activate
Real Time Data
6.2.1 Configure
Real Time Data
6.2.2 Implement
Real Time Data
6.2.3 Activate
Real Time Data
6.3 Test Real
Time Data
6.3.1 Test
Specific Real
Time Data
6.3.2 Develop
Test Plans
6.3.3 Capture
Test Results
6.4 Track And
Manage Real
Time Data
Provisioning
6.4.1 Coordinate
Real Time Data
Provisioning
Activity
6.4.2 Track Real
Time Data
Provisioning
Activity
6.4.3 Manage
Real Time Data
Provisioning
Activity
6.4.4 Update
Real Time Data
Repository
6.5 Report Real
Time Data
Provisioning
6.5.1 Monitor
Real Time Data
Order Status
6.5.2 Distribute
Real Time Data
Order
Notification
6.5.3 Distribute
Real Time Data
Provisioning
Reports
6.6 Close Real
Time Data Order
6.7 Issue Real
Time Data
Orders
6.7.1 Assess
Real Time Data
Request
6.7.2 Create
Real Time Data
Orders
6.7.3 Mark Real
Time Data Order
For Special
Handling
6.8 Recover Real
Time Data
6.8.1 Develop
Real Time Data
Recovery Plan
6.8.2 Provide
Real Time Data
Recovery
Proposal
Notification
6.8.3 Request
Real Time Data
Recovery
Authorisation
6.8.4 Commence
Real Time Data
Recovery
6.8.5 Complete
Real Time Data
Recovery
6.8.6 Recover
Specific Real
Time Data

46. Step 7 - Real Time Data Collection And Distribution –
Processes And Functions Details
March 8, 2016 46
7 Real Time Data
Collection And
Distribution
7.1 Collect
Management
Information And Data
7.1.1 Intercept
Events/Information
7.1.2 Deliver
Management
Information
7.2 Process
Management
Information And Data
7.2.1 Determine
Recipients For
Information/Data
7.2.2 Filter
Information/Data
7.2.3 Aggregate
Information/Data
7.2.4 Format
Information/Data
7.3 Distribute
Management
Information And Data
7.3.1 Distribute
Information/Data
7.3.2 Manage
Distribution
7.3.3 Confirm
Distribution And Clean-
Up
7.4 Audit Management
And Security Data
Collection And
Distribution

47. Step 8 - Real Time Data Trouble Management –
Processes And Functions Details
March 8, 2016 47
8 Real Time Data
Trouble
Management
8.1 Survey And
Analyse Real
Time Data
Trouble
8.2 Localise Real
Time Data
Trouble
8.3 Correct And
Resolve Real Time
Data Trouble
8.4 Track And
Manage Real
Time Data
Trouble
8.5 Report Real
Time Data
Trouble
8.6 Close Real
Time Data
Trouble Report
Flow
8.7 Create Real
Time Data
Trouble Report

48. Step 9 - Real Time Data Performance Management –
Processes And Functions Details
March 8, 2016 48
9 Real Time Data
Performance
Management
9.1 Monitor Real Time
Data Performance
9.1.1 Manage Real Time
Data Performance Data
9.1.2 Record Real Time
Data Performance Data
9.1.3 Correlate Real Time
Data Performance Event
Notifications
9.2 Analyse Real Time
Data Performance
9.2.1 Perform Specific
Real Time Data
Performance Diagnostics
9.2.2 Manage Real Time
Data Performance Data
Collection Schedules
9.3 Control Real Time
Data Performance
9.3.1 Instantiate Real
Time Data Performance
Controls
9.3.2 Instantiate Real
Time Data Trouble
Controls
9.4 Report Real Time
Data Performance
9.4.1 Monitor Real Time
Data Performance
Degradation Report
9.4.2 Distribute Real
Time Data Quality
Management Reports
And Summaries
9.5 Create Real Time
Data Performance
Degradation Report
9.5.1 Generate Real Time
Data Performance
Degradation Problem
9.5.2 Convert Report To
Real Time Data
Performance
Degradation Report
Format
9.6 Track And Manage
Real Time Data
Performance Resolution
9.6.1 Coordinate Real
Time Data Performance
9.6.2 Request Other
Parties Performance
Degradation Report
Creation And Update
9.6.3 Update First In Real
Time Data Testing
Results
9.6.4 Cancel Real Time
Data Performance
Degradation Report
9.6.5 Escalate/End Real
Time Data Performance
Degradation Report
9.6.6 Clear Real Time
Data Performance
Degradation Report
Status
9.6.7 Engage External
Party Real Time Data
9.7 Close Real Time Data
Performance
Degradation Report

49. Step 10 - Real Time Data Aggregation And Reporting
– Processes And Functions Details
March 8, 2016 49
10 Real Time Data
Aggregation And
Reporting
10.1 Aggregation
Real Time Data
Records
10.1.1 Validate
Real Time Data
Records
10.1.2 Normalise
Real Time Data
Records
10.1.3 Convert
Real Time Data
Records
10.1.4 Correlate
Real Time Data
Records
10.1.5 Remove
Duplicate Real
Time Data Records
10.2 Report Real
Time Data Records

50. Using Real Time Data Strategy and Implementation
Approach
March 8, 2016 50
Activity Timeline
Real Time Data Strategy and Implementation Plan
1 Real Time Data Strategy And Planning
1.1 Gather And Analyse Real Time Data Information
1.1.1 Gather Real Time Data Information
1.1.2 Analyse New Real Time Data
Requirements
1.1.3 Analyse To Develop New/Enhance Real
Time Data Requirements
1.2 Manage Real Time Data Research
1.2.1 Manage Real Time Data Research
Investigations
1.2.2 Manage Administration Of Real Time
Data Research
1.2.3 Define Real Time Data Research
Assessment Methodologies
1.3 Establish Real Time Data Strategy And
Architecture
1.3.1 Establish Real Time Data Strategy
1.3.2 Develop Real Time Data Strategy
1.3.3 Establish Real Time Data Delivery Goals
1.3.4 Establish Real Time Data
Implementation Policies
1.4 Define Real Time Data Support Strategies
1.4.1 Define Real Time Data Support
Principles
1.4.2 Define Real Time Data Support Policies
1.4.3 Define Real Time Data Support
Performance Standards
…

51. Using Real Time Data Strategy and Implementation
Approach
• Use the proposed work breakdown to produce a detailed
plan
March 8, 2016 51

52. Real Time Architecture High-Level Components
March 8, 2016 52
Data Sources
Data Collection
And Data Source
Management
Communications
And Security
Data
Integration
Data Quality/
Summary/
Filter/
Transformation
Data Storage
Data Storage
Infrastructure
Data Reporting
and Analysis
System
Management,
Administration
and Control
External
Systems (Asset
Management,
Workforce
Management)

53. Real Time Architecture High-Level Components
March 8, 2016 53
Component Description
Data Sources These are the data collection/generation sources/sensors installed in the data collection landscape and new
sensor devices and signal data sources. Over time the approach to data collection may be standardised and old
equipment replaced.
Data Collection And
Data Source
Management
This logical component acts as a local front-end to existing signal data collection/generation sources/sensors. It
eliminates the need to replace existing devices. It offers a standard interface. It manages the sensor
infrastructure and landscape
Communications And
Security
This is the communications infrastructure used to securely transmit remotely collected data from local data
sources/sensors and data collection units to the central facility.
Data Integration This component manages the receipt of multiple data types in multiple formats from multiple data sources.
Data Quality/
Summary/ Filter/
Transformation
This component applies data quality algorithms to intrinsically noisy real time data to make it more usable.
Data may be filtered, summarised and transformed prior to storage
Data Storage This is the software component for storing in a structured manner and providing access to real time data.
Data Storage
Infrastructure
This is the underlying data storage infrastructure. The volumes of real time data are potentially very substantial
– hundreds of millions of data points per day. The data storage requirements could amount to thousands of
Terabytes over several years. This components includes facilities for backup, recovery, archiving and deletion.
System Management,
Administration and
Control
This component provides facilities to manage, administer and control the overall real time system.
Data Reporting and
Analysis
This provides reporting and analysis facilities to meet a wide variety of business requirements.
External Systems Real time data can be merged with other data such as asset to provide a usage dimension to static asset data
and to integrate with workforce management to manage sensor infrastructure.

54. Possible Approaches To Real Time Data Architecture
• You can expend a great deal of time, resources and money
on defining the requirements of an idealised real time data
architecture before embarking on any procurement and
implementation
OR
• You can research the viable products and their
functionality and what other water utilities have
implemented and define requirements in terms of what is
realistically achievable, reducing costs and time and
delivering results and learning more quickly
March 8, 2016 54

55. Getting Real Time Data Right
• Avoid false starts. Balance implementation urgency and pace with
organisational readiness and maturity. Success requires change
management
• Cannot do it all at once
• Real time data is an enabler and not an end in itself
• Embed use of real time data data in the organisation
• Don't turn it into a "finding the right tool" decision
• Recognise the interaction of governance, processes and tools that enables
the organisation optimise its real time data investment
• Focus on value, risks and prioritisation with active engagement of key
stakeholders
• Avoid using it solely for one-time annual budget decisions. It is about
continuous alignment, tracking and benefits realisation of existing real
time data systems and new projects
• Clarify governance and put a process in place that uses portfolio
management approach to consider, make and enforce decisions
March 8, 2016 55

56. Summary
• Real time data includes Telemetry, Big Data, Smart Metering
and Internet of Things
• Represents an idealised organisation and process breakdown
across entire spectrum of real time data from strategy to
workforce management to device installation, data collection
and data usage and actioning
• Provides a basis for developing a work breakdown and an
implementation plan
• Real time data infrastructure and systems without organisation
and processes will yield few benefits
• Represents a comprehensive structure that an organisation can
adapt to meet its long-term real time data needs
• Enables application and use of real time data to be embedded
in the organisation
March 8, 2016 56