Environmental Data Management and Analytics Please visit for more details www.emcentre.com

1. Environmental Data Analytics
Dr Prasad Modak, Tausif Farooqui
1

2. Structure of Presentation
2

3. EMC’s Experiences
Environmental Data Analytics
3

4. EMC
Experiences on
Environmental
Data analytics
• Dr Prasad Modak had setup the first data base
management system in India for environmental data for
CPCB in 1986 Which included
• Air Data management
• Water data management,
• Consent managements
• Cess calculations
• Water Quality data analysis for Godavari
• Monthly data for 7 years
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
4

5. EMC
Experiences on
Environmental
Data analytics
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
5

6. EMC
Experiences on
Environmental
Data analytics
• Industries are responsible to collect data for mandatory
submission to regulator, by law.
• Industries also use these data sets for disclosures
• And in sustainability reports
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
6

7. EMC
Experiences on
Environmental
Data analytics
• Egyptian Pollution Abatement Programme-phase 2
• Dr Prasad Modak had setup an online environmental
monitoring system which was to serve regulator and
financer
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
7

8. EMC
Experiences on
Environmental
Data analytics
• Analysis of real-time water quality monitoring data of River
Ganga – 2015
• Data generated by sensors installed at 10 stations.
• Geo-Database for Punjab water supply department – 2016
• Analysis and Dashboard for National Ground Water
Management Improvement program- India 2016
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
8

9. EMC
Experiences on
Environmental
Data analytics
• Analysis of air quality in a air shed with 2 power plants -
2016
• Data from Ambient monitoring, stack measurements,
automatic real-time monitoring, and metrological real time
monitoring were analyzed
• Air quality model was built for spatial prediction of
pollutant concentration
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
9

10. EMC
Experiences on
Environmental
Data analytics
• EMC had setup an EMIS for Rourkela steel plant in 2004
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
10

11. EMC
Experiences on
Environmental
Data analytics
• Corporate Sustainability Report for Glenmark
Pharmaceuticals -2015
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
11

12. EMC
Experiences on
Environmental
Data analytics
• Web based GIS for management for the Ahmedabad-
Mehsana Toll Road.
• Data of Maintenance and toll
• Cause-effect analysis for accidents
• Showcased by ESRI an one of the first application of ARC on
the web
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
12

13. EMC
Experiences on
Environmental
Data analytics
• Designing the data structure
• Designing the MIS, EMIS
• Online/offline applications
• Environmental Data analysis
• Environmental modeling
• GIS applications
• Community applications – crowd sourcing
Indian
Regulators
International
Financing
institution
Manufacturing
Industries
Infrastructure
and Transport
Industry
Our
Strengths
13

14. Structure of Presentation
14

15. Characteristics of
Environmental Data
Environmental Data Analytics
15

16. Characteristics
of
Environmental
Data
Multivariate
Environmental
Data
Stack Emission
Ambient Air
Quality
Meteorological
Measurements
Health Data
•Patients, Diseases,
Mortality…
Vegetation Data
•Bio Diversity,
Green Cover…
Agricultural
Data
•Productivity,
Yield..
BIG Data Irregular Fuzzy
Primary Data and Meta Data
16

17. Characteristics of
Environmental
Data
BIG DATA
• Summing up all the characteristics of environmental data
• Essentially a BIG DATA management and processing system
is needed
• To generate meaningful and actionable reports from the
data
Multivariate BIG Data Irregular Fuzzy
17

18. Characteristics of
Environmental
Data
• Various data sets have different frequency of
measurements
• Ambient Air Quality parameters are monitored twice a
week while stack and metrological data is monitored in real
time (15 min period).
• Other data sets like heath, agriculture are recorded and
published monthly or seasonally.
• It creates a complicated situation to merge and consolidate
all data sets.
Multivariate BIG Data Irregular Fuzzy
18

19. Characteristics of
Environmental
Data
Data from different types of instrumentation
Multivariate BIG Data Irregular Fuzzy
19

20. Characteristics of
Environmental
Data
• Large volumes of data
• Uncertainties are very high
• Frequent and large Data gaps
• Accuracy of measurement is questionable at some
instances
• It makes the data not ready for consumption of decision
makers
Multivariate BIG Data Irregular Fuzzy
20

21. To derive value from these data sets, advanced
application of DATA analytics is required
21

22. Role Based
reports
Characteristics of
Environmental
Data
Raw Data –Site Engineer,
Developer
Sub indices – Line
Manager
Indicators –
Department Managers
Dashboard – CEO, CFO,
CxOs
22

23. Structure of Presentation
23

24. Dashboard
Environmental Data Analytics
24

25. Dashboard
• Visual representation of the Data
• All the information at one place
• Key Indicators
• A single screen for insights and analytics
• Objective oriented or goal based
• Advanced statistical/mathematical models
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
25

26. Dashboard
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
26

27. Dashboard
• Do you record data or measurements ? Yes
• Where do you use the recorded data ?
• For decision making
• For compliance
• Information dissemination – mass communication
• How do you use the recorded data ?
• Print ?
• Excel, pivot tables ?
• Or Visualize ?
• For BIG data which method would you prefer?
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
27

28. Dashboard
BIG DATA Analytics
• To understand the data
• To manage the data
• To process the captured data in a hierarchical manner
• To visually communicate data
• To generate Role based reports instantly
• To make data driven business or operations decisions
• To create alerts
28
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau

29. Dashboard
http://www.forbes.com/
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
29

30. Dashboard
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
30

31. Dashboard
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
31

32. Dashboard
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
Dedicated Management
Information System - MIS
Environmental Management
Information System - EMIS
Analytics and Business
Intelligence tools
32

33. Dashboard
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
Analytics and
Business
Intelligence tools
SAS Qlikview Tableau
And many
more
33

34. Dashboard
• Tableau is a business intelligence (BI) tool
• Tableau connects easily to nearly any data source
• Tableau allows transforming data into visually
appealing, interactive visualizations
• Compatibility across Multiple Platforms - desktop
tool, web browser, iPad or mobile phone.
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
34

35. Dashboard
• Tableau's Data Analytic and Visualization Features
Interactive visualizations
• Simultaneous Connections with multiple data sources
• Allows to create custom calculations
• Charts, Graphs, Heat maps etc.
• Time series functions
• Map visualizations (point, line, polygon)
• Access control
• Multiple simultaneous dashboards
What is a
Dashboard?
Why
Dashboard?
Sample
Dashboards
Tools Tableau
35

36. Structure of Presentation
36

37. Dashboard for Air Quality
Management
Environmental Data Analytics
37

38. What does Real
Time
Environmental
Data Analytics
Involve?
Dashboard
for Air Quality
Management
• Allows Quick and Dynamic Visualizations
• Mines the BIG data to Optimize Sampling Frequencies and
get More Value
• Uses Real Time Algorithms with Advanced Statistical Tools
• Detects Trends, Patterns and allows Forecasting
• Helps in establishing association between Emissions and
Ambient Quality or understand Source Influences
Features Input
Parameters
illustrations New
Concepts
The Future
38

39. • Emissions
• Ambient Air Quality
• Meteorology
• Ambient, Stack and Automatic monitoring and
metrological data in a single dashboard for
effective interpretation, reporting and taking
action
What are the
important
factors for air
quality
management?
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
39

40. Analytical
features for
Ambient Air
Dashboard for
Air Quality
Management
• Missing Values treatment
• Outliers
• Basic Statistics (Mean, Deviations, Percentiles, Type of
Distribution)
• Box Whisker Plots
• Histograms and Frequency Diagrams
• Compliance with standards with timestamps
• Detecting simultaneous Violations
• Location Importance Index
• Correlations – Inter parameter and Inter station
• Trend Analysis (magnitude, direction and statistical significance)
• Air Quality Index
Features Input
Parameters
illustrations New
Concepts
The Future
40

41. • Time – Date of observation/ measurement
• Observed/Measured values of
• Concentration of Particulate Matter (PM10, PM2.5)
• Concentration of Gasses ( CO2, NOx )
• Location of the station
• Meta data like up time, calibration date, battery level,
sensor make, sensor accuracy etc.,.
Parameters for
Ambient Air
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
41

42. Analytical
features for
Stack Emission
Dashboard for
Air Quality
Management
• Missing Values
• Outliers
• Basic Statistics (Mean, Deviations, Percentiles,
Type of Distribution)
• Box Whisker Plots
• Histograms and Frequency Diagrams
• Compliance with standards with timestamps
• Detecting simultaneous Violations
• Stack Importance Index
• Predictive Emission Modelling
Features Input
Parameters
illustrations New
Concepts
The Future
42

43. Parameters for
Stack Emission
Dashboard
for Air Quality
Management
• Time – Data of observation/ measurement
• Observed/Measured values of
• Concentration of SPM, other pollutants
• Temperature
• Generation
• Location of the stack
• Meta data like up time, calibration date, battery level,
sensor make, sensor accuracy etc.,.
Features Input
Parameters
illustrations New
Concepts
The Future
43

44. Analytical
features for
Metrological
data
Dashboard for
Air Quality
Management
• Wind rose,
• Pollution rose
• Persistent wind rose
• Non-Parametric Wind Regression (NWR) for
Source Identification
• Stack-station Influence analysis
Features Input
Parameters
illustrations New
Concepts
The Future
44

45. Parameters for
Meteorological
data
Dashboard for
Air Quality
Management
• Time – Data of observation/ measurement
• Observed/Measured values of
• Temperature
• Wind Speed
• Wind Direction
• Relative Humidity
• Solar Radiation
Features Input
Parameters
illustrations New
Concepts
The Future
45

46. Missing Value
Dashboard for
Air Quality
Management
• an ‘observation not recorded’
• Treatment of Missing Value
• Replaced by Mean of Window or the selected time
span
• Replaced by Mean of Neighbors
Features Input
Parameters
illustrations New
Concepts
The Future
46

47. Demo Dashboard
Visualize parameters
1. NOx
2. SOx
3. PM10
4. PM2.5
Can be extended to
include other
parameters
Demo is prepared for
AIR, a similar dashboard
can be developed for any
other Environmental
Data
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
47

48. Options in dropdown boxes
Select Visualization Type
– Maps, Statistics, Charts
Map
Chart
Stats
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
48

49. Dashboard for
Air Quality
Management
User can only
download the PDF
and Image of the
screen.
User cannot
however
download the raw
data behind it.
Features Input
Parameters
illustrations New
Concepts
The Future
49

50. Station 1 Station 2Parameter 1 Parameter 2
AveragingPeriod
User options
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
50

51. Selected stations are located on
the map,
Map is sourced from ISRO-
Bhuvan
Click to open satellite view of the
location in Google maps
Map
Visualization
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
51

52. Green band – permitted
values in designated water
quality class
Black lines – higher and lower
Statistically acceptable
permitted values, beyond this
value is an outlier
Blue line – Chart of selected
parameter
Red dots – Both of the
selected parameters are
violating applicable standards
simultaneously
Stations
Parameter 1
Parameter 2
-
Charts
Visualization
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
52

53. Basic Statistics
Number of Records
Max, Min, Mean, Median
SD, CoV, Number of Outliers
Percentage violations, and
Longest Contiguous violation
for both the selected
parameters
StationsParameter 1
Parameter 2
Percentage simultaneous
violations by both parameters
against relevant standard.
Longest Contiguous and
Simultaneous violation
Crosstab
Visualization
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
53

54. • An outlier is an observation
point that is distant from
other observations.
• User option to select
method of treating outliers
• User option: Show/Hide
outlier
• Outlier is removed and
treated as missing value
Outlier
Dashboard
for Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
54

55. Basic Statistics
Box Whisker
Plots
Dashboard for
Air Quality
Management
Annual Maximum
Annual Minimum
Annual Mean
CPCB standard
Standard Deviation
Features Input
Parameters
illustrations New
Concepts
The Future
55

56. Compliance
with standards
with
timestamps
Dashboard for
Air Quality
Management
Dates of violations
Features Input
Parameters
illustrations New
Concepts
The Future
56

57. • Radial Chart
• Distribution of Wind speeds Vs Direction
Wind Rose
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
57

58. Inter-station
correlations
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
58

59. Trend Analysis
(magnitude,
direction and
statistical
significance)
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
59

60. • Radial Chart
• Distribution of Pollutant Concentration Vs Direction
Pollution Rose
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
60

61. Compliance
Histogram
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
61

62. Detecting
simultaneous
Violations
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
62

63. • Percent exceedance (how
many times is the standard
for a parameter violated in
relation to the length of the
data chain)
• Extent of contiguous
exceedance (how long have
been the violation over
standard on a continuous
basis in relation to the
length of the data chain)
• Magnitude of exceedance
(how high are the violations
when there is an
exceedance over standard,
expressed in sum of squares
of deviations)
Location
Importance
Index (LII)
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
63

64. Location
Importance
Index (LII)
Dashboard for
Air Quality
Management
Locations could be a receptor, ambient station or a stack
Features Input
Parameters
illustrations New
Concepts
The Future
64

65. • Distribution of Consistency in wind direction Vs Direction in the number of
hours in a Radial Chart
• Data Quality check for automatic metereology stations
0
2
4
6
8
N
NNE
NE
ENE
E
ESE
SE
SSE
S
SSW
SW
WSW
W
WNW
NW
NNW
2014
Persistent
wind rose
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
65
0
50
100
150
200
250
N
NNE
NE
ENE
E
ESE
SE
SSE
S
SSW
SW
WSW
W
WNW
NW
NNW
2012

66. Stack-station
Influence
analysis
Dashboard for
Air Quality
Management
• Influence of emissions
from a particular stack
on a particular ambient
station
• Instances of influence
are plotted as time-
series
Features Input
Parameters
illustrations New
Concepts
The Future
66

67. Stack-station
Influence
analysis
Dashboard for
Air Quality
Management
Air Quality measurements are
taken at Stack and Ambient
Estimates influence of a stack
emission on a specific ambient
monitoring station
Longest contiguous duration of a
receptor under influence of stack
Features Input
Parameters
illustrations New
Concepts
The Future
67

68. Non-
Parametric
Wind
Regression
(NWR) for
Source
Identification
Dashboard for
Air Quality
Management
• NWR has been applied for better understanding of emission
sources of influence since early 2000
• Technique of NWR is used to identify presence and locations of
major emission influence on the observed pollutant
concentrations
• NWR requires high frequency (e.g. one hour) concentration and
meteorological data at the location where emission influences
are to be assessed
Features Input
Parameters
illustrations New
Concepts
The Future
68

69. Non-
Parametric
Wind
Regression
(NWR) for
Source
Identification
Dashboard for
Air Quality
Management
Features Input
Parameters
illustrations New
Concepts
The Future
69

70. Source
Diagnostics or
Environmental
Forensics
Dashboard for
Air Quality
Management
• Data science + Environmental science
• Application of Sophisticated modelling tools, AERMOD for
air quality
• Receptor Modelling
• Source identification using Met and Pollution
Concentration Data
• Analyzing of special events – Technique of Intervention
analyses
• Predictive emissions modelling system
Features Input
Parameters
illustrations New
Concepts
The Future
70

71. AERMOD
Air Dispersion
Modelling
71
• US EPA based Model
• Gaussian plume air dispersion model
• Predict downwind pollutant concentrations based on source emissions,
meteorological field, and site parameters (land use, terrain features
etc.)
• Model can predict accurately for 50km
Features Input
Parameters
illustrations New
Concepts
The Future

72. Source
apportionment
Air Dispersion
Modelling
72
Features Input
Parameters
illustrations New
Concepts
The Future
Click on the
box to see
Percentage
contribution
of two
power plants
of ambient
PM10 at a
sensitive
receptor

73. Modelled percentage contribution of two power plants
of ambient PM10 at a sensitive receptor
Period % contribution of A % contribution of B
Winter 36.1 4.7
Summer 14.3 13.9
Post monsoon 45.3 7.9
Annual 32.2 8.5
73

74. Integration of
Models and
Dashboard
Integrated
Approach
74
• Air Emission and Air Quality Dashboard integrated with
the Dispersion models
• Model could be “fired” from the dashboard itself and
required data on emissions, air quality and
meteorology will be picked up automatically from data
available.
• Model will keep re-calibrating itself with actual real
time data and this will increase the accuracy of the
model outputs.
• Models will also help to “validate” the data collected,
thereby improving the quality of monitored data.
Features Input
Parameters
illustrations New
Concepts
The Future

75. PEMS
Predictive
Emissions
Monitoring
System
75
Features Input
Parameters
illustrations New
Concepts
The Future

76. PEMS
Predictive
Emissions
Monitoring
System
76
• Developed on Artificial Neural
Network (ANN) concepts
• The model automatically predict
emissions data in real time while
exploiting the inherent correlation
between process variables (flow,
temperature and pressure) and
emission properties (NOx, SO2, CO,
CO2)
• PEMS can replace CEMS and hence
save on costs
Features Input
Parameters
illustrations New
Concepts
The Future
Click on the box to
see Comparison
between CEMS &
PEMS

77. CEMS PEMS
Cost Large capital & operational expenditure Less expensive (50% to 75%) than CEMS.
Maintenance Periodic maintenance and calibration of
the sensor and analyser is required
No additional instrument maintenance (other than
normal process sensors) is required
Accuracy CEMS are the standard to which PEMS
are measured
PEMS are initially calibrated using CEMS. The CEMS are
removed when the PEMS becomes operational. PEMS are
less accurate than CEMS when the latter is well
maintained and calibrated
Uses CEMS alarms when emissions are
approaching or exceeding an
environmental standard
PEMS can alarm, but also predict future outcomes and
optimize processes in real-time. PEMS therefore leads to
higher production efficiency and hence lower cost of
production.
77
Comparison of CEMS and PEMS

78. Thank you
Please visit
http://dev-data-portal.pantheonsite.io/
Dr Prasad Modak – Prasad.modak@emcentre.com
Tausif Farooqui – Tausif.farooqui@emcentre.com
78

79. Structure of Presentation
79