Integrate SUNSHINE pilot smart meter data

WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161
Page 1 of 45
"This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the
Competitiveness and Innovation Framework Programme by the European Community"
(http://ec.europa.eu/ict_psp).
Annex to D4.5 - Meter Data Management Service #2
Specifications for Data Ingestion
via Green Button
WP 4 – Integration of SUNSHINE pilot smart urban services
Task 4.9 – Integration of new smart services
with existing service infrastructures
Task 4.2 – Meter data management service
Revision: v1.1

CIP-ICT-PSP-2012-6 – 325161
Page 2 of 45
REVISION HISTORY AND STATEMENT OF ORIGINALITY
Revision Date Author Description
v. 0.1 3rd
September 2014 Luca Giovannini Document created.
v. 0.3 4th
September 2014 Rossana Bambili First draft of guidelines
v. 0.9 18th
September 2014 Luca Giovannini Final draft
v. 1.0 14th
November 2014 Stefano Pezzi Release
v. 1.1 7th
January 2015 Stefano Pezzi Revision after pilot installation
v.1.2 16th
January 2015 Stefano Pezzi
Added some information on the
pre-population of the datacustodian
tables
Statement of originality:
This deliverable contains original unpublished work except where clearly indicated otherwise.
Acknowledgement of previously published material and of the work of others has been made through
appropriate citation, quotation or both.
Moreover, this deliverable reflects only the author’s views. The European Community is not liable for any
use that might be made of the information contained herein.

CIP-ICT-PSP-2012-6 – 325161
Page 3 of 45
Table of contents
1 Purpose...............................................................................................................................5
1.1 Green Button in a nutshell.......................................................................................................5
1.1.1 Green Button & Sunshine.....................................................................................................6
1.2 OAuth.....................................................................................................................................9
1.2.1 OAuth v.1.0 vs v.2.0 .......................................................................................................... 14
1.3 OAuth & Green Button ......................................................................................................... 16
1.4 Green Button in Sunshine scenario ........................................................................................ 17
2 Software Installation .........................................................................................................20
2.1 Software Requirements (DataCustodian) ............................................................................... 20
2.2 Installation instructions......................................................................................................... 20
2.2.1 Java JRE............................................................................................................................. 21
2.2.2 Apache Tomcat.................................................................................................................. 21
2.2.3 MySQL............................................................................................................................... 22
2.2.4 DataCustodian application................................................................................................. 27
3 Configuration and Use.......................................................................................................29
3.1 datacustodian DB structure ................................................................................................... 29
3.2 Populating the datacustodian database ................................................................................. 31
3.2.1 ‘retail_customers’ table ..................................................................................................... 31
3.2.2 ‘application_information’ table.......................................................................................... 32
3.2.3 ‘application_information_scopes’ table.............................................................................. 32
3.2.4 ‘time_configurations’ table................................................................................................ 32
3.2.5 ‘usage_points’ table........................................................................................................... 33
3.2.6 ‘reading_types’ table ......................................................................................................... 34
3.2.7 ‘meter_readings’ table....................................................................................................... 34
3.3 Import data service ............................................................................................................... 35
3.3.1 Service url ......................................................................................................................... 36

CIP-ICT-PSP-2012-6 – 325161
Page 4 of 45
3.3.2 Consumption data XML...................................................................................................... 36
3.4 Export data ........................................................................................................................... 38
3.5 Security underneath the application layer.............................................................................. 39
3.6 Generation of the access token.............................................................................................. 39
Annex 1 – DST rules..................................................................................................................42
Annex 2 – Codelists ..................................................................................................................44
ServiceCategoryKind................................................................................................................................ 44
AccumulationBehaviourType .................................................................................................................. 44
CommodityType ...................................................................................................................................... 44
TimeAttributeType .................................................................................................................................. 45
UomType ................................................................................................................................................. 45
CurrencyCode .......................................................................................................................................... 45

CIP-ICT-PSP-2012-6 – 325161
Page 5 of 45
1 Purpose
The purpose of this guideline is to support the pilots in implementing at their own premises a Green Button
web service, the DB and all the backend supporting services behind it. Through this web service, they will
be able to expose their consumption data to the central Sunshine platform.
1.1 Green Button in a nutshell
Green Button is an initiative of the U.S. Government that aims to allow consumers to access their own
meter data (called Energy Usage Information, EUI) about energy consumption. In the original context, the
name Green Button (in the rest of the document we’ll use GB) denotes the whole initiative, but in our
project we will call GB only the suite of web services that makes the data access possible.
There are three actors involved in the scenarios of making meter data available:
Retail Customer (RC) = is the person (a real one or an entity) that consumes energy and is the actual
owner of the data.
Data Custodian (DC) = is typically the utility that delivers the energy and measures it with meters in
order to get it paid by the retail customer. To do this, the data custodian has to
record consumptions data and store them for its operational purposes.
Third Party (TP) = is a service provider that offers value-added services to the retail customer,
services that elaborate his/her meters data
These three actors interact among them usually in these two alternative ways:
1. Retail Customer directly accesses his/her own data by using data custodian services.
2. Retail Customer delegates a Third Party to access his/her own data. The Third Party uses the Retail
Customer’s data to provide some sort of special services to the Retail Customer, probably using
some other information available from further data providers (meteorological data, global energy
consumption data …). Some examples of services could be: advanced reporting, hints about energy
saving, etc.
In GB jargon, the first is called “GB Download My Data” and the second “GB Connect My Data”.

CIP-ICT-PSP-2012-6 – 325161
Page 6 of 45
1.1.1 Green Button & Sunshine
The Sunshine scenario can be described by the latter of the two GB interactions (GB Connect My Data) with
some peculiarity. In Sunshine the actors are the following:
Pilot = is the owner or conductor of a building that consumes energy and is the actual
owner of the consumption data.
Pilot technical partner = is a subject that acts on behalf of the utility company in the sense that it is in
charge of gathering the consumption data in various manner (in real time or off
line). It is an intermediary of the utility company because this one could not be
active part of the project: by means of smart meters, retrofitted meters, or
simply reading the old bills, consumption data are retrieved and made available.
The Pilot Technical Partner is strictly related to the Pilot and sometimes it can
overlap completely with it.
Sunshine platform = is the platform of services, data and software components that Sunshine has put
in place.
Let see know the correspondence with GB actors:
GB actor Sunshine actor
Retail Customer Pilot
Data Custodian Pilot Technical Partner
Third Party Sunshine platform
Table 1 - Correspondence between GB and Sunshine actors
Like sketched in Figure 1, the Retail Customer is actually the owner of the building where the energy is
consumed and measured. In Sunshine’s consortium, the building owners are either the Pilot Technical
Partners (see HEPESCO) or the Pilot (i.e. Municipality of Ferrara buildings).

CIP-ICT-PSP-2012-6 – 325161
Page 7 of 45
Utility
company
Pilot
Technical
Partner
Sunshine
Pilot
EUI EUI (GB)
Energy
Data Custodian & Retail Customer
Third Party
Figure 1 - Energy Usage Information flow in Sunshine
The real Data Custodian should be the utility company that distributes the energy, but, due various
technical and “political” problems, it’s quite impossible to get the data directly from the utility company. So
we can consider the pilot’s technical partner as a surrogate of a Data Custodian, since it manages to get the
meter data either by asking them to the utility in a non-automatic manner or by adding its own meter
reading infrastructure to the utility’s one.
The Third Party is the Sunshine platform that needs to access the consumption data to process them and
provide value added services.
Figure 2 - The GB scenario

CIP-ICT-PSP-2012-6 – 325161
Page 8 of 45
The first two actors are almost the same and this is the peculiarity of the Sunshine scenario: Retail
Customer could easily access directly its data, because it is very close to the Data Custodian or sometimes
it’s the Data Custodian itself, but requires the services that Sunshine platform can offer.
The original context of the GB scenario is the one depicted in Figure 2. Here we can see the flows between
the 3 actors. Notice the two button icons that indicate the respective flows of data: in the diagram red
arrows show the “GB Download My Data” flow, while the blue ones are involved in the “GB Connect My
Data” use case.
The Sunshine’s context is a profile of the “GB Connect My Data” context of GB scenario, and we can
simplify the diagram in that of Figure 3, taking away the “GB Download My Data”.
Figure 3 – The Sunshine scenario
The dotted line arrows represent the authorization flows, in particular those of the delegated access
implemented by OAuth. The term “One-Time Authorization” is used to highlight the fact that once has
happened it may last for a long time in order to avoid a new authorization delegation exchange every time
a data access has to be performed.
The solid blue line arrow is the real data exchange between Third Party and Data Custodian, the one that
interests most.
The green line arrow is an interaction between Third Party and Data Custodian that aims to register the
Third Party in the Data Custodian system as a subject that is being given a generic pre-authorization to use
the Data Custodian APIs or services. This registration operation allows creating a trust between DC and TP,

CIP-ICT-PSP-2012-6 – 325161
Page 9 of 45
and letting the DC know all the details of the actions that the TP could execute on the RU resources (i.e.
read and add documents from a web drive) once obtained the authorization from the owner. The
registration operation can take place offline although GB services allow to do it dynamically; in order to
simplify the exchange of data in Sunshine scenario, since the DC and the TP are well known from the start,
we will consider only the offline registration.
The OAuth protocol, on which GB bases its security features, relies also on HTTP redirection because the
main use case is the one in which the user access the Third Party services by means of his/her browser (in
the diagrams it’s called “agent”) and the first redirection is from TP’s web portal to DC’s web portal, where
the RU can authenticate and delegate access to the TP services.
In Sunshine the access delegation is like the TP registration: it can be done once and offline for the same
reasons; furthermore the DC web portal and the TP web portal, in the sense of the web portal functionality
involved in the authorization delegation operations, does not play any roles. So we can achieve a further
simplification of the diagram as we show in the Figure 4.
Figure 4 - Further simplification of Sunshine context
1.2 OAuth
As we said before, the three actors’ scenario of GB recalls the more generic one of delegated access: this is
an increasingly widespread situation in which a web user is involved almost every day. The user owns some
resources (emails, images, documents, address book, path recordings, bank …) that are maintained by
some provider (Google, Flickr, bank …). Other service providers offer more advanced services on these data

CIP-ICT-PSP-2012-6 – 325161
Page 10 of 45
(professional printing, special visualization, instant messaging, spatial analysis …), but, of course, they need
access to the original data do some work on them. This kind of interaction strongly involves security aspects
like granting access to other subjects to own protected resources without having to share own
authentication credential. OAuth1
protocol is about this problem and points out a particular authorization
flow among the involved subjects in order to fulfil the descripted objective.
In the OAuth terminology, the actors involved in the delegated authorization scenario are:
Resource Owner = the owner of the resources that we want to secure.
Client = an application making protected resource requests on behalf of the Resource
Owner and with its authorization. The term “client” does not imply any
particular implementation characteristics (e.g., whether the application executes
on a server, a desktop, or other devices).
Resource Server = The server hosting the protected resources, capable of accepting and
responding to protected resource requests using access tokens, that is the
artefact that substitutes the Resource Owner credential.
Authorization Server = the server issuing access tokens to the client after successfully authenticating
the Resource Owner and obtaining authorization.
User-agent = typically a web browser or other software under the control of the Resource
Owner.
We are talking of a three actors’ scenario, but here we’ve just listed five subjects: some of these actors
actually overlap in a higher level architecture and their roles are split only from an implementation and
technological point of view. In particular:
Resource Server ≡ Authorization Server
Resource Owner ≡ User-agent
Other terms used in the OAuth protocols are the following2
:
protected resources = an access-restricted resource that can be obtained from the server using an
OAuth-authenticated request.
1
OAuth v.1.0 is based on two proprietary protocols used by Google and Flickr: “AuthSub” and “API Auth”,
respectively.
2
http://tools.ietf.org/html/rfc5849

CIP-ICT-PSP-2012-6 – 325161
Page 11 of 45
credentials = Credentials are a pair of a unique identifier and a matching shared secret. OAuth
defines three classes of credentials: client, temporary, and token, used to identify
and authenticate the client making the request, the authorization request, and
the access grant, respectively.
token = A unique identifier issued by the server and used by the client to associate
authenticated requests with the resource owner whose authorization is requested
or has been obtained by the client. Tokens have a matching shared-secret that is
used by the client to establish its ownership of the token, and its authority to
represent the resource owner.
In the sample flow below, let’s imagine an user Bob that wants to use some professional printing services
offered by a web company PrintEveryThing that advertises its capacity of printing Facebook, Flickr, Google
Drive, Picasa … web stored photos. The roles are the followings:
 Bob = Resource Owner
 Flickr = Resource Server
 PrintEveryThing = Client
The flow is described by the following steps; step # 0 represents the preconditions of the flow:
0. Bob owns some photos at Flickr. Flickr implements OAuth mechanism and has an Authentication
Server and an Authorization Server as well. The PrintEveryThing is a registered Client at Flickr. This
means that PrintEveryThing has been given an ID by Flickr and shares a secret for encrypting its
messages to assure they come from it.
1. Bob accesses the PrintEveryThing web portal in order to use some of its services.
2. At a certain point of the interaction with the PrintEveryThing’s portal, Bob confirms that he wants
to print his photo album at Flickr. This action makes PrintEveryThing communicates with Flickr
asking for an unauthorized request token; the release of this token means that Flickr has
recognized PrintEveryThing as a registered Third Party, but without an explicit authorization from
the user, the token can’t be used to access the photos.
3. Bob’s browser gets redirected by PrintEveryThing’s portal to the Flickr portal. The redirection
contains a call-back URL and the unauthorized request token, plus some other information.
4. Flickr’s portal asks for Bob’s credentials and authenticates him by means of its Authentication
Server.

CIP-ICT-PSP-2012-6 – 325161
Page 12 of 45
5. Flickr’s portal recognizes the unauthorized request token and, retrieving the information stored in
the TP registration step (not shown in this flow), asks Bob what kind of permissions to give to
PrintEveryThing and for which protected resources (photos).
6. Bob chooses the privileges and resources and then confirms his intention to let PrintEveryThing
access them. This action makes Flickr produce a request token that is returned to PrintEveryThing
by means of the next step. We can say that the user granting operation transforms the
unauthorized request token into a request token.
7. Flickr redirects back BOB’s browser to the call-back URL at PrintEveryThing’s site bearing some
other information: the request token. This token contains the information that the user has granted
the Third Party the privileges to his resources, but can’t be used to directly access the resource yet.
8. Finally PrintEveryThing exchanges with Flickr its request token with an access token3
9. PrintEveryThing uses the access token to retrieve the protected photos from Flickr using its APIs
and then does its job on them.
In the diagram of Figure 5 the OAuth flow is shown with only the Client and the Resource Server
highlighted; user (the Resource Owner) or user’s browser is involved in the passages represented with a
solid line.
Actually, the OAuth flow that we exemplified here is one of the possible because several variants can be
sketched: for instance, the user could start interaction at the Resource Server portal instead of the Client’s
and by means of some redirections that go the other way around, the same steps can be performed and
the same result obtained. Instead of starting from the client’s portal, choosing the Resource Server
3
The access token is the final result of the OAuth flow: it is the string representing an access authorization issued to
the client, rather than using the resource owner's credentials directly.

CIP-ICT-PSP-2012-6 – 325161
Page 13 of 45
Client Resource Server
Requests
unauthorized request
token
Grants
unauthorized request
token
Redirects user to
resource server
Authenticates user
Obtains user
authorization
Redirects user
to Third Party
Requests
Access token
Unauth. req token
Request token
Grants
access token
Request token
Unauth. req token
Accesses
Protected resources
Access token
Protected
resources
Access token
User browses portal
Figure 5 - OAuth flow
The three fundamental interchanges can be further simplified in the schema of Figure 6.

CIP-ICT-PSP-2012-6 – 325161
Page 14 of 45
Client Resource Server
Authorization request
Authorization grant
Authorization grant
Access Token
Access Token
Protected Resource
Figure 6 – High level OAuth interchanges
1.2.1 OAuth v.1.0 vs. v.2.0
The OAuth authorization flow described up to here refers to version 1.0 that was first published at the end
of 2007 and that became a standard (RFC 5849) in mid-2010 (after an update release 1.0a fixed a security
leak). A subsequent version of OAuth, the v.2.0, has been released afterwards, the first draft in the very
same period of the 2010; this version is not backwards compatible, but it maintains the overall architecture
and the approach descripted above. Now the stable version4
is in a proposal state under the code RFC
6749.
This new release has been also a source of controversy: some of the original creators of OAuth abandoned
the project because some vulnerability has been introduced in exchange for more simplicity at client side,
but here we don’t want to discuss this aspect. The fact is that the major web companies have adopted
OAuth v.2.0 even if many features are still changing and being added.
Green Button relies on v.2.0, so it’s better that we take a quick look at the most important differences;
some of these are both simplifying the protocol and making it less secure, as critics says:
4
https://tools.ietf.org/html/rfc6749

CIP-ICT-PSP-2012-6 – 325161
Page 15 of 45
1. Cryptography. Client applications do not have to use HMAC (keyed-hash message authentication
code) to encrypt tokens and request string anymore, but they can send plaintext token over an
https secure channel.
2. Signature. A simpler signature is used using only one secret5
instead of two.
3. Bearer token. It is a special type of token that can be used instead of an access token. Every
request that has a bearer token with it, can access the protected resources for which it has been
issued, no matter who is the requestor. No need of the “proof-of-possession” that is proving that
who is sending the request is in possess of cryptographic key material. An access token instead is
used along with a signature that identifies the requestor.
4. Access token refresh. Access tokens can be long-lived and even have unlimited lifetime. This could
be a security issue in case an access token (or better, a bearer) is stolen. In OAuth v.2.0 server can
issue short-lived access tokens in order to minimize the time windows in which is possible to use
the access token fraudulently. The short-lived access token is released together with long-lived
refresh tokens that can be used to reissue the access token. The difference is that to use a refresh
token you need to use the secret and the client ID.
5. Separation of roles. OAuth v.2.0 separates the roles of Authorization server from the one of
Resource Server. Authorization server is responsible for obtaining user authorization and issuing
the relative tokens. Resource Server checks handles the API calls to the protected resources and
enforces the authorization policies. The original flow of Figure 6 can be re-drawn like in Figure 7. In
this diagram the client request to the resource owner is usually made indirectly via the
authorization server as an intermediary.
5
In cryptography a secret is a data known only by the two parts involved in a secure communication. This data, usually
a string or a big number, can be shared before the communication starts or at the start of it. The secret is used to
encrypt the messages exchanged between the parts.

CIP-ICT-PSP-2012-6 – 325161
Page 16 of 45
Client
Authorization request
Authorization grant
Authorization grant
Access Token
Access Token
Protected Resource
Resource Owner
Authorization server
Resource server
Figure 7 - OAuth v.2.0 interchanges
In the Sunshine scenario we don’t have a user interacting with either the Third Party’s portal or the
Resource Server’s one. So, a part from the overlap of the Resource Owner with the Resource Server, we
have this further characteristic of an absence of a human interaction. This means that all the outcomes of
the interaction must be produced in another way, in advance or off-line.
An example is given by the transformation of the “unauthorized request token” in a “request token” that is
the result of the user granting authorization to the third party.
1.3 OAuth & Green Button
The Green Button and the OAuth scenarios resembles very much. But these are not only analogies: as a
matter of fact GB services are strongly based on the OAuth protocol, in particular OAuth v.2.0.
In OAuth, the actors’ names are slightly different from the GB ones, so it’s better to remind the
correspondence:
GB actor OAuth actor
Retail Customer Resource Owner (RO) or user

CIP-ICT-PSP-2012-6 – 325161
Page 17 of 45
Data Custodian Resource Server6
(RS) or server or service provider
Third Party Client or consumer
Table 2 - Correspondence between GB and OAuth actors
So the use cases faced by GB can be classified in two main groups: the first that deals with the authorization
interchanges and that resembles the one of OAuth and the second that handles the real data transmission.
GB provide for dynamic authorization phase, in which actors “meet” them for the very first time and
registration, trust release and specific authorization for the resources are all managed using the OAuth
protocol and the OAuth interchanges. In Sunshine the parties involved are known from the start and some
passages can be done offline in order to simplify the operations.
In particular in the scenario “GB connect my data” the consumer is involved only in approving access to the
data, but is not (necessarily) involved in the movement of the data from the Data Custodian to a third party
service provider, and it results in a machine-to-machine communication.
In particular we’ll see in the next chapter how to install and configure an open source implementation of
Green Button services that offers a wide range of web APIs that can manage all the possible communication
between the parties. We won’t use all these APIs precisely because we want to keep the system simple.
1.4 Green Button in Sunshine scenario
In the Sunshine scenario, pilots that want to transmit data by means of the GB mechanism will have to:
1. Act like a retail customer and give Sunshine platform the authorization to access its data; this
action will be executed by a human user that interacts with the software capable of generating an
OAuth access token. The user will deal with both the Data Custodian and the Third Party software
that will interchange messages and redirect the user’s browser like in the interactions described
above. The access token will be finally stored in the Sunshine platform and used to access the
resources.
2. Expose the EUI by means of a GB service; therefore the data shall be ingested from the original
format into the database structures known by the service.
6
The Resource Server plays several roles that in OAuth can be highlighted as separated because they actually can be
implemented by different components of the architecture: Authentication server and Authorization server. OAuth
v.2.0 explicitly introduces this separation.

CIP-ICT-PSP-2012-6 – 325161
Page 18 of 45
To achieve these results, Sunshine is going to reuse the software developed as a reference implementation
by the initiative EnergyOS7
(Open Source for Energy Infrastructure) and the private company Pivotal Labs8
,
called OpenESPI.
In particular, the open source software packages implemented are the
1. DataCustodian module, that is the component that publishes the energy consumption data and the
2. ThirdParty module that communicates with it for the authorization steps.
A third component, developed in the Sunshine project, is the GreenButton Client, that is the module that,
once obtained the authorization token, will handle the mere EUI transfer operation.
Another function that the DataCustodian offers is the ingestion of the EUI data from original sources. A
service is exposed to facilitate the ingestion of the readings in the EUI database: the pilot can either
activate the “B” flow using this service (encoding the readings in a XML request) or the “A” flow loading the
data directly in the database with some custom scripts (see the blue arrows in Figure 8).
Like written above, in Green Button jargon, Pilot plays the role of Data Custodian and Sunshine platform
the Third Party one. Therefore the software module DataCustodian must be installed at the Pilot’s side that
whilst the ThirdParty module will be installed on the Sunshine platform.
DataCustodian
ThirdParty
EUI
internal
flow
EUI
Pilot
Sunshine
datacustodian
DB
(EUI)
Token
DB
Original
consumption
data
Oauth
flow
SOS
GreenButton
Client
A
B
B
Figure 8 – The interactions between Pilot and Sunshine platform
7
http://www.energyos.org/
8
http://pivotallabs.com/

CIP-ICT-PSP-2012-6 – 325161
Page 19 of 45
In the next chapter only the instruction to install the DataCustodian module will be given. Then, we’ll
describe the procedure to assign to the Sunshine platform a token to allow the EUI access and finally how
to feed the GB service with the consumption data.

CIP-ICT-PSP-2012-6 – 325161
Page 20 of 45
2 Software Installation
The DataCustodian is available as a web application packed in “war” file that is downloadable at the
address:
http://sunshine.sinergis.it/download/greenbutton/1.1/DataCustodian.war
**The package is based on the version 1.1-SNAPSHOT released on 4th
September 2014 and has been
adapted specifically for Sunshine needs. The current version available at the link above and on which are
shaped the current guidelines differs from the one that was available for download at the time of release of
version v0.9 of these guidelines, so pilots are invited to download and use the current WAR version.
2.1 Software Requirements (DataCustodian)
The installation of the following software is required:
 Java JRE (Java Run-Time Engine) 1.7.0 or higher (downloadable from http://www.java.sun.com)
 Servlet container Apache Tomcat v. 7.0.54 or higher (downloadable from
http://jakarta.apache.org/tomcat)
Moreover, the availability of a mySQL relational database is required9
to host the token store. The EUI
database can be implemented either by a mySQL DB or by a PostgreSQL.
2.2 Installation instructions
The deployment diagram of the DataCustodian application is shown in Figure 9.
The DB Server and the Application Server may be one and the same server.
The two databases maintain the consumption data that are published by the DataCustodian services, but
also support the OAuth mechanism implemented by GB. They could be two instances installed on separate
DB servers, but to keep it simple, consider them two SCHEMAs (same as two DATABASEs) in the same
server and in the same mySQL instance.
9
Next release should support also PostgreSQL and HSQL

CIP-ICT-PSP-2012-6 – 325161
Page 21 of 45
Figure 9 - Deployment diagram of DataCustodian application
2.2.1 Java JRE
The following instructions refer to a MS Windows environment (64bit), but are valid also for a Unix/Linux
environment, provided that the relevant path adjustments are applied:
1. Create a subfolder named greenbutton in the root folder of your system.
2. Download and execute jdk-7u65-windows-x64.exe.
3. Select as installation folder: C: greenbuttonjre1.7.0_45.
4. Set the environment variable JAVA_HOME to JAVA_HOME = C: greenbutton jre1.7.0_45.
Even if a JRE is already present, check that the installation path (step 3) has no blank spaces and that the
environment variable JAVA_HOME (step 4) is properly set.
2.2.2 Apache Tomcat
The following instructions refer to a MS Windows environment, but are valid also to a Unix/Linux
1. Create a subfolder named greenbutton in the root folder of your system.
2. Download apache-tomcat-7.0.54.zip.
3. Unpack the package apache-tomcat-7.0.54.zip in the greenbutton folder.
4. Set the environment variable CATALINA_HOME to CATALINA_HOME = C: greenbuttonapache-
tomcat-7.0.54.

CIP-ICT-PSP-2012-6 – 325161
Page 22 of 45
5. Open file <CATALINA_HOME>/bin/startup.bat and set memory allocation parameters for Java Virtual
Machine (JVM) as follows: set CATALINA_OPTS=-server –Xms2048m –Xmx2048m -XX:PermSize=256m
-XX:MaxPermSize=256m.
Even if Apache Tomcat is already installed in the server, check that the installation path has no blank spaces
and that the environment variable CATALINA_HOME (step 4) is properly set.
2.2.3 MySQL
The following instructions refer to a MS Windows environment, but are valid also to a Unix/Linux
1. Download MySQL Installer from http://dev.mysql.com/downloads/installer/ and execute it.
2. Choose the appropriate Setup Type for your system (developer or custom).
3. Follow the wizard’s instructions.
2.2.3.1 DataCustodian DB (EUI)
The datacustodian DB is meant to host the readings (EUI). It is automatically created by the application
DataCustodian when it is run for the very first time with some particular configuration settings. See the
paragraph 2.2.4.1 for the details; here the instruction for mySQL DB are illustrated, but the postgreSQL case
is different just for the connection parameters and the configuration file name. Here and in the rest of the
document the EUI database has been named “datacustodian”, but the pilot can choose whatever name and
modify consequently the configuration files.
2.2.3.2 Token DB
The Token DB is just for the authorization phase. It must be created manually running the SQL creation
scripts that you’ll find in
http://sunshine.sinergis.it/download/greenbutton/1.1/createSunshineTS_mySQL.sql and listed hereinafter.
CREATE DATABASE IF NOT EXISTS `tokenstore` /*!40100 DEFAULT CHARACTER SET utf8 */;
USE `tokenstore`;
-- MySQL dump 10.13 Distrib 5.5.35, for debian-linux-gnu (x86_64)
--
-- Host: 127.0.0.1 Database: tokenstore
-- ------------------------------------------------------
-- Server version 5.5.32
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;

CIP-ICT-PSP-2012-6 – 325161
Page 23 of 45
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `clientdetails`
--
DROP TABLE IF EXISTS `clientdetails`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `clientdetails` (
àppId` varchar(255) NOT NULL,
`resourceIds` varchar(256) DEFAULT NULL,
àppSecret` varchar(256) DEFAULT NULL,
`scope` varchar(256) DEFAULT NULL,
`grantTypes` varchar(256) DEFAULT NULL,
`redirectUrl` varchar(256) DEFAULT NULL,
àuthorities` varchar(256) DEFAULT NULL,
àccess_token_validity` int(11) DEFAULT NULL,
`refresh_token_validity` int(11) DEFAULT NULL,
àdditionalInformation` varchar(4096) DEFAULT NULL,
àutoApproveScopes` varchar(256) DEFAULT NULL,
PRIMARY KEY (àppId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Table structure for table òauth_access_token`
--
DROP TABLE IF EXISTS òauth_access_token`;
CREATE TABLE òauth_access_token` (
`token_id` varchar(256) DEFAULT NULL,
`token` mediumblob,
àuthentication_id` varchar(256) DEFAULT NULL,
ùser_name` varchar(256) DEFAULT NULL,
`client_id` varchar(256) DEFAULT NULL,
àuthentication` mediumblob,
`refresh_token` varchar(256) DEFAULT NULL
--
-- Table structure for table òauth_approvals`
--
DROP TABLE IF EXISTS òauth_approvals`;
CREATE TABLE òauth_approvals` (
ùserId` varchar(256) DEFAULT NULL,
`clientId` varchar(256) DEFAULT NULL,
`status` varchar(10) DEFAULT NULL,
èxpiresAt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

CIP-ICT-PSP-2012-6 – 325161
Page 24 of 45
`lastModifiedAt` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00'
--
-- Table structure for table òauth_client_details`
--
DROP TABLE IF EXISTS òauth_client_details`;
CREATE TABLE òauth_client_details` (
`client_id` varchar(255) NOT NULL,
`resource_ids` varchar(256) DEFAULT NULL,
`client_secret` varchar(256) DEFAULT NULL,
àuthorized_grant_types` varchar(256) DEFAULT NULL,
`web_server_redirect_uri` varchar(256) DEFAULT NULL,
àuthorities` varchar(256) DEFAULT NULL,
àccess_token_validity` int(11) DEFAULT NULL,
`refresh_token_validity` int(11) DEFAULT NULL,
àdditional_information` varchar(4096) DEFAULT NULL,
àutoapprove` varchar(256) DEFAULT NULL,
PRIMARY KEY (`client_id`)
--
-- Table structure for table òauth_client_token`
--
DROP TABLE IF EXISTS òauth_client_token`;
CREATE TABLE òauth_client_token` (
`token` mediumblob,
àuthentication_id` varchar(256) DEFAULT NULL,
ùser_name` varchar(256) DEFAULT NULL,
`client_id` varchar(256) DEFAULT NULL
--
-- Table structure for table òauth_code`
--
DROP TABLE IF EXISTS òauth_code`;
CREATE TABLE òauth_code` (
`code` varchar(256) DEFAULT NULL,
àuthentication` mediumblob
--
-- Table structure for table òauth_refresh_token`
--

CIP-ICT-PSP-2012-6 – 325161
Page 25 of 45
DROP TABLE IF EXISTS òauth_refresh_token`;
CREATE TABLE òauth_refresh_token` (
`token` mediumblob,
àuthentication` mediumblob
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
Download the SQL script
http://sunshine.sinergis.it/download/greenbutton/1.1/createSunshineTS_mySQL.sql and run inside the
“mySQL workbench” or some other SQL console connected to the mySQL instance that will host the
tokenstore DB .
2.2.3.3 Populating the token DB
The tokenstore DB should be prepopulated with some records in order to enable the application to create
the access token for the Sunshine platform.
A SQL script containing the insert statements can be downloaded at the address
http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineTS_mySQL.sql.
For the data exchange the needed record is the first one that describes the third party user (Sunshine
Platform). The other records are respectively:
 a user that the pilot can use to write and read his own data;
 a user that the pilot can use to read all the authorizations released to the third party applications.
In all these records a string “secret” is used as a default value for the field “client_secret”, but this should
be changed: they are the string that are used to encrypt the messages exchanged between the parties
involved in the OAuth flow. Obviously the three strings can be different: so change them in the SQL file

CIP-ICT-PSP-2012-6 – 325161
Page 26 of 45
created from the below snippet. This string must be pre-shared in some way with the third party (Sunshine
Platform)10
.
The value of “scope” is a coded string that specifies the parameters of the reading that Sunshine Platform
(the third party) is interested in.
The “access_token_validity” is expressed in seconds and is set to one year, while the
“refresh_token_validity” is set to five years.
Very important is the value of the call back URL that is set to
http://sunshine.sinergis.it/ThirdParty/espi/1_1/OAuthCallBack. It’s the address of the endpoint
of a ThirdParty service exposed at the Sunshine Platform and necessary for the authorization token release.
INSERT INTO `oauth_client_details` VALUES
('sunshine_platform', NULL, 'secret', 'read', 'authorization_code,refresh_token',
'http://sunshine.sinergis.it/ThirdParty/espi/1_1/OAuthCallBack', 'ROLE_USER', '31536000',
'157680000', NULL, 'FALSE');
('data_custodian_admin', NULL, 'secret', 'DataCustodian_Admin_Access',
'client_credentials', NULL, 'ROLE_DC_ADMIN', '31536000', NULL, NULL, 'FALSE');
('third_party_admin', NULL, 'secret', 'ThirdParty_Admin_Access', 'client_credentials',
NULL, 'ROLE_TP_ADMIN', '31536000', NULL, NULL, 'FALSE');
Download the SQL script located at
http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineTS_mySQL.sql.
 Change the “secret” strings contained in the script.
 Run the script inside the “mySQL workbench” or some other SQL console connected to the mySQL
instance that hosts the tokenstore DB .
10
Instead of using unsecure communication like mail or document transfer, one could use the web site
https://onetimesecret.com/ that allows exchanging privately small amount of data in a very simple manner.

CIP-ICT-PSP-2012-6 – 325161
Page 27 of 45
 Communicate the “secret” string and the endpoint of your DataCustodian installation to the Sunshine
Platform administrators in order to let them configure the ThirdParty.
2.2.4 DataCustodian application
After downloading the war file, it is necessary to modify some configuration files inside it. It is important to
do it inside the zipped war so use an editor that can do it or extract the file, modify it and then re-insert it in
the war file.
These are configurations that Tomcat must read the first time it is run, that is when unpacks and deploys
the war file content in its folder
<CATALINA_HOME>/webapps/
Then Tomcat is started for the first time and in this run it will do several one-shot actions; after this first run
it must be stopped, some other configurations must be done (this time in the deployed unzipped files) and
then Tomcat finally restarted to have the application working.
2.2.4.1 Pre-install configurations
The files to be configured tell the application which is the database to connect to, the behaviour to adopt
with the DB, and which tables create; the files that need to be modified are the following:
1. The first file is a “.properties”:
WEB-INFclassesspringmysql-data-access.properties
and must be modified setting the DB connection parameters:
jdbc.url=jdbc:mysql://<dbserver>:<dbport>/datacustodian
jdbc.username=<username>
jdbc.password=<password>
2. Then there are two XML files; the first is :
WEB-INFclassesspringbusiness-config.xml
and must be modified inserting the “create-drop” value in the following element:
<prop key="hibernate.hbm2ddl.auto">create-drop</prop>
This tells the application to create the tables needed eventually dropping the existing ones.
In the same file, insert the file name mysql-data-access.properties in the location attribute of the
element context:property-placeholder as follows:
<context:property-placeholder location="classpath:spring/mysql-data-
access.properties" system-properties-mode="OVERRIDE"/>
The second XML file is:

CIP-ICT-PSP-2012-6 – 325161
Page 28 of 45
WEB-INFclassesspringdatasource-config.xml
and in this file we must do the last same modification:
<context:property-placeholder location="classpath:spring/mysql-data-
access.properties" system-properties-mode="OVERRIDE"/>
3. The third configuration regards another XML file:
WEB-INFclassesspringoauth-AS-config.xml
Here find and edit the element “property” with attribute “baseURL” with the appropriate URL for the
DataCustodian application:
<property name="baseURL" value="http://<pilot DataCustodian server>/DataCustodian"/>
Inside this file there are also a lot of references to the “token DB” and the jdbc-style connection URL
with the user and relative password; these credentials must be changed to match the ones of the
installed instance of the mySQL DB and the connection URL must be set to the real one if it is not the
same server of the DataCustodian application (it’s default setting is
jdbc:mysql://localhost:3306/tokenstore).
2.2.4.2 Installation and first run
Copy the modified war file into folder:
<CATALINA_HOME>/webapps/
when Apache Tomcat is not running. Then start Tomcat and check if it connects to the database and creates
some tables in the “datacustodian” database inside the mySQL instance.
2.2.4.3 Post-installation configurations
Stop Apache Tomcat again; the datacustodian tables structure has been created during the first run and
now we must instruct the application not to drop and recreate the tables each time it is started. To do this,
we must modify the file business-config.xml anew, but this time the unzipped one in the Tomcat’s folder:
<CATALINA_HOME>webappsDataCustodianWEB-INFclassesspringbusiness-config.xml
Open it and insert the value “update” in the following element, overwriting the previous value:
<prop key="hibernate.hbm2ddl.auto">update</prop>
You can also drop the war file from the webapp folder and finally start Tomcat again and the DataCustodian
application should be up and running.

CIP-ICT-PSP-2012-6 – 325161
Page 29 of 45
3 Configuration and Use
In this chapter we will see how to set up the EUI database, feed it with the data coming from the meters
and let them be harvested by the Sunshine platform. In order to do so, the pilot should set up a mechanism
that feed the datacustodian DB ingesting the data from the original source (the meters loggers, the AMI,
the files coming from the utilities company …) to the standard tables of the datacustodian database and
expose the service that enables the transmission of data to Sunshine.
DataCustodian application provides two services (import & export) that allow these operations: the former
can be used by the pilot back-end procedures and the latter will be used by the Sunshine platform. Both are
protected by OAuth mechanism so they can be exposed on the internet.
3.1 datacustodian DB structure
The structure of the datacustodian DB that is relevant to the implementation of the Web Service and deals
with the consumption data is described in Figure 10.
Figure 10 - Database structure
Table retail_customers stores the authentication information of the service’s users. The users to set up are
two: the service administrator, who is represented by the pilot technical partner, and the “retail user”, that
is the owner of the data.

CIP-ICT-PSP-2012-6 – 325161
Page 30 of 45
Table usage_points stores the meter information. Every meter is a usage point and one or more usage
points can be associated to a single retail customer.
Table time_configuration stores the information about time zone and daylight saving time rules for the
area where the usage point is located. Consumption timestamps in the DB are stored in UTC format and
this type of information is used in order to convert data into local time frame, if needed. For the purposes
of Sunshine project conversion into local time is not needed, so the time configuration can be set to UTC
(see the following sub-section).
Table meter_reading lists the available types of reading coming from the usage points (the meters).
Examples of meter readings form the same usage point (i.e. a gas meter) can be “the set of readings for a
certain period” or “the readings at an interval of 4 hours” to be distinguished from “the readings at an
interval of 15 min”. Typically however every usage point is associated to just one meter reading.
Table reading_types describes the properties of the readings. Every meter reading is associated to a
reading type describing i.e. the type of energy measured, the frequency, the nature of the measurement
(absolute, integrated, etc).
Table interval_blocks stores the information about the regular data feeds into the DB. Each ingestion of
consumption data (be it a single reading or a chunk of readings) correspond to an interval block and is
associated to a specific meter reading.
Table interval_readings stores the actual data about consumption and cost. Each entry corresponds to a
single reading (energy consumption and its related cost, if available) and is linked to the interval block it
was part when it was imported in the DB.
As Figure 10 shows, part of the tables require only a one-time set-up, while tables interval_blocks and
interval_readings require regular updates as they store the actual dynamic consumption data. Section 3.2
describes how one-time tables can be set-up via direct SQL INSERT calls, while section 3.3 describes how to
insert dynamical consumption data into the DB and section 3.4 illustrates a procedure to extract the data
from the DB to verify its content.

CIP-ICT-PSP-2012-6 – 325161
Page 31 of 45
3.2 Populating the datacustodian database
This section is a collection of template INSERT statements to populate the one-time tables of datacustodian
DB. Important note: each entry of every table has to have a unique UUID11
which has to be generated by
the pilots.
The insert statements are gathered in a SQL file available for download at the address
http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineDC_postgreSQL.sql.
Note that some information is duplicated and are present in the tokenstore DB as well as in the
datacustodian DB.
As a general consideration, all these tables contain fields with names “self_link_href” or “up_link_href”
coupled with fields “self_link_rel” and “up_link_rel”. The values for these fields are respectively the URLs of
the resource itself (self_link_href) or of the resource of the upper hierarchical level (up_self_link) and the
qualifiers of these links. This information is used to form the response of the REST services following the
principle of the HATEOAS constraint of the REST architecture.
As a common characteristic, records have a UUID as external identifier, so this UUID must be regenerated
and substituted before running the script.
3.2.1 ‘retail_customers’ table
This table contains the users of the datacustodian application: the first user represents the retail customer
and will have a role that allows him or her to give a third party the authorization to access the data;
another role of this user is the feeding of the EUI data, using a specific import service as we’ll see in § 3.3.
Another user is the data custodian itself that is enabled for some administration activities that are not
important in this context.
The ids of a resource (in this case of a retail customer) are very important as in REST syntax they are used to
form the URIs that allow manipulating the resource itself: here we are using a progressive.
Retail user (building owner)
INSERT INTO retail_customers(
11
http://en.wikipedia.org/wiki/Universally_unique_identifier. An online UUID generator derived from the current
time can be found here (just for testing purposes): http://www.famkruithof.net/uuid/uuidgen

CIP-ICT-PSP-2012-6 – 325161
Page 32 of 45
id, description, published, self_link_href, self_link_rel, up_link_href,
up_link_rel, updated, uuid, enabled, first_name, last_name, password,
role, username)
VALUES (
1,'Ferrara municipality','2014-12-31','/RetailCustomer/1','self',
'/RetailCustomer','up','2014-12-31','106E8B03-0299-467E-972A-
A883ECDCC598','1','Servizio energia','Ferrara','F3rr@r@','ROLE_USER',
'ferrara');
Data custodian (Pilot admin)
INSERT INTO retail_customers(
id, description, published, self_link_href, self_link_rel, up_link_href,
up_link_rel, updated, uuid, enabled, first_name, last_name, password,
role, username)
VALUES (
2, 'Pilot Ferrara Data Custodian', '2014-12-31', '/RetailCustomer/2',
'self','/RetailCustomer','up','2014-12-31','106E8B03-0299-467E-972A-
A883ECDCC599','1','Sinergis', 'Sinergis', '21n3r612','ROLE_CUSTODIAN',
'admin');
3.2.2 ‘application_information’ table
This table contains information about the applications that are registered in order to use the datacustodian
services. Of particular importance is the application that has the attribute “kind” with value
“THIRD_PARTY”, that is the Sunshine platform.
3.2.3 ‘application_information_scopes’ table
This table contains the scopes related to the applications listed in the previous table.
3.2.4 ‘time_configurations’ table
Here we insert the two records that represent the UTC and CET Summer time zones codifications.
Time configuration: UTC

CIP-ICT-PSP-2012-6 – 325161
Page 33 of 45
INSERT INTO time_configurations(
id, description, uuid, dstoffset, tzoffset)
VALUES (
1, 'UTC', '/LocalTimeParameters/1', 'self', '/LocalTimeParameters', 'up',
'106E8B03-0299-467E-972A-A883ECDCC593', 0, 0);
Time configuration: Central European (Summer) Time
Annex 1 – DST rules describes how to compute the values of dststartrule and dstendrule in order to define a
Daylight Saving Time rule.
INSERT INTO time_configurations(
id, description, uuid, dstendrule, dstoffset, dststartrule, tzoffset)
VALUES (
2, 'Central European (Summer) Time', '/LocalTimeParameters/2', 'self',
'/LocalTimeParameters', 'up', '106E8B03-0299-467E-972A-A883ECDCC595',
decode('AE0E2000', 'hex'), 3600, decode('3E0E2000', 'hex'), 3600);
If you use a time zone different from UTC or CET summer time, please add it.
3.2.5 ‘usage_points’ table
A “usage point” in GreenButton is roughly equivalent to a single meter. It has a relation 1:n with the
“retail_customers” table that is a retail customer can be linked to more than one usage point. The code list
for the servicecategory_kind attribute is described in Annex 2 – Codelists.
INSERT INTO usage_points(
id, description, self_link_href, self_link_rel, up_link_href,
up_link_rel, uuid, servicecategory_kind, localtimeparameters_id,
retail_customer_id)
VALUES (
1, 'FER-001', '/RetailCustomer/1/UsagePoint/1', 'self',
'/RetailCustomer/1/UsagePoint', 'up', '106E8B03-0299-467E-972A-
A883ECDCC594', 1, 1, 1);

CIP-ICT-PSP-2012-6 – 325161
Page 34 of 45
3.2.6 ‘reading_types’ table
A reading type is a group of readings that share the same characteristics in terms of:
 type of commodity (gas, electricity …)
 unit of measure
 the type of measure (incremental or cumulative)
 the currency for eventual costs
and other more detailed and not mandatory features. A reading type is related to 1 or more meter reading.
The codelists for the attributes accumulationbehaviour, commodity, currency, timeattribute and uom are
described in Annex 2 – Codelists.
Just as examples the accumulationbehaviour code indicates if a measure of energy regards an interval of
time of the cumulative measure since the start of measuring; the commoditytype is the kind of energy
mean; currency is always euro; timeattribute is the reading frequency and finally the uom is the unit of
measure of the reading.
INSERT INTO reading_types(
up_link_rel, uuid, accumulationbehaviour, commodity, currency,
timeattribute, uom)
VALUES (
1, 'GASR_MCU_1h', '/ReadingType/1', 'self', '/ReadingType', 'up',
'106E8B03-0299-467E-972A-A883ECDCC593', 4, 7, 0, 7, 42);
Here is shown a gas meter reading, with a frequency of an hour.
3.2.7 ‘meter_readings’ table
A meter reading is not the real reading coming from the meter, but a way to group them in order to
simplify the relation with the usage point and the characteristics of the reading stored in the reading_types
table. In fact the real readings are stored in the interval_block table where only the relation with the
meter_reading table is necessary.
INSERT INTO meter_readings(

CIP-ICT-PSP-2012-6 – 325161
Page 35 of 45
up_link_rel, uuid, reading_type_id, usage_point_id)
VALUES (
1, 'FER-001_GASR_MCU_1h',
'/RetailCustomer/1/UsagePoint/1/MeterReading/1', 'self',
'/RetailCustomer/1/UsagePoint/1/MeterReading', 'up', '106E8B03-0299-467E-
972A-A883ECDCC592', 1, 1);
Here is shown the gas meter reading type is associated with the usage point #1.
Download the SQL script located at
http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineDC_postgreSQL.sql.
(If you installed the DataCustodian DB in a mySQL instance, just add single quotes to table and field names
and to values to get the mySQL syntax.
Change the UUID, the date values, the client secret and the DataCustodian endpoint. These changes
are indicated in the script file by comments. Even the retail user’s credential should be changed.
 Run the SQL script inside a SQL console connected to the datacustodian DB.
 Communicate to the Sunshine Platform admin the name of the registered retail users12
3.3 Import data service
In order to populate the datacustodian DB with the dynamical consumption data coming from the energy
meters, the DataCustodian application is provided with an import service. Consumption readings can be
imported one by one or in a contiguous group (as meters may not always be capable of sending reading in
real time, but instead may store them temporarily and send them in bundles) and each data import
corresponds to a new interval block item in the DB.
Consumption data has to be formatted in XML and posted to the import service whose URL specifies the
RetailCustomer, UsagePoint and MeterReading the data refers to. The service is a RESTlike service whose
12
At the moment the ThirdParty application has not a self registration functionality.

CIP-ICT-PSP-2012-6 – 325161
Page 36 of 45
URL indicates the resource to be modified (the IntervalBlock); a POST to this URL should be invoked with a
payload coded as an XML. Service URL and XML template are described below.
3.3.1 Service url
The URL of the import service is like this:
http://<pilot-greenbutton-server>/DataCustodian/espi/1_1/resource/<interval-block-up-link>
where <interval-block-up-link> has the form:
RetailCustomer/X/UsagePoint/Y/MeterReading/Z/IntervalBlock
Labels X, Y and Z are respectively the IDs of the RetailCustomer whose data are being uploaded and the
relative UsagePoint and the MeterReading they refer to.
Important note: generally speaking, different interval blocks of consumption data will correspond to
different UsagePoints/MeterReadings (as each pilot have more than one meter), so be careful to set X, Y
and Z values accordingly each time you call the import service.
i.e. http://sunshine-fe.sinergis.it/DataCustodian/espi/1_1/resource/RetailCustomer/1/UsagePoint/3/
MeterReading/2/IntervalBlock
Where "sunshine-fe.sinergis.it" is the base URL for the server exposing consumption data for the pilot in
Ferrara, and RetailCustomer id = 1 ("Ferrara municipality"), UsagePoint id = 3 (gas meter in kindergarten
Rampari) and MeterReading id = 4 (hourly reading of total cubic meters of consumed gas).
3.3.2 Consumption data XML
The XML-formatted data provided below is a template for the format consumption data that has to be
provided in to the data import service:
<ns3:entry xmlns:espi="http://naesb.org/espi"
xmlns:ns3="http://www.w3.org/2005/Atom">
<ns3:id>urn:uuid:3061204d-0148-1000-0000-000000000318</ns3:id>
<ns3:link href="http://sunshine-fe.sinergis.it/DataCustodian/espi/
1_1/resource/RetailCustomer/1/UsagePoint/3/MeterReading/4/IntervalBlock"
rel="up"/>
<ns3:content>
<espi:IntervalBlock>
<espi:interval>
<espi:duration>7200</espi:duration>
<espi:start>975628800</espi:start>

CIP-ICT-PSP-2012-6 – 325161
Page 37 of 45
</espi:interval>
<espi:IntervalReading>
<espi:timePeriod>
</espi:timePeriod>
<espi:value>1023</espi:value>
<espi:cost>1987</espi:cost>
</espi:IntervalReading>
<espi:timePeriod>
</espi:timePeriod>
<espi:value>1107</espi:value>
<espi:cost>2139</espi:cost>
</espi:IntervalReading>
</espi:IntervalBlock>
</ns3:content>
<ns3:published>2000-12-01T02:00:00+01:00</ns3:published>
<ns3:updated>2000-12-01T02:00:00+01:00</ns3:updated>
</ns3:entry></ns3:entry>
<ns3:id>
this will be the UUID of the interval block we are importing. As for all the items in the DB, it has to be
unique and has to be generated by the user13
.
<ns3:link >
this is exactly equal to the import service URL specified in the previous sub-section and is likewise
dependent on the specific IDs of the RetailUser, UsagePoint and MeterReading the imported consumption
data refer to.
<espi:interval>
This block describes the overall start and duration of the uploaded group of consumption readings.
Duration is expressed in seconds while the start date is expressed in Unix time14
, that is defined as the
number of seconds that have elapsed since the midnight UTC of January 1st
1970, not counting leap
seconds. Important note: This means that start date is expressed in UTC time reference frame.
13
See footnote 11
14
http://en.wikipedia.org/wiki/Unix_time. An online conversion tool (just for the sake of testing) can be found here:
http://www.onlineconversion.com/unix_time.htm

CIP-ICT-PSP-2012-6 – 325161
Page 38 of 45
This block describes start date and duration of the single reading, following the same format of the
espi:interval tag, as well as the measured consumption value and the related cost.
Consumption value has to be expressed as thousandth (10-3
) of the measuring unit (i.e. 1 kWh has to be
written as 1.000).
Cost has to be expressed as hundred-thousandth (10-5
) of the currency unit (i.e. 1 Euro has to be written as
100.000). Cost can be omitted if not available.
<ns3:published>
Both tags ns3:published and ns3:updated have to be populated and they have to report the same value,
corresponding to the timestamp of the end of the interval block, that is to say the start time of the
espi:interval plus its duration.
The format of the timestamp is the ISO Time15
format. Time can be specified in any time zone reference,
but UTC reference is preferred (i.e. 2000-12-01T01:00:00Z [UTC] and 2000-12-01T02:00:00+01:00 [CET] are
equivalent and are both accepted).
3.4 Export data
Consumption data properly uploaded into the DB will be then periodically harvested by the central
sunshine platform in order to feed the central sensor DB and be from there exposed to the final sunshine
user.
In order to allow sunshine central platform to properly ingest data coming from GreenButton export
service, each pilot has to fill a mapping file along the lines of the following template. Pilots should also
specify the ID of the RetailUser associated with the sunshine platform (recommended ID is ID = 1).
Meter identification
(from meter_mapping file)
Meter Reading ID Reading Type ID Usage Point ID
FER-003_GASA_MCU_1m 4 1 3
[…]
15
http://en.wikipedia.org/wiki/ISO_8601

CIP-ICT-PSP-2012-6 – 325161
Page 39 of 45
Pilots can test the export service by calling the URL
http://<pilot-greenbutton-server>/
DataCustodian/espi/1_1/resource/Batch/RetailCustomer/X/UsagePoint/Y
where labels X and Y are respectively the IDs of the RetailCustomer (has to be the “sunshine platform” user
defined in the previous section, typically id = 1) and UsagePoint of interest.
The export service must be exposed on the internet and should be reachable with HTTP protocol.
3.5 Security underneath the application layer
OAuth works at the same OSI layer of HTTP (application layer or layer 7). In order to achieve a complete
security of the data flow in GB “Connect My Data” scenario, OAuth is not enough and Transport Layer
Security (TLS) is required for all exchanges. Despite its name, TLS works at presentation layer (OSI 6 layer)
and during the negotiation is on OSI layer 5, session layer. The developers of the protocol recommend that
all implementations move to TLS version 1.2. However, most client web browsers are still at TLS 1.0 or at
best TLS 1.1. So, the following levels of TLS are required for the different actors:
 Data Custodian – Must implement TLS 1.2
 Third Party – Must Implement TLS 1.1 or higher
 Retail Customer – Can Implement TLS 1.0, 1.1, or 1.2
Each party must choose the highest level of TLS mutually supported during the TLS negotiation. Thus in the
rest of the document whereas in an URL an http: schema is written, in production you should use the
https: one.
3.6 Generation of the access token
The generation of the access token is done interactively by the retail customer accessing the GUI of the
DataCustodian web application installed at the pilot’s server and through a redirection to the GUI of the
ThirdParty web application installed on the Sunshine Platform, reproducing the interaction of the OAuth
protocol.
The generation of the access token involves the active participation of two actors over the web: the pilot
(DataCustodian) and the Sunshine Platform (ThirdParty). Thus they must know of each other
and must see each other over the internet. In other word their configuration must provide the
URL reference of the necessary service, the must share the same “secret”, the human user that
does the operation must be registered on both platforms.

CIP-ICT-PSP-2012-6 – 325161
Page 40 of 45
The steps are the following:
1. Access the DataCustodian GUI at the address:
http://<pilot-greenbutton-server>/DataCustodian/
(i.e. http://sunshine-fe.sinergis.it/DataCustodian/ )
2. Login the application
Here the retail customer should authenticate him/herself with the credentials that have been assigned to
him/her by the administrator of the DataCustodian application.
3. The application will show the user a welcome page where a menu bar allows to activate the following
functions:
a.Usage Points (list the configured usage points whose data are maintained in the DB)
b.Third Parties (list the configured third parties known to the Data Custodian and that can be
authorized to access the retail customer’s EUI data).
c. Logout
4. Click the “Third Parties” menu item and a radio button list (probably formed by just one element) of
organizations that can play the third party role will be shown. Aside the third party, there will be the
URL to which the retail user will be redirected for the scope selection phase. This is an address of the
ThirdParty application installed in the Sunshine platform.
5. Choose the “Sunshine platform” item and click next.
6. You will be redirected to the Third Party login page since you’re not authenticated in the Sunshine
platform, but only at the pilot’s side.
7. After the login, a radio button list with the type of authorizations that can be granted (scope) will be
shown. For our purposes, only a scope “read” will be in the list: choose it and click next.
8. The ThirdParty application will therefore redirect the user’s browser to a DataCustodian page (no
authentication needed this time) to confirm the intention of giving “read” authorization access to
“Sunshine platform”. Choose “Approve” and click “Submit”.
9. Now, another redirection back to ThirdParty application, to a page where the Access Token (and its
relative Refresh Token) generated by the sequence of operations just concluded are shown.
The generated tokens are also stored in the ThirdParty DB to be used by the Sunshine platform for
accessing the EUI data.

CIP-ICT-PSP-2012-6 – 325161
Page 41 of 45
Generate the access token for the Sunshine Platform by following the steps above. Before beginning,
assure that the retail user has been registered on the pilot’s DataCustodian application and on the Sunshine
platform ThirdParty application. There’s no self-registration, so these operations must be done by the
administrators. Take contact with the Sunshine Platform administrator to share the user name and his/her
password.

CIP-ICT-PSP-2012-6 – 325161
Page 42 of 45
Annex 1 – DST rules
Daylight Saving Time (DST) is defined by two rules, one identifying the start date of DST within a year and
the second identifying the end date. Each country or group of countries define its own rules, for instance
North America and EU use two slightly different set of rules. Each rule can be encoded in an 8-digit hex
string following the following schema.
The bit encoding:
Bits 0 - 11: seconds 0 - 3599
Bits 12 - 16: hours 0 - 23
Bits 17 - 19: day of the week 0 = not applicable, 1 - 7 (Monday = 1)
Bits:20 - 24: day of the month 0 = not applicable, 1 - 31
Bits: 25 - 27: operator (detailed below)
Bits: 28 - 31: month 1 - 12
Rule value of 0xFFFFFFFF means rule processing/DST correction is disabled.
The operators:
0: DST starts/ends on the Day of the Month
1: DST starts/ends on the Day of the Week that is on or after the Day of the Month
2: DST starts/ends on the first occurrence of the Day of the Week in a month
3: DST starts/ends on the second occurrence of the Day of the Week in a month
4: DST starts/ends on the third occurrence of the Day of the Week in a month
5: DST starts/ends on the forth occurrence of the Day of the Week in a month
6: DST starts/ends on the fifth occurrence of the Day of the Week in a month
7: DST starts/ends on the last occurrence of the Day of the Week in a month
An example:
EU DST rule for CET (UTC+01:00)
Start: Last Sunday in March
End: Last Sunday in October
Time: 1.00 am (01:00) Greenwich Mean Time (GMT)
http://wwp.greenwichmeantime.com/time-zone/rules/eu/

CIP-ICT-PSP-2012-6 – 325161
Page 43 of 45
The hex strings:
<dstEndRule>AE0E2000</dstEndRule>
<dstStartRule>3E0E2000</dstStartRule>
The conversion:

CIP-ICT-PSP-2012-6 – 325161
Page 44 of 45
Annex 2 – Codelists
The codelists reported in this annex are limited to the values that are used for the consumption meters in
the project sunshine. The following subsections will provide the mapping between the meter properties
already described by each pilot in the meter_mapping file into the proper value of the relevant codelist.
Each meter listed in the meter_mapping file is associated with a meter identification string:
<meterID>_<type><abs/rel>_<unit>_<freq> (i.e. FER-003_GASA_MCU_1m)
Where the following relations with GreenButton DB schema apply:
<meterID> UsagePoint description FER-003
<type> ServiceCathegory/Commodity GAS
<abs/rel> AccumulationBehaviour A
<unit> Uom MCU
<freq> TimeAttribute 1m
The following sections describe the mapping between the codelist values defined in the meter_mapping file
and the corresponding relevant values of the GreenButton codelists.
ServiceCategoryKind
Meter_mapping ServiceCategoryKind (GreenButton) Value
ELE 0 electricity
GAS 1 gas
THE 5 heat
AccumulationBehaviourType
Meter_mapping AccumulationBehaviour (GreenButton) Value
A 3 Cumulative
R 4 DeltaData
CommodityType
Meter_mapping CommodityType (GreenButton) Value
ELE 1 Electricity secondary
GAS 7 NaturalGas
THE 12 HeatingFluid

CIP-ICT-PSP-2012-6 – 325161
Page 45 of 45
TimeAttributeType
Meter_mapping TimeAttributeType (GreenButton) Value
15 2 15-minute
1h 7 60-minute
1d 11 Daily
1w 24 Weekly
1m 13 Monthly
1y 0 Not Applicable
ir 0 Not Applicable
UomType
Meter_mapping UomType (GreenButton) Value
KWH 72 Real energy (Watt-hours)
MCU 42 m3
(Cubic Meter)
LTR 134 Liter
CurrencyCode
Follows codes defined in ISO 421716
.
CurrencyCode (GreenButton) Value
0 Not Applicable
840 US Dollar
978 Euro
16
http://en.wikipedia.org/wiki/ISO_4217

Integrate SUNSHINE pilot smart meter data

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Integrate SUNSHINE pilot smart meter data

Similar a Integrate SUNSHINE pilot smart meter data (20)

Más de SUNSHINEProject

Más de SUNSHINEProject (7)

Último

Último (20)

Integrate SUNSHINE pilot smart meter data