Webcast Four: Ruby Business Intelligence and Analytics on Bluemix
Meetupslides 150409100501-conversion-gate01
1. IBM BLUEMIX ALLOWS YOU
TO QUICKLY CONNECT TO
THE CLOUD
Meetup Atlanta April 2015
Valerie Lampkin
vlampkin@us.ibm.com
Twitter: @vjlam30188
2. AGENDA
Internet of Things (IoT) and Cloud
What is Bluemix?
- Cloud offering that includes dashboard, catalog, services,
docs, runtimes, boilerplates, recipes
IoT Foundation
- powered by MessageSight technology, runs on Softlayer,
stores info in Historian
MQTT protocol
Quickstart and registered devices
Cloud Foundry, Docker, VMs
Node-RED
Troubleshooting, dwAnswers forum
Real world examples
8. My husband says pizza is the answer to everything
Using “pizza-as-a-service” analogy:
•On Premise = you buy everything and make the pizza at home
•IaaS = take and bake (pick up the pizza, you cook it at home)
•PaaS = pizza delivered
•SaaS = dining in the restaurant
9. On Premise = your business handles everything, provisioning hardware, software,
IaaS = the infrastructure, storage, networks, other fundamental resources are
provided and you manage the software and your applications.
PaaS = the hardware (cloud environment), software and runtimes are provided for
you but you manage your own applications
SaaS = everything is provided by another party and you use their software service
12. Bluemix Dedicated provides additional isolation for your sensitive data while leveraging
public Bluemix services. Using your most sensitive data, you can build and run apps in
Bluemix Dedicated, as well as pull in services from IBM’s public Bluemix catalog
13.
14.
15.
16.
17. Sign up for a free trial at:
https://ibm.biz/Meetup_Atlanta
18.
19.
20.
21.
22.
23. Leverage the power of Docker to deploy a container
on Bluemix using the IBM Containers service
https://developer.ibm.com/bluemix/2014/12/04/ibm-containers-beta-docker/
24.
25.
26. Bluemix allows you to integrate your IoT devices and applications
with other Bluemix services for a complete IoT solution
27.
28.
29.
30.
31. When connecting to the QuickStart service no authentication (or registration) is required
A Device must be registered before it can connect and authenticates using a
client ID in the format: d:org_id:device_type:device_id
An Application authenticates using a client ID in the format: a:org_id:app_id
43. RELIABLY AND QUICKLY DELIVER IOT DATA WITH MQTT
Open royalty-free spec
Wide variety of clients and
servers
•Hobbyist to enterprise
•Open source to commercial
Minimal pub/sub messaging
semantics
•Asynchronous (“push”)
delivery
•Simple set of verbs --
connect, publish, subscribe
and disconnect
Minimized on-the-wire format
•Smallest packet size 2 bytes
Scalable
Low footprint
•Clients: C=30Kb; Java=100Kb
Three qualities of service
•0 – at most once delivery
•1 – assured delivery dups ok
•2 – once and once only delivery
Copes with loss of contact between
client and server.
•“Last will and testament” to publish a
message if the client goes offline.
49. 1. Create a Bluemix application
2. Write code in the cloud
3. Store code in Git repository
4. Plan and track teamwork
5. Build, deploy, and test your cloud app
60. CHALLENGES OF BOAT RACING
0
Fans
Commentators
• Web Streaming, On Shore
Viewing
• Limited Viewing & Vantage Point
• Multiple Classes (Who is
Leading?)
Race Officials • Validation of Race Winner
Race Teams
• Immediate Detection of Component
Failures
• 80+ Telemetry Data being Captured 5/sec
• Need to Predict Component Failures
Solution: SilverHook, Virtual Eye and IBM
61.
62. IoT Zone in Bluemix https://bluemix.net/solutions/iot
Explore the Recipes https://developer.ibm.com/iot/
Visit the main Landing page https:/internetofthings.ibmcloud.com
Follow the community on Twitter @IBMIoT
Start your IoT project today!
63. IBM Bluemix will be offered as the cloud
development platform for the
NASA Virtual Space App Challenge. This
code-a-thon is April 10- 12, developers
can rapidly build applications that
contribute to space exploration and
solve global challenges.
https://2015.spaceappschallenge.org/
64. ADDITIONAL RESOURCES AND LINKS
Cloud Foundry: http://www.cloudfoundry.org
Get Started with Bluemix: https://ibm.biz/Meetup_Atlanta
IoT Foundation: https://internetofthings.ibmcloud.com/
OASIS: https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt
Bluemix Doc: https://www.ng.bluemix.net/docs/#overview/overview.html#overview
dwAnswers: https://developer.ibm.com/answers/smart-spaces/12/bluemix.html
Recipes for IoT: https://developer.ibm.com/iot/
Node-RED: http://nodered.org/
MQTT: http://mqtt.org/
DOCKER: https://www.docker.com/
Notas del editor
Intro Slide
Internet of things, proliferation of intelligent devices.
creates a need for businesses to process big data and provide real-time analytics and response.
The value of IoT is in the analytics of the data captured.
Sensors and other monitoring devices are embedded into many objects, systems and processes
In the globalized world, people, systems, objects and processes are connected and communicating with one another in entirely new ways.
Leveraging the data generated provides intelligence to help do things better, improving responsiveness and the ability to predict and optimize for future events
Benefits of real-time data from IoT
Benefit of cloud – elasticity.
Albert Barron is the IBM Bluemix team member who came up with this idea to help people grasp the difference types of services.
Bluemix allows developers to expedite their release times by eliminating the need to manage and maintain infrastructure. Bluemix allows developers to focus on the code without the worry for getting hardware provisioned and software installed. IBM is handling those tasks.
By obtaining cloud services, you can increase standardization, lower costs and observe a faster time to value. With Bluemix handling the servers, storage, software the time to develop and deploy is quicker. Software upgrades and fixpack installations are handled by IBM, this helps customer stay up-to-date on supported current versions that offer latest features.
Bluemix is now evolving to be more than just a public PaaS. Softlayer provided dedicates servers, managed hosting and cloud computing services.
For customers who have confidential and sensitive data and may be reluctant to use the cloud, IBM has dedicated Bluemix instances available. Customers are provided with a private, single tenant instance of Bluemix running in a Softlayer environment that is completely managed by IBM.
With Bluemix public, dedicated and the addition of local customers can have Bluemix running where the business needs.
We envision our customers will actually uses a combination of Bluemix dedicated and local as well as public.
For example, an app could be developed and deployed in dedicated or local but then want to consume a service provided by that app in another app running in Bluemix public. With these three options available, a true hybrid cloud solution is possible using Bluemix.
Visibility, control and security everywhere data and services exist
Transparency to see all data and apps across infrastructures
Control and governance to manage and orchestrate workloads and access
A multi-layered approach to ensure security across every interaction point
Developer productivity to easily integrate, compose and deliver modern apps
connect and compose data and services for enterprise, web and mobile apps
bring apps closer to data or data closer to apps
quickly develop, refine and deploy apps with hybrid cloud DevOps
Data and analytics to provide the best insight from all relevant data inside and outside the organization and optimize data sovereignty and locality
Data services to replicate and synchronize data
Analytics services to refine data and provide insight
Cognitive computing services to interact with and change the world
Internet of Things Foundation provides capabilities including device registration, connectivity, control, rapid visualization and storage of data derived from the Internet of Things
The IoT Foundation lets your apps communicate with and consume data collected by your connected devices and sensors. Your apps can use real-time and REST APIs to communicate with your devices and consume the data you set them up to collect.
First you get your devices connected, then you can build an app that talks to your devices. For example you might have a sensor that collects and sends humidity readings every minute. Our REST and real-time APIs allow you to quickly pull that device data into your apps for further analysis.
Free 30 day trial period https://ibm.biz/Meetup_Atlanta
A Bluemix boilerplate is a runtime and includes a sample application, the boilerplate makes it easy to get started.
The IoT boilerplate includes a Node-RED application and a Cloudant database. You can try the sample flow with a simulator and customize it for your own devices.
Many different languages and runtimes are available on Bluemix. Watson cognitive analytic services may also be incorporated into your application. IBM also provides various other services (open source and/or 3rd party services). Some services may be offered as experimental or “beta” services.
Bluemix also provides Watson services, an artificial intelligence-enhanced analytics system allowing you to build cognitive apps that help enhance, scale, and accelerate human expertise
Cloud Foundry is at the heart of Bluemix. The Open-source technology enables the deployment of applications (and scaling and monitoring). The Cloud Foundry Foundation brings open governance, ensuring consistent functionality and behavior of core components across vendors.
IBM Containers are powered by Docker, and allow developers to package their applications and runtimes then deploy as containers on Bluemix
IBM Container service now exists alongside Cloud foundry apps deployed in Bluemix, allowing users to deploy containers as they would any other app in Bluemix.
So if you package up and app and runtime with docker, you can then deploy it into Bluemix using the container service.
Interconnect conference last month announced that users can now create VMs in Bluemix (like you would get from an IaaS like Softlayer). The VMs will live right alongside your Cloud Foundry aopps and Docker containers in Bluemix.
IoT foundation is powered by MessageSight technology (can handle a million concurrent connections and 30 million messages per second). It can be purchased as a standalone service (through the IBM Cloud Marketplace) or as part of Bluemix
Bluemix allows you to integrate your IoT devices and applications with other Bluemix services to create a complete end-to-end IoT solution
When you create a IoT service in Bluemix, you can then bind it to your application
Free 30 day trial then pay as you go.
Different levels (Bronze, Silver, Gold) are available for the IoT Foundation service.
Historian - paying customers get a longer retention period such as 6 or 12 months. Freemium customers have only a 1 month retention period. Customers may download their Historian data before it is purged at the end of their retention period
The Bluemix website has a wealth of documentation to help users get started with IoT ( and other services as well).
A very easy way to try it out is to use the IoT device simulator QuickStart example (you don’t need a device to test out an IoT app).
A Device must be registered to authenticate if you are not using the “quickstart” feature.
Here I have a Raspberry Pi device on the right and an Arduino Uno on the left. The recipes can help you go from unpacking “out of the box” to testing a live application in the time it takes to drink 2 beers.
IoT recipes are available to help you configure, connect and visualize your devices and develop applications to gather insights from the data.
New recipes are being added and we welcome partners to join the IBM ecosystem. Contributing a new recipes is a great way to demonstrate the technologies of IBM partners (both as advertising and technical how-to).
The “Use my own device” recipes provide all the information and background needed to create your own .
Connectivity to the IoTFoundation service uses MQTT as the “event/message oriented” IoTprotocol
An MQTT client library is used by the “quickstartdevice” application to handle the MQTT communication
There are many MQTT client libraries available
–A list can be found here: https://github.com/mqtt/mqtt.github.io/wiki/libraries
–IBM contributes and maintains a set of libraries in the Eclipse.org Paho projects. This includes Java, JavaScript, C, embedded C libraries. https://eclipse.org/paho/
–The embedded C client is designed for use on constrained systems with ease of porting device recipe
When a device is connected to the IoT Foundation, the data from remote devices can be managed within the cloud, using predictive analytics and real-time processing. Eclipse Paho MQTT clients can be downloaded from:
http://www.eclipse.org/paho/download.php
IBM Mobile Messaging and M2M Client Pack MA9B provides two Java client libraries. For more details about MA9B, see website:
http://www-01.ibm.com/support/docview.wss?uid=swg27038199
Connecting remote apps directly to a business’ master data management backend system can create challenges. As the mobile apps scale, the load on the back end systems grow enormously. Additionally, if you have to make a change to your system of record, your mobile apps experience an outage. A “system of engagement” tier allows for cloud scaling and to mash up information from systems of record as well as other cloud sources and the Internet of things.
Connect, Collect, Assemble, Manage
TLS v1.2 is used for IoT Foundation security
Bluemix IoT Foundation utilizing API keys and tokens to authenticate ClientIDs. TLS v1.2 support is provided for connections utilizing MQTT protocol.
The IOT Foundation uses MQTT protocol. The MQTT protocol is based on the principle of publishing messages and subscribing to topics (a PUB/SUB model).
MQTT has a simple API. Connect, publish, subscribe, disconnect.
Devices located at the edge are able to participate as “systems of engagements” and communicate with enterprise systems on the backend (systems of record) this allows you to create a complete end-to-end solution including the internet of things.
MQTT’s origins go back nearly two decades. It’s co-inventor, Andy Standford-Clark is a distringuished engineer at IBM. His industry partner was Arlen Nipper, an American engineer and expert in embedded systems for oil field equipment. Together they wrote the initial version of MQTT in 1988. They have continued to improve their open-source messaging software over the years.
https://www.oasis-open.org/news/announcements/mqtt-version-3-1-1-becomes-an-oasis-standard
https://www.oasis-open.org/news/pr/foundational-iot-messaging-protocol-mqtt-becomes-international-oasis-standard
MQTT v3.1.1 was ratified as OASIS standard end of 2014, many devices still have the older version but the new version
Includes some enhancements such as the ability to use a longer string for your ClientID
In version 3.1.1 the maximum length of the MQTT Client Identifier (ClientId) is increased
MQTT was chosen by IBM as the optimal protocol for its IoT foundation offering.
MQTT is Open, lean, simple and reliable.
Node-RED is lightweight runtime built on Node.js The Node-RED drag and drop feature allows for easy implementation.
Here is a simple node-RED flow that I have tested. Whenever I make a change, I then simply click the red “Deploy” button again to update my application.
You can increase the number of instances and memory as needed.
There is where the elasticity of the cloud comes in handy. Bluemix allows you to quickly develop and deploy your application, then you can scale it as needed at a later date when volumes increase.
While on the Bluemix dashboard, if you have an application open, you will see the “ADD GIT” feature in the upper right hand corner. By clicking on that a Git Repository will be created within DevOps and a Git URL will then be shown with a button to Edit Code. You will also get an email with Project and GIT URLs
Clicking that button will link you over to DevOps and you can then edit your code inside the Git Repository
IBM’s Bluemix DevOps Track and Plan services can be used to create stories, tasks and effects to describe and track project work and use agile planning tools for backlog, releases and sprints.
If you are using a node-RED flow (such as created by IoT starter boilerplate) the flow itself will be stored in a cloudant database which is provisioned when you create the boilerplate.
The “community” is great feature of Bluemix
When you enter the community/forum you are taken to the dwAnswers website within developerWorks. Here you will see questions tagged with Bluemix (and/or other tags such as IoT).
Paying customers can create Parature or RTC tickets, but may still choose to browse and search forums first for a quick answer. Doesn’t everyone google something these days when they don’t know how to do it? The forum experience is what customers are using these days (similar to stack overflow). It allows you to search and find answers to problems, or post a new question which may be answered by an IBM expert or another developer who has tried the same thing and figured out how to make it work!
Hackathon Example
“idiot driver” application. Uses the familiar in-vehicle Human Machine Interfaces (HMI) such as Voice Command, Steering wheel and radio buttons.
Imagine you are driving down the road and some “idiot” cuts you off and swerves in front of you. Since you have the PureFocus application on your smartphone and it’s paired with the SYNC AppLink head unit, you hit the command button on the steering wheel and say “REPORT DRIVER.” The voice recognition system in the car passes the command to our app through SYNC AppLink and then asks the driver, “WHAT HAPPENED?”
Using the Audio Pass Through feature of SYNC Applink, we capture audio from the driver who say something like: “Some jerk just cut me off.” The app sends audio clip back to Watson, which can interpret the language (even if it’s not as PG as “idiot”)
How do we know who they are?
We determine the offending driver using their geo-location, steering maneuvers, speed, and braking pattern data that comes from the vehicle through SYNC AppLink. Assuming they have the application installed, their profile is tagged with an “IDIOT” event that decrements their safe driving score for the offending action. Additionally, the app allows you to gamify the whole thing and challenge your friends on Facebook or Twitter to be safer drivers.
The application was built in iOS using IBM Bluemix, Watson Speech to Text, Watson Concept Expansion, Cloudant, and IoT Foundation. A demo was also created using Byan’s Connected Vehicle Maps hackathon kit hosted on m2m Demos site to visualize how the solution works.
Real World Examples
Here are some existing real world examples for how the Internet of Things devices and data are being used today.
Connected cars, wearables, health monitoring, and others. Internet of things is
Fans of boat racing are on the shore, when the boats are out on open water either in the ocean or large lakes, they spectators can’t truly always see what is happening. Bluemix IoT helped create a solution to allow fans to be more engaged and get real-time updates from the boats even when they were not close enough to visually see from shore.
Solution: SilverHook, Virtual Eye and IBM
The SilverHook Telemetry application provides a foundation for
Real time processing of telemetry data to monitor the boat for potential failure conditions
Logging of all raw telemetry, scores, and generated alerts for post-race analysis
Real-time 3D rendering to enhance the fan and team experience
The application is built upon multiple Bluemix capabilities, including:
Internet of Things Foundation
dashDB
Scoring service
Java / Liberty app server runtime
Python buildpack (third party)
Start your IoT project today!
NASA announced that IBM Bluemix will be offered as the cloud development platform for the NASA Virtual Space App Challenge. This code-a-thon will help developers rapidly build applications that contribute to space exploration and solve global challenges.
https://2015.spaceappschallenge.org/
Cloud Foundry: http://www.cloudfoundry.org
Get Started with Bluemix: https://ibm.biz/Meetup_Atlanta
IoT Foundation: https://internetofthings.ibmcloud.com/
Recipes for IoT: https://developer.ibm.com/iot/
MQTT: http://mqtt.org/
OASIS: https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt
Node-RED: http://nodered.org/
Bluemix Documentation: https://www.ng.bluemix.net/docs/#overview/overview.html#overview
dwAnswers: https://developer.ibm.com/answers/smart-spaces/12/bluemix.html
Docker: https://www.docker.com/