Atmosphere 2016 - Adam Walach - Continuous IoT - with Docker, Go and Jenkins
•Descargar como PPTX, PDF•
1 recomendación•446 vistas
Technological advancements have taken solutions known from server rooms and introduced them into the IoT world. Modern operating systems can now be run on credit card sized computers, and their increasing processing power encourages us to use tools like containers and higher level programming languages on entire new class of devices. But managing complex IoT systems is not an easy task. Luckily software like Jenkins and Ansible can be used to create a well organized IoT environment. Devices with powerful communication capabilities can provide new features for E-commerce Platforms. This talk will show you how you can get into exciting new world of IoT. During the presentation we will combine all the technologies mentioned above into a working IoT solution. Warning - it’s not purely theoretical introduction to IoT for devops - I promise a lot of moving and blinking IoT toys! We’re gonna have fun!
Recomendados
Recomendados
Más contenido relacionado
Similar a Atmosphere 2016 - Adam Walach - Continuous IoT - with Docker, Go and Jenkins
Similar a Atmosphere 2016 - Adam Walach - Continuous IoT - with Docker, Go and Jenkins (20)
Último
Último (20)
Atmosphere 2016 - Adam Walach - Continuous IoT - with Docker, Go and Jenkins
- 2. Continuous IoT with Docker, Go and Jenkins Adam Wałach
- 3. ABOUT ME • Linux admin, programmer • Working at Hybris (now an SAP company)
- 4. The world has ended (this time for real) • current transistor size – 14nm • switch to 10nm delayed (Intel) • at 2nm – transistors would be just 10 atoms wide THE MOORE’S LAW IS DEAD!!! MORE THAN MOORE! • focus on low-power processor • diverse array of sensors
- 5. Exponentials can't go on forever, because they will gobble up everything -- Carl Sagan IoT on the rise! sharing, not just sending information
- 6. Tech evolution – unforeseen consequences “We won't experience 100 years of progress in the 21st century — it will be more like 20,000 years of progress (at today's rate)” - Ray Kurzweil
- 7. … a long time ago … my first PC (1998)!! and few years later…
- 8. What kind of IOT? – Industrial!
- 9. Mixed approach to IoT Sensors IoT hubs Cloud
- 10. Continuous IOT Continuous integration Continuous delivery Continuous testingContinuous deployment
- 11. RoboArm
- 12. How it’s made? + +
- 13. Big picture push source webhook push image commands
- 14. - It gives: - updates - on thousands of devices - very fast! - With Docker 1.10.0 it is finally possible to build Docker for ARM from the offical Docker repository docker pull ubuntu:14.04 – has no multi-architecture support Containerization
- 15. - Very fast compilation - Easy cross-compilation - Go compiles to machine code - Go supports concurrency at the language level - Binaries are statically linked Go
- 16. Jenkins 2 • Improved Usability • Easily define simple and complex pipelines through the DSL in a Jenkinsfile. • Backward Compatible
- 17. Software stack
- 18. Demo - RoboArm
- 19. Be careful with image processing
- 20. E-Commerce
- 21. E-Commerce
- 22. I'm Going To Build My Own Button - with Linux - and Go - and Docker
- 24. How it’s made? + +
- 26. Demo - yButton programming
- 27. Another use case – Happy Button Proposal: Every time you feel sad - press the button: it orders something random for 2$ from eBay/Aliexpress with free shipping
- 28. Is it really so simple? Raspberry Pi Compute Module
- 29. Ask yourself
- 30. References • https://github.com/adamwalach • http://www.itrs2.net/uploads/4/9/7/7/49775221/irc-itrs-mtm-v2_3.pdf • http://www.digikey.com/en/articles/techzone/2014/jan/short-range-low-power-wireless-devices- and-internet-of-things-iot • http://arstechnica.com/information-technology/2016/02/moores-law-really-is-dead-this-time/ • https://www.linkedin.com/pulse/digital-abundance-second-half-chessboard-keith-haviland • http://modberry.techbase.eu/
Notas del editor
- In the past I worked as linux sysadmin, I did a lot of weird stuff working as Oracle database administrator. I am programmer and a living example of nerd, in my free time I enjoy soldering stuff together [klik] and seems to me I’ve been like that for a very long time [klik]. Currently I work as an Infrastructure Architect at SAP Hybris For the last few years I was involved in creating multiple industrial IoT projects. Today I’d like to show you how to use skills and knowledge you already have to create IoT environment in best and easiest way.
- At the beginning I have an sad announcement. (dodaj słowa exponent grow) The world we know has come to an end. The fundamental law we all know is dead. Yes, Moore's law has died at the age of 51. It was sick for a longer period of time, but now its dead. Gone. But don’t worry, we’ll find another one! Why this happened? - It’s because manufacturing process of microprocessors can no longer be easily improved. Rather than focus on the technology used in the chips, the new roadmap will take an approach it describes as "More than Moore." The growth of smartphones and Internet of Things, for example, means that a diverse array of sensors and low power processors are now more important to chip companies. The highly integrated chips used in these devices mean that it's desirable to build processors that are not just logic and cache. They also include RAM, power regulation, analog components for GPS, cellular, and Wi-Fi radios, or even microelectromechanical components such as gyroscopes and accelerometers. Technology advancement will not stop simply because we are not able to make smaller transistors. It’s because manufacturing process of microprocessors can no longer be easily improved. Right now we’re at 14nm, it's unclear just how much further scaling is possible; at 2nm, transistors would be just 10 atoms wide, and it's unlikely that they'd operate reliably at such a small scale. Intel originally planned to switch to 10nm in 2016. In July of last year, the company changed this plan. 10nm are still planned but are not due until the second half of 2017. http://arstechnica.com/information-technology/2016/02/moores-law-really-is-dead-this-time/
- As Carl Sagan wrote: "Exponentials can't go on forever, because they will gobble up everything. So Moore exponential growth is over, lets see if there are other interesting ones. In fact there is one that’s still on the rise - it is the number of devices connected to the internet. And following that, the number of devices that are smart enough to interchange information with each other. And this is exactly what IoT is about – devices that are capable of sharing the information with their environment. Not just blindly sending collected data to some central location. Actually the death of Moore law helps with this process, because those smart devices need more of a System On Chip processors. The largest growth is predicted from all of connected small devices in areas such as home automation, smart energy, elderly home care, transportation, asset tracking, and many others. But remember - some of those connected devices may be smart, some are just primitive sensors http://www.digikey.com/en/articles/techzone/2014/jan/short-range-low-power-wireless-devices-and-internet-of-things-iot
- Unfortunately, almost by the definition, we are not able to predict how IoT will exactly evolve It’s the same situation like with moore law. Nobody in the 80’s was able to guess the outcomes of incrising scale of cpu integration There is a very adequate term that describes period of time that bring significant changes. It’s called singularity – as its an event that entirely flips rules of the game and you are unable foresee what coming up couse the whole ecosystem is far too complex. Ray Kurzweil describes it as a epoch of miraculous tech-driven change. Well, we’ll see about that https://www.linkedin.com/pulse/digital-abundance-second-half-chessboard-keith-haviland https://chasmtech.wordpress.com/2013/09/11/the-second-half-of-the-chess-board-intersects-with-moores-law-in-2013/
- Just a real world example: (Sorry, I could not resist, I just love this comparison) Take as an example my first pc, it was 1998, it cost something like 800Euros (german Marks back then) – and it was lot more money than it is today. It had 32MB of ram vs 256MB in rpi one. It had smaller storage space and exactly the same cpu performance. This step took us 15 years. But then, just 3 years later we have rpi 3 with 1gb of ram, 4 cpu cores and it costs 35$. Take a look at this chart, even with one core rpi3 its twice as fast as zero (or rpi1 – since it’s the same processor) The pace of technology advancement is accelerating. http://blog.pimoroni.com/raspberry-pi-3/
- And why are we talking about the all those changes? I mean it’s all fun and interesting but is there any other reason? Yes, there is: Because the extremely important fact is that smart devices that we will have to develop in near feature can not be made as a static, preconfigured set of components . They will have to evolve along the changing requirements of its own environment. I will be talking about that in the context of industrial IoT because in this environment it is easier to see problems or opportunities that are present in almost all embedded environments. And in industry adoption of new technologies can be very rapid If it brings savings of time and money. Those savings can be achieved when you bring together: - Intelligent machines Advanced analytics
- And the first problem one will encounter will be the amount of data you gather from different sensors. You don’t want to loose them, you care about every single measurement. And you want to analyze this data (in the cloud if possible). So on the one hand we have a lot of information to push and process, and on the other hand there is network infrastructure with limited bandwidth and stability issues. One solution is to use simple measurement modules and IoT hubs that will gather and preprocess the data. And this is the point where we can use all the power of modern operating systems and high level programming languages. Of course if you are not constrained by energy consumption or cost of sensor modules you can easily use embedded computers as endpoints – your infrastructure then can be all software defined. What does it even mean? It means hardware becomes a commodity and software abstracts and defines form and function of a device or product. My bet is: this is the direction the technology will take with the ongoing progress with “system on chip” technologies.
- One of major concerns surrounding the Industrial IoT is interoperability between devices and machines that use different protocols and have different architectures. This is a very hard problem to solve, there are multiple approaches to solving it. I can guarantee one thing. If you have development environment that is pleasant to work in, You can easily test your solutions and safely upgrade components then changing one standard of communication to another will be a breeze. And this is precisely what continuous IOT is all about
- Ok, this was a bit long, I don’t know about you (on your place?), but I’d like to see some of the real life stuff in action. Measuring temperature can be boring. Therefore as my first example I will use RoboArm and Raspberry Pi2 to show how modern environment for developing IoT can look like. But before that let’s go briefly over the technologies we will use. Tu trzeba wykonać jakiś ruch robotem.
- Its cheap, easly available Raspberry Pi2 Adafruit Mini Kit 16 3 servomechanisms Frame that keeps all the components together Very important thing is that all of those components are general purpose, its universal microcomputer, servo controller, or rather pwm generator - you could probably use it to dim Led light. It’s not ”doesn’t require any asemble kit”
- Schemat całego flow
- Dodaj kontekst continous Brak problemów które rozwiązuje Probably all of you have already used docker or containers in general, so only a few words about docker in the context of IoT. Docker gives us very needed separation from the operating system running underneath. On many embedded platforms there are problems with running newer versions of Linux distributions. Sometimes physical interfaces require special kernel modules and those are provided only for specific kernel versions. And you may want to use library that is not provided for the version of OS you have to support. What's more the version of library you require may be in conflict with the one that is running right now. Containerization of your app solves it. Next thing is testing: by embedding all the dependencies inside the container you can be 100% sure that your image on production will behave exactly as it would on your testing platform. Very important information: Docker plans to add multi-architecture support in docker registry, so for instance if you say: Pull …. it will be able to download version adequate for your system. https://github.com/docker/distribution/pull/993 https://github.com/BrianBland/distribution/blob/b0f41ba1a313a536c640270e0c64b542e002a0e2/docs/spec/manifest-v2-2.md
- In continuous environment its beneficial to have programming language that compiles vary fast The obvious choice for modern programing language is Go. Go compiles to machine code - Its very good choice for platforms with limited processing power. There is also no need for runtime environment, so your docker image can be smaller Go supports concurrency at the language level – almost always have to do few things simultaneously – read data from few interfaces, update lcd screen Go has garbage collection - its just safer then wi Strings and maps are built into the language Binaries are statically linked Go is very strongly typed Easy cross-compilation Cross compilation for arm platform Easy to read, first language when I am able to read source code of build in libraries and actually understand it!
- Last but not least For new users, Jenkins 2 starts up with suggested plugins to get them started with the right set of tools to use Jenkins effectively from day one. There are a number of user interface improvements, such as the improved job configuration page Jenkins 2 is a drop-in replacement of the Jenkins 1.x series of releases and fully backward compatible. There is practically no reason not to upgrade!
- The same on all devices, very thin operating system layer Easy to reproduce environment You could say: raspbian sux, some packages are very old and so on
- Those were the basic technologies, the rest I will explain throughout the demo Now a question: how many of you used Jenkins pipelines with Jenkisfile in the same repo as project Demo session - How to create a set of steps in Jenkins, well you can click through all of the step manually show pipeline – the only thing you have to configure is owner, the rest is completely automatic explain how it works Show go project, tell about vendoring Go through steps of Jenkins file https://github.com/adamwalach/go-robo-arm-demo/settings/hooks/8276037 Look how simple it is, but how efficiently you’ll be able to orchestrate your environment. You can sit in one place, with one project, and control whole network of devices. So what does the python program do:
- Cat tree recognized as human face
- I promise will get back to live demo soon Currently the big thing is E-commerce, so we can build IoT solution interacting with ecommerce systems. The smart fridge has been used as the real-world example of the connected home for many years. The idea is that your fridge will know when you're running out of milk and order the product from your online grocer who in turn delivers the product straight to your door. In some cases right now you can change the temperature setting of your fridge from your mobile app, but a smart fridge should be able to tell you exactly what is missing and communicate things like when then food is close to the expiry day. And bathroom scale on the right also have all the features that qualifies it as a device belonging to ecommerce system - it is internet connected - it sells you useful information - it wants your money The fact that it tries to earn it through extortion is irrelevant. http://www.visionmobile.com/blog/2015/12/the-internet-of-things-is-about-to-reshape-e-commerce/ http://www.essentialretail.com/essential-ecommerce/article/55dc3eeac79c7-how-could-internet-connected-smart-fridges-change-the-face-of-online-retailing
- Example of IoT appliance that works in real life is something much simpler Amazon created something called Dash Button Easy to use. You just press it to order your favorite products. You can say that one does not simply “Push a button”. Yes he does! And it works quite nicely. It is smart enough to not order new product with every push of a button, at least until the previous order is delivered. And you know what is comming up next, dont you?
- And you know well what’s next, don’t you? I'm Going To Build My Own Button with Linux and go and docker
- We can have our own, in Hybris we have a great ecommerce platform. What we need is an IoT device. Maybe we can create one with RaspberryPi zero Of course we can, and this is how we will go about doing it: Just a disclaimer: it is not a SAP Hybris product, its just something I created because, like I said, I like to soldier stuff. And because in Hybris we have something called D-Shop (developers’ workshop) I was able to use our new 3d printer to create a case for it.
- Show Hybris shop Raspberry pi zero and how it looks like And this time will try to be a little more ambitious about the pipeline: - hipchat integration – in the development process of embedded devices this is a novelty - artifacts - logs size of docker container
- https://xkcd.com/576/ Either you get into addiction, or you’ll feel less sad Of course the example is here just to remind you about infinite number of possibilities you have. If whole environment is software defined the only limit is your imagination.
- You can say that I am overestimating the simplicity of creating hardware. 3d printed case is not production ready, but it makes a great prototype You are limited by your imagination, when you want to create prototype of IoT device hardware cost is no longer a factor Raspberry Pi SO-DIMM compute module, powered by Broadcom 700MHz To sum up: I showed you the variety of different use cases and some demo devices to prove that hardware components are really power full and in your reach. And in order to solve IoT software problem that is in fact not yet precisely defined you have to be equipped with development tools that will allow you to adapt to constantly changing requirements. Software defined solutions are the future of IoT and computing
- You really should take a look at new tools that are available. Technological advancement is very rapid, even very small devices now have capabilities to run high level software. Your whole IoT environment can be software defined My recommendation: Don't be afraid, try to explore new ways of solving problems If you want to see the source code please check my github account. And that’s all Thanks for your attention. Now I'll try to answer any questions you may have.