MiCADOscale presented at EGI conference 2019
•Descargar como PPTX, PDF•
0 recomendaciones•92 vistas
MiCADO (=MiCADOscale) is the outcome of the Research and Innovation Action "Cloud Orchestration at the Level of Application", short: Project COLA. Open source as a matter of course Scale your software at application level - MiCADOscale allows automated scaling at the level of the virtual machine as well as at the level of the container. Multi Cloud Support - MiCADOscale supports various cloud infrastructures like AWS, OpenNebula, OpenStack, CloudSigma & Cloudbroker Open source based framework - MiCADOscale is a generic framework, supporting Kubernetes, Occopus, Prometheus etc.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a MiCADOscale presented at EGI conference 2019
Similar a MiCADOscale presented at EGI conference 2019 (20)
Más de Project COLA
Más de Project COLA (7)
Último
Último (20)
MiCADOscale presented at EGI conference 2019
- 1. MiCADO - A highly customisable multi-cloud orchestration and auto- scaling framework Tamas Kiss, James DesLauriers, Gregoire Gesmier, Gabriele Pierantoni, Gabor Terstyanszky University of Westminster, UK Jozsef Kovacs, Peter Kacsuk MTA SZTAKI, Hungary Andreas Ocklenburg CloudSME, Germany
- 2. • Take-up is still relatively low – especially by smaller companies (SMEs) • Multi-cloud applications are rare – vendor lock-in • Elasticity and dynamic auto-scaling at the level of applications is limited • Security concerns imposed by virtualization and third party provisioning Motivations - Cloud computing reality check ? ? ? ? ?? ? ?? ? ? ? ?? www.project-cola.eu
- 3. Dynamic application level autoscaling Achieve resource scalability and efficient resource utilisation • to serve variable number of customers with dynamic resource demands • to optimise resource consumption and costs www.project-cola.eu Application 1 Application 2 Application N Service 1 Service 2 Service 3 Service 4 Service 5 Baseline resource consumption Variable resource consumption Cloud services Dynamic demand Manually adjusted supply Resource requirements To be replaced by automatically adjusted supply
- 4. www.project-cola.eu Coordination/Orchestration App1 App2 AppN Developer produces Deployment executor Deplo yment instru ction Security facilitator Optimisation decision maker Cloud Measurement & metrics collection Data The way to achieve scalability
- 5. The COLA Project COLA – Cloud Orchestration at the Level of Application • EU H2020 • 1st January 2017 - 30th June (30th October) 2019 • Project value: 4.2 million Euros • 14 project partners from 6 European countries • 10 companies and 4 academic/research institutions • More information: https://project-cola.eu www.project-cola.eu
- 6. Project objectives Overall objective: • Define a generic pluggable framework, called MiCADO (Microservices-based Cloud Application-level Dynamic Orchestrator) that supports optimal and secure deployment and run-time orchestration of cloud applications. Detailed objectives: • Design, prototype, test and demonstrate a set of services in a generic framework that cloud application developers can utilise from their application source code. • Pilot, demonstrate and validate the technical feasibility of the MiCADO framework in SME and public sector case-studies. • Validate economic feasibility of the implemented use-cases. • Define common and widely applicable application templates. • Provide access to heterogeneous, federated and distributed cloud resources. • Develop solutions to address security, reliability and trustworthiness. • Maximise impact by focused dissemination and marketing campaign. www.project-cola.eu
- 7. MiCADO Microservices-based Cloud Application-level Dynamic Orchestrator - Target audience: cloud application developers – application operators - Automated scaling based on highly customisable scaling policies - Scaling at both container and virtual machine levels - Multi-cloud support – for e.g. AWS, OpenStack, CloudSigma - Standardised TOSCA-based application and policy description - Modular architecture based on open source components - Easy Ansible-based deployment - Intuitive dashboard - Policy driven security settings - Open source - https://github.com/micado-scale www.project-cola.eu
- 8. MiCADO Architecture – generic modular design www.project-cola.eu Node/container monitor Node/container monitor MICADO WORKER NODE Info on nodes/containers Container create/destroy/scale up/down, node evacuation, etc. Container Orchestrator Worker node create/destroy/scale upCloud Orchestrator Monitoring System MiCADO Submitter Policy Keeper Register policies Scale/update worker nodes Scale/update containers description on infrastructure and policies Create Worker nodes MICADO MASTER NODE container container container Optimiser Advice Parameters MICADO WORKER NODE Container Executor Create container infra Container Executor
- 9. 1. Deploying MiCADO by customizing Ansible configuration files 2. Describing your application (for e.g. virtual machine, scaling policy, etc.) by creating/ customizing TOSCA-based ADT file (Application Description Templates) 3. Submitting your ADT file visa REST API call 4. Tracking MiCADO master and worker nodes in Dashboard www.project-cola.eu Cloud MiCADO Master Ansible MiCADO MasterADT topology_template: inputs: app_image: type: string description: Docker image to run for the application required: yes … Worker MiCADO – How to use it?
- 10. An application example Resource-based scaling • Collecting and analysing Twitter posts to find out people’s opinion about local government services and to tailor and improve these services to their needs. SARGA is an Aragón Government public enterprise Aragón region Government 50.000 employees Aragón Region 1.300.000 pop. 50.000 km2 area
- 11. www.project-cola.eu Resource-based scaling MICADO MASTER Client node: 1 deploy 2 undeploy REST Calls (ADT) WEB view Dash board REST API CMD-line view Scaling logic magician MICADO WORKER Scale up when CPU utilisation is above threshold Scale down when CPU utilisation is below threshold
- 14. Repast Symphony (open source agent-based simulation framework): It has large number of variable lengths jobs that need to complete by a set deadline with minimum or at least constrained set of resources www.project-cola.eu An application example Deadline-based scaling • Infection network simulation • 3 types of agents: • Susceptible • Infected • Recovered
- 15. www.project-cola.eu An application example Deadline-based scaling Amazon/CloudSigma Cloud jQueuer Master webUI port 8081 MiCADO Master TOSCAsubmitter API calls … External File Server Downloadinputfile Uploadoutputfile MICADO_*: IP, port and SSL credentials for accessing the MiCADO Master DEADLINE: Max running time (seconds) DURATION: Estimated time for one job VMS_MAX: max number of VMs CONTAINERS_MAX: max number of containers per VM WORKER CPU/RAM: Specification of worker nodes JOBS: Array of jobs to complete, each containing a command and arguments (credentials / file-server URL / models) to be passed to a Docker container) experiment.json file (input parameters) Location of input/output files Occopus Cloud orchestration PolicyKeeper scaling VirtualMachine 1 Repast Job JQueuerAgent VirtualMachine 2 VirtualMachine N … Prometheus metrics Docker Swarm Container orchestration Jobs to add
- 16. www.project-cola.eu An application example Deadline-based scaling – performance results 200 REPAST infection model runs Max. VMs: 10 Deadline: 1:04:55 Average number of VMs used by MiCADO: 3.86 Compared to fix 5 VM execution
- 17. Open beta testing Open beta testing is in progress to collect feedback and improve the software • Details are at https://www.micado-scale.eu/ • Contact: • Tamas Kiss, COLA Project Director - kisst@wmin.ac.uk • Andreas Ocklenburg, CEO CloudSME UG - andreas@cloudsme.eu MiCADO is Open Source – supported and marketed by CloudSME UG www.project-cola.eu
- 18. Project Director: Prof. Tamas Kiss, University of Westminster, UK The COLA Project – Cloud Orchestration at the Level of Application (COLA) - receives funding from the European Union´s Horizon 2020 research and innovation programme under grant agreement No 731574 Any questions? www.project-cola.eu
Notas del editor
- Typical industry and public sector applications require resource scalability and efficient resource utilization in order to serve a variable number of customers with dynamic resource demands, and to suitably optimize resource consumption and costs. To overcome the problem of manual allocation of resources for applications, the MiCADO framework is developed as part of the COLA project. The overall objective of the COLA project is to define and provide a reference implementation of a generic and pluggable framework that supports the optimal and secure deployment and run-time orchestration of cloud applications.
- Project COLA: an Innovation Action funded by the European Commission as part of … Multi-cloud support : which are tested? TOSCA is a specification that aims to standardize how we describe software applications and everything that is required for them to run in the “cloud”, provides a way to describe not only an application, but also its dependencies and supporting (cloud) infrastructure. Modular architecture: proved Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration Dashboard is used to track scaling/ cpu consumption
- 3 working groups: architecture, ADT, security Generic and pluggable framework that supports the optimal and secure deployment and run-time orchestration of cloud applications. MiCADO Master node: receive application description and control scaling MiCADO worker node: run application On Master node: Submitter: collect application description and dispatch information to other components Cloud Orchestrator: scale up/ down VM Container orchestrator: scale up/ down containers Policy keeper: based on received scaling policies from Submitter and monitoring information from worker nodes to command Cloud Orchestrator/ Container Orchestrator do scaling Monitoring system: collects monitoring info from worker nodes, involving info about VM and containers On worker node: Node/ container monitor: collect runtime info about VM/ container Container executor: hosts application’s service
- Dockerized simulation application
- Spanish technology company established in 1982. SARGA is an Aragón Government public enterprise Memory/ network load based scaling
- Zorp Flask-User Hashicorp vault SGX
- Zorp Flask-User Hashicorp vault SGX
- Familiar with linux