Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

UCL Ph.D. Confirmation 2018

88 visualizaciones

Publicado el

My presentation for the UCLouvain Ph.D. Confirmation
https://kkpradeeban.blogspot.com/2018/01/ucl-phd-confirmation.html

Publicado en: Educación
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

UCL Ph.D. Confirmation 2018

  1. 1. A Software-Defined Service-Oriented Architecture for Distributed Workflows Pradeeban Kathiravelu Supervisors: Prof. Luís Veiga Prof. Peter Van Roy Prof. Marco Canini LightKone M12 General Meeting, UNINOVA, Caparica, Portugal. 16/01/2018.
  2. 2. Agenda ● Overview and Motivation ● Goal of the Thesis ● Current Work – Composing Network Service Chains at the Edge ● Ongoing and Future Work
  3. 3. 3/39 Overview and Motivation I
  4. 4. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 4/39 Services ● A core element of the Internet ecosystem. ● Various types of Services – Web services and microservices ● key in modern cloud applications. – Network services / Virtual Network Functions ● firewall, load balancer, proxy, .. – Data services ● data cleaning, data integration, .. ● Interesting common research challenges: – Service placement. – Service instance selection. – Service composition or “service chaining”.
  5. 5. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 5/39 Why Service-Oriented Architectures for our systems? ● Beyond data center scale. – Thanks to the fact that services are standardized. ● SOA and RESTful reference architectures. – Multiple implementation approaches such as Message- Oriented Middleware. ● Service endpoints to handover messages internally to the broker. ● Publish/subscribe to a message broker over the Internet. ● Flexibility, modularity, loose-coupling, and adaptability.
  6. 6. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 6/39 Software-Defined Networking (SDN) ● Enables global view of the data center network on a single controller. ● Separation of control-plane and data-plane ● Improved configurability – Bring the control of the network to the software developer! ● Key technology enabling separation of mechanisms from policies.
  7. 7. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 7/39 SDN for Cross-layer Optimizations (Program for: application ↔ network)
  8. 8. 8/39 Goal of the Thesis II
  9. 9. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 9/39 Our Vision ● Optimal placement and migration of service chains beyond data centers. – Minimal latency and efficient bandwidth usage. – Quality of Experience, adaptability, and resilience. ● Bring the control back to the service user. – As it was in the pre-cloud, pre-multi-tenancy era. – Focus on users consuming several third-party services. ● Services typically deployed in distributed clouds and edge. ● An approach inspired by Software-Defined Networking (SDN).
  10. 10. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 10/39 Our Proposal ● An approach inspired by SDN for an adaptive placement and resilient execution of service chains. – Software-Defined Service Composition. – Providing global awareness through a combined approach ● at application-level (e.g. web service engines and service registries) ● at network-level (SDN controller). ● Challenge: – We have a much larger scale and problem complexity. ● Compared to the classic SDN.
  11. 11. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 11/39 Thesis Contributions (1) ● Web services – An adaptive and resilient architecture for web service compositions and workflow management in the wide area network, extending SDN. (ICWS’16(ICWS’16 and a book chapter).and a book chapter). – Reprogram easily for service failures or congestions. ● Network Services – Model and execute network services through a unified orchestrator ● Deploy execution and simulation units through a coherent model. ● (CoopIS’16, SDS’15, and IC2E’16 (short)). – Resilience in multi-tenant environments. ● NCA’16, IM’17 (short), and EI2N’16.
  12. 12. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 12/39 Thesis Contributions (2) ● SDN-inspired Approach for scalable user-driven service compositions – Chaining data services into big data processing workflow ((CoopIS’15CoopIS’15,, DMAH’16, DMAH’17, and a book chapter).). – Microservice compositions at the edge clouds to enable smart environments and Cyber-Physical Systems ((SDS’16SDS’16, M4IoT’15, and, M4IoT’15, and SDS’17SDS’17).).
  13. 13. 13/39 Composing Network Service Chains at the Edge: A Resilient and Adaptive Software-Defined Approach III
  14. 14. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 14/39 Introduction ● A network flow goes through various services. ● Increasingly network services placed at the edge. – Limitations in hosting all the network services on-premise. – Closer to the user than centralized clouds. – Various shades of the edge. ● Heavy vs light edge, edge clouds, community clouds, ..
  15. 15. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 15/39 Network Service Chaining (NSC) ● Chaining a number of connected network services. ● Dynamically composable.
  16. 16. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 16/39 Challenges in achieving Service Chaining at the Edge ● Dependencies among the network services. – Need to be accessible from each other. ● Service Level Objectives of the service chain users. – Latency, throughput, monthly cost, .. ● Finding the optimal service chain for a user request. – In general, an NP-hard problem.
  17. 17. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 17/39 Service Chain: s1 → s2 → s3 → s4 ● Goals – Services close to the user. – Services close to the following services in the chain. – Satisfying user Service Level Objectives!
  18. 18. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 18/39 Our Contribution: Évora ● A graph-based algorithm to incrementally construct and deploy service chains at the edge. ● An Orchestrator in the user device or a surrogate, to place and migrate service chains, adhering to the user policies. ● An architecture extending SDN to wide area to efficiently support the service chains at the edge.
  19. 19. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 19/39 Évora Approach ● Initialize once per user device: – Step 1) Construct a service graph. ● Initialize once per a user’s service chain. – Step 2) Find matching subgraphs for the user’s service chain as partial, potential chains. – Step 3) Complete matches → Potential Chains. – Step 4) Service chain placement at the best fit among the possible chains. ● Execute following the service chain.
  20. 20. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 20/39 1) Data center graph → service graph ● Construct a service graph in the user device. ● As a snapshot of the available services at the edge.
  21. 21. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 21/39 2.1) Matching subgraphs in the service graph for the service chain. ● Matching subgraphs → partial chains. ● Consider alternative representations.
  22. 22. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 22/39 2.2) Find Potential Chains, in Parallel. ● Construct matching subgraphs as potential chains. – while noting the individual service properties ● monthly cost, throughput, latency, .. ● Incrementally calculate a “penalty value” for each potential chain that is being constructed.
  23. 23. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 23/39 2.3) Record the complete matches with their penalty value ● Évora defines a penalty value based on – Total monthly cost (C) for the chain, – End-to-end latency (L) for the chain, – The inverse of throughput (T-1) (defined by the minimal throughput service in the chain). – Can be extended for additional properties such as uptime. ● Évora aims to minimize the penalty value. – With user giving weight to each of the properties.
  24. 24. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 24/39 Objective: Minimize the penalty value ● Penalty function, with normalized values of C, L, and T. ● Solve this as a Mixed Integer Linear Problem. ● α,β,γ ← Non-negative integers specified by user. ● The penalty function can be extended with powers.
  25. 25. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 25/39 3) Complete matches → Potential Service Chain Placements ● Ability to place the entire service chain in the matching subgraph. – Complete matching subgraph, i.e. a potential service chain placement is found. ● Record. ● Stop procedure once all the nodes are traversed.
  26. 26. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 26/39 4) The service chain placement ● Service chain is placed on the service composition with the minimal penalty value among the alternatives (matching subgraphs). – Possible updates and migrations. – Future service unavailability → choose the next.
  27. 27. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 27/39 Solution Architecture: A federated controller deployment for the edge ● Extending the control offered by SDN Controllers – from data centers to a multi-domain environment. – With Message-Oriented Middleware.
  28. 28. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 28/39 Notifications for Service Availability ● Service chain placements and migrations.
  29. 29. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 29/39 Evaluation ● Microbenchmark how user policies are satisfied with Évora for service chains among various alternatives. – Complexity of the problem space. – Algorithm effectiviness in satisfying user policies. ● Efficacy: Closeness to optimal results – minimizing penalty function results in improved quality of experience ● Efficiency: execution times depending on problem space
  30. 30. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 30/39 Problem Scale: Representation of the service graph from the data center graph ● The number of links in this service graph grows – linearly with the number of edges or links between the edge data centers – exponentially with the average number of services per edge data center.
  31. 31. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 31/39 User policies with two attributes ● Location of the circles → Properties (C, L, and T). ● Darker circles – chains with minimal penalty, the ones that we prefer (circled). T ↑ and C ↓ T ↑ and L ↓ C ↓ and L ↓ ● Results show user policies supported fairly well.
  32. 32. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 32/39 ● Policies with three attributes: One given more prominence (weight = 10), than the other two (weight = 3). ● Results show efficient support for multiple attributes with different weights. Radius of the circles – Monthly Cost
  33. 33. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 33/39 ● Two attributes given more prominence (weight = 10), than the third (weight = 3). ● Results show efficient support for multiple attributes with different weights. Radius of the circles – Monthly Cost
  34. 34. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 34/39 Conclusion ● More and more services hosted at the edge. ● Service chains have more constraints than stand-alone services. ● Évora supports efficient chaining of network service. – Leveraging a software-defined approach for services ● Extending SDN. ● Future Work: – Evaluate the performance with network service deployments at the edge with actual workload.
  35. 35. 35/39 Ongoing and Future Work IV
  36. 36. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 36/39 1) Software-Defined CPS workflows in the light-weight edge ● Can we tackle some operational and scale challenges of Cyber-Physical Systems? – By representing them as composable service chains at the edge? – Target: CLUSTER Journal. ● Invitation for selected papers from SDS’17. ● Deadline: Jan 31st.
  37. 37. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 37/39 2) A Service-Oriented Workflow for Big Data Research at the Edge ● Analyse decentralized big data (TB-scale) with a service based data access and virtual integration approach. – Addressing data related optimizations as service chains. ● Data cleaning, incremental data integration, and data analysis. – Target: Distributed and Parallel Databases Journal. ● Invitation for selected papers from DMAH’17. ● Deadline: March 31st.
  38. 38. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 38/39 3) Architecture to Realize the Network Services at the Edge ● Can the migrations between the third-party service providers be seamless? – An overlay network for anyone to join and offer services? – Architectural alternatives such as Blockchain solutions. – Target: CoNEXT (Tentative, June).
  39. 39. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 39/39 The road ahead (2018).. ● On-going journal and conferenec submissions. – January – June 2018. ● CAT, February 2018. ● Thesis Defence, December 2018 or after. Thank you! pradeeban.kathiravelu@tecnico.ulisboa.pt
  40. 40. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 40/39 ~ Thanks ~
  41. 41. Additional Slides
  42. 42. What has Message-Oriented Middleware got to do with the controller? ● Expose the internals from controller (e.g. OpenDaylight) – Through a message (e.g. AMQP as northbound) API – Publish/Subscribe with a broker (e.g. ActiveMQ). ● What can be exposed – Data tree (internal data structures of the controller) – Remote procedure calls (RPCs) – Notifications. ● Thanks to Model-Driven Service Abstraction Layer (MD-SAL) of OpenDaylight. – Compatible internal representation of data plane.
  43. 43. MILP and Graph Matching can be computation intensive ● But initialization is once per user service chain with a given policy. – This procedure does not repeat once initialized. – unless updates received from the edge network. ● New data center with the service offering at the edge. ● An existing data center or a service offering fails to respond.
  44. 44. Algorithm
  45. 45. Algorithm
  46. 46. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 46/39 User policies with two attributes ● Location of the circles → Properties (C, L, and T). ● Darker circles – chains with minimal penalty, the ones that we prefer. T ↑ and C ↓ T ↑ and L ↓ C ↓ and L ↓ ● Results show user policies supported fairly well.
  47. 47. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 47/39 ● Policies with three attributes: One given more prominence (weight = 10), than the other two (weight = 3). Radius of the circles – Monthly Cost ● Results show efficient support for multiple attributes with different weights.
  48. 48. Composing Network Service Chains at the EdgeComposing Network Service Chains at the Edge 48/39 ● Two attributes given more prominence (weight = 10), than the third (weight = 3). Radius of the circles – Monthly Cost ● Results show efficient support for multiple attributes with different weights.

×