Marvin Platform – Potencializando equipes de Machine Learning

•

2 recomendaciones•902 vistas

Marvin é um ambicioso projeto de código aberto que se concentra em ajudar equipes a entregar soluções de machine learning de maneira ágil. A plataforma oferece uma arquitetura padronizada e agnóstica de linguagem, de alta escala e baixa latência enquanto simplifica o processo de exploração e modelagem de projetos de IA.

Datos y análisis

Daniel Takabayashi
Lucas Bonatto
From exploratory models to production
BIG DATA WEEK
SÃO PAULO 2017

Daniel Takabayashi
daniel.takabayashi@b2wdigital.com@
linkedin.com/in/danieltakabayashi
github.com/takabayashi
IT Manager @ B2W Digital

Lucas Bonatto
lucas.bonatto@b2wdigital.com@
linkedin.com/in/lucasbonatto
twitter.com/lucasbonatto
github.com/lucasbm88
IT Specialist @ B2W Digital

B2W Digital: e-commerce leader in LatAm
Source: 2016 Results from ri.b2w.digital
Total GMV (R$)
12,458 MM
Market share (%)
26,2%
The Digital Platform that
connects People,
Businesses,Products
and Services.

Outline
• Context
• Data-driven culture
• Artificial Intelligence
• Domains of knowledge
• Problem Statement
• Marvin
• Main components
• Architecture
• DASFE pattern
• General features
• Case
• Roadmap

Context: data-driven culture
Single source of truth
Data dictionary
Broad data access
Data literacy
Decision making
Why is it important to
be data-driven?

Context: data-driven culture
Optimize decision-making

Context: Artificial Intelligence
Machine Learning NLP Computer Vision
• Buy Box
optimization
• Forecast demand
• Fraud detection
• Adspend
optimization
• Feature extraction
from product
description
• Product category
classification
• Image matching to
find associated
products

Context: model lifecycle
Models Management
Support & Feedback
Model ServingExploration & Development

Problem statement
How can we abstract the complexity in
the creation of an AI application?
Building AI projects is not a simple task.
One is required to have advanced
knowledge in different domains.

GitHub.com/marvin-ai
Artificial Intelligence Platform

Marvin Artificial Intelligence Platform
Empowers data science teams to deliver
AI applications, simplifying the process
of exploitation and modeling.

Marvin: main components
ENGINE EXECUTOR
ENGINE
Data acquisitor
Prediction preparator
Predictor
Trainingpreparator
Feedback
Trainer
TOOLBOX
Evaluator

Marvin: DASFE pattern
Batch Data Acquisition
& Cleaning
Training
Preparation
Model
Training
Model
Evaluation

Marvin: DASFE pattern
Online Prediction
Preparation
Model
Prediction
Batch Data Acquisition
& Cleaning
Training
Preparation
Model
Training
Model
Evaluation

Marvin: DASFE pattern
Online Prediction
Feedback
Online Prediction
Preparation
Model
Prediction
Batch Data Acquisition
& Cleaning
Training
Preparation
Model
Training
Model
Evaluation

Marvin: general features
• Training pipeline REST interface
• Experiment and artifacts versioning
• Engine project scaffold generator
• Data sampling and import CLI
• Engine test framework (unit, functional, dryrun)
• Toolbox: Python support
• Artifacts persistence layer: HDFS support
• Remote provisioning and deployment

Case: risk analysis model
• XGBoost in python
• Dataset: 1,0 M of orders
• Training pipeline: 15 min
• REST HTTP predictions: 15 ms
• Load test: 100 rps w/ 15 ms mrt

Case: how marvin helped the team?
“... Jupyter notebook integration with Spark through Marvin’s
toolbox lib was very helpful during prototyping phase...”
“... the data importation utility speeds up
data collection and sampling... ”
“... it was easy to do feature engineering, feature selection and
model choice using the DASFE model... ”
“... we automated the training and deployment phase without
having a dev/ops in our team...”

Marvin: roadmap
• Admin module
• Toolbox: Java and Scala support
• Feedback server
• Artifacts persistence layer: S3 and local FS support
• Remote provisioning and deployment: Azure, AWS and GCP
• Customized notebook kernel
• Automate feature engineering
• Hyper parameters support
• ML for no-data scientists
• …

Artificial Intelligence Platform
Fork me on GitHub.com/marvin-ai
and feel free to contribute!
Thank you!
@
GitHub.com/marvin-ai
twitter.com/_marvin_ai
marvin-ai@googlegroups.com

Más contenido relacionado

La actualidad más candente

Data & ML projects bring many new complexities beyond the traditional software development lifecycle. Unlike software projects, after they were successfully delivered and deployed, they cannot be abandoned but must be continuously monitored if model performance still satisfies all requirements. We can always get new data with new statistical characteristics that can break our pipelines or influence model performance.

Developing ML-enabled Data Pipelines on Databricks using IDE & CI/CD at Runta...

Databricks

This talk will present R as a programming language suited for solving data analysis and modeling problems, MLflow as an open source project to help organizations manage their machine learning lifecycle and the intersection of both by adding support for R in MLflow. It will be highly interactive and touch on some of the technical implementation choices taken while making R available in MLflow. It will also demonstrate using MLflow tracking, projects, and models directly from R as well as reusing R models in MLflow to interoperate with other programming languages and technologies.

MLflow with R

Databricks

Description Data Science and ML development bring many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. MLflow addresses some of these challenges during an ML model development cycle. Abstract ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure. In this session, we introduce MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size. With a short demo, you see a complete ML model life-cycle example, you will walk away with: MLflow concepts and abstractions for models, experiments, and projects How to get started with MLFlow Using tracking Python APIs during model training Using MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics

MLFlow: Platform for Complete Machine Learning Lifecycle

Databricks

Robust MLOps with Open-Source: ModelDB, Docker, Jenkins, and Prometheus

Manasi Vartak

Use MLflow to manage and deploy Machine Learning model on Spark

Herman Wu

ScottHovey_Resume

Scott Hovey

Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure. For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps). Agenda - Data Quality and why it matters - Challenges and solutions of Data Testing - Challenges and solutions of Model Testing - MLOps pipelines and why they matter - How to expand validation pipelines for Data Quality

MLOps and Data Quality: Deploying Reliable ML Models in Production

Provectus

GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...

Neo4j

Magdalena Stenius: MLOPS Will Change Machine Learning

Lviv Startup Club

While the adoption of machine learning and deep learning techniques continue to grow, many organizations find it difficult to actually deploy these sophisticated models into production. It is common to see data scientists build powerful models, yet these models are not deployed because of the complexity of the technology used or lack of understanding related to the process of pushing these models into production. As part of this talk, I will review several deployment design patterns for both real-time and batch use cases. I’ll show how these models can be deployed as scalable, distributed deployments within the cloud, scaled across hadoop clusters, as APIs, and deployed within streaming analytics pipelines. I will also touch on topics related to security, end-to-end governance, pitfalls, challenges, and useful tools across a variety of platforms. This presentation will involve demos and sample code for the the deployment design patterns.

AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)

dtz001

This talk describes migrating a large random forest classifier from scikit-learn to Spark's MLlib. We cut training time from 2 days to 2 hours, reduced failed runs, and track experiments better with MLflow. Kount provides certainty in digital interactions like online credit card transactions. One of our scores uses a random forest classifier with 250 trees and 100,000 nodes per tree. We used scikit-learn to train using 60 million samples that each contained over 150 features. The in-memory requirements exceeded 750 GB, took 2 days, and were not robust to disruption in our database or training execution. To migrate workflow to Spark, we built a 6-node cluster with HDFS. This provides 1.35 TB of RAM and 484 cores. Using MLlib and parallelization, the training time for our random forests are now less than 2 hours. Training data stays in our production environment, which used to require a deploy cycle to move locally-developed code onto our training server. The new implementation uses Jupyter notebooks for remote development with server-side execution. MLflow tracks all input parameters, code, and git revision number, while the performance and model itself are retained as experiment artifacts. The new workflow is robust to service disruption. Our training pipeline begins by pulling from a Vertica database. Originally, this single connection took over 8 hours to complete with any problem causing a restart. Using sqoop and multiple connections, we pull the data in 45 minutes. The old technique used volatile storage and required the data for each experiment. Now, we pull the data from Vertica one time and then reload much faster from HDFS. While a significant undertaking, moving to the Spark ecosystem converted an ad hoc and hands-on training process into a fully repeatable pipeline that meets regulatory and business goals for traceability and speed. Speaker: Josh Johnston

Moving a Fraud-Fighting Random Forest from scikit-learn to Spark with MLlib, ...

Databricks

How to Empower a Platform With a Data Pipeline At a Scale

Deepak Sood

GraphDay Paris - CAST IMAGING - Un IRM pour les systèmes IT complexes

Neo4j

What's Next for MLflow in 2019

Anyscale

Enjoy the webinar recording here: https://youtu.be/Lll1qwQJKVw. Driverless AI speeds up data science workflows by automating feature engineering, model tuning, ensembling, and model deployment. In this presentation, Arno Candel (CTO, H2O.ai), gives a quick overview and guide attendees through an interactive hands-on lab using Qwiklabs. Driverless AI turns Kaggle-winning recipes into production-ready code and is specifically designed to avoid common mistakes such as under or overfitting, data leakage or improper model validation. Avoiding these pitfalls alone can save weeks or more for each model, and is necessary to achieve high modeling accuracy. With Driverless AI, everyone can now train and deploy modeling pipelines with just a few clicks from the GUI. Advanced users can use the client/server API through a variety of languages such as Python, Java, C++, go, C# and many more. To speed up training, Driverless AI uses highly optimized C++/CUDA algorithms to take full advantage of the latest compute hardware. For example, Driverless AI runs orders of magnitudes faster on the latest Nvidia GPU supercomputers on Intel and IBM platforms, both in the cloud or on-premise. There are two more product innovations in Driverless AI: statistically rigorous automatic data visualization and interactive model interpretation with reason codes and explanations in plain English. Both help data scientists and analysts to quickly validate the data and models.

Driverless AI - Intro + Interactive Hands-on Lab

Sri Ambati

Starting with outlining the history of conventional version control before diving into explaining QoDs (Quantitative Oriented Developers) and the unique problems their ML systems pose from an operations perspective (MLOps). With the only status quo solutions being proprietary in-house pipelines (exclusive to Uber, Google, Facebook) and manual tracking/fragile "glue" code for everyone else. Datmo works to solve this issue by empowering QoDs in two ways: making MLOps manageable and simple (rather than completely abstracted away) as well as reducing the amount of glue code so to ensure more robust pipelines.

Version Control in AI/Machine Learning by Datmo

Nicholas Walsh

Continuous Delivery of ML-Enabled Pipelines on Databricks using MLflow

Databricks

ML-Ops:Philosophy, Best-Practices and Tools

Jorge Davila-Chacon

Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN

Josh Patterson

Sirius is the easiest framework when it comes to creating graphical designers in the EMF ecosystem. With Sirius Web, we present the next generation. Rewritten from the ground up and with first-class support for Web applications, Sirius Web, as the successor of Sirius Desktop, also brings new features and support for existing designer specifications. What’s in there? How does it differ from the classic Sirius Desktop? The answers to these questions and others will be given in this talk, together with a dive into example applications. This live session will show you how to use Sirius Web to be super productive when working on new tooling. We’ll demonstrate all the additional features to create, update, and iterate on your designer, including super fast dynamic iteration across your domain definitions, easy creation of your graphical representations, and more. Tune in to this session to catch up with the latest development around Sirius Web. Stéphane Bégaudeau, Obeo

Supercharge Your Sirius Web Apps!

Obeo

La actualidad más candente (20)

Developing ML-enabled Data Pipelines on Databricks using IDE & CI/CD at Runta...

MLflow with R

MLFlow: Platform for Complete Machine Learning Lifecycle

Robust MLOps with Open-Source: ModelDB, Docker, Jenkins, and Prometheus

Use MLflow to manage and deploy Machine Learning model on Spark

ScottHovey_Resume

MLOps and Data Quality: Deploying Reliable ML Models in Production

GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...

Magdalena Stenius: MLOPS Will Change Machine Learning

AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)

Moving a Fraud-Fighting Random Forest from scikit-learn to Spark with MLlib, ...

How to Empower a Platform With a Data Pipeline At a Scale

GraphDay Paris - CAST IMAGING - Un IRM pour les systèmes IT complexes

What's Next for MLflow in 2019

Driverless AI - Intro + Interactive Hands-on Lab

Version Control in AI/Machine Learning by Datmo

Continuous Delivery of ML-Enabled Pipelines on Databricks using MLflow

ML-Ops:Philosophy, Best-Practices and Tools

Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN

Supercharge Your Sirius Web Apps!

Similar a Marvin Platform – Potencializando equipes de Machine Learning

Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j

Neo4j

Neo4j GraphTalk Oslo - Building Intelligent Solutions with Graphs

Neo4j

GraphTalk Wien - Intelligente Lösungen mit Graphen erstellen

Neo4j

Big Data for Data Scientists - Info Session

WeCloudData

Neo4j GraphTalk Düsseldorf - Building intelligent solutions with Graphs

Neo4j

Introducing Neo4j

Neo4j

The Briefing Room with Radiant Advisors and IBM Live Webcast on February 25, 2014 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=53c9b7fa2000f98f5b236747e3602511 The power of Big Data depends heavily upon the context in which it's used, and most organizations are just beginning to figure out where, how and when to leverage it. One key to success is integration with existing information systems, many of which still rely on relational database technologies. Finding ways to blend these two worlds can help companies generate measurable business value in fairly short order. Register for this episode of The Briefing Room to hear Analysts Lindy Ryan and John O'Brien as they explain how the combination of traditional Business Intelligence with Big Data Analytics can provide game-changing results in today's information economy. They'll be briefed by Eric Poulin and Paul Flach of Stream Integration who will share best practices for designing and implementing Big Data solutions. They'll discuss the components of IBM BigInsights, and explain how BigSheets can empower non-technical users who need to explore self-structured data. Visit InsideAnlaysis.com for more information.

Big Data in Action – Real-World Solution Showcase

Inside Analysis

Continuum Analytics and Python

Travis Oliphant

Data Con LA 2020 Description Coming from a grand belief of data democratization, I believe that in order for any team to be successful collaborators, it has to be data centric and data should be accessible to all. *To ensure that your non software or software engineering centric team has maximum efficiency, data should be visible, data lake should be accessible. *Form a database for analytics summaries, talk about the different technologies(SQL, NoSQL) cost of deployment, need, team driven structure. Build an API for this database for external/inter team crosstalk. *Build analytics and visual layer on top of it. Flask/Django/Node, etc.., to enable the team to have high visibility in their analysis, and to ensure a higher turnaround of data. *Talk about an easy way of enabling the team to run code, could be local/cloud, JupyterHub is a great way of doing so, talk about the tremendous value added in that and the potential it enables *Talk about the common tools user for version control/CICD/Coding technologies, etc.. *Finally summarize the value of the mixture of all these tools and technologies in order to ensure the maximum efficiency. Speaker Nawar Khabbaz, Rivian, Data Engineer

Enabling Data centric Teams

Data Con LA

Watch here: https://bit.ly/3i2iJbu You will often hear that "data is the new gold". In this context, data management is one of the areas that has received more attention by the software community in recent years. From Artificial Intelligence and Machine Learning to new ways to store and process data, the landscape for data management is in constant evolution. From the privileged perspective of an enterprise middleware platform, we at Denodo have the advantage of seeing many of these changes happen. Join us for an exciting session that will cover: - The most interesting trends in data management. - Our predictions on how those trends will change the data management world. - How these trends are shaping the future of data virtualization and our own software.

What is the future of data strategy?

Denodo

More and more employees (not just data analysts or C-level execs) are expected to make data-driven decisions, yet only 20–25% of workers have access to business intelligence (BI) products. And when asked, end users don’t want to use a “BI tool”—another interface to learn, another login—they want easily accessible answers. Instead of offering a standalone dashboard, the new trend is to embed analytics into applications that are already used every day. This workshop series features a brand new demo application—created by TIBCO Jaspersoft and projekt202—that illustrates and teaches you how to create answer-generating applications. Over four webinars, we will show you to the what and why of data-as-a-feature applications and how you can build your own. Data-as-a-Feature webinar sessions: - Build answer-generating apps that users love: Learn best practices for making data valuable to your application users - Getting started: How to set up your data-as-a-feature project - Your data in your app: Best practices for embedding interactive reports and visualizations in your app - Managing the experience + ad hoc reporting: Handling security, multi-tenancy, and self-service reporting for your data-as-a-feature app

Build Answer-generating Apps that Users Love: Development best practices for ...

TIBCO Jaspersoft

Neo4j GraphDay Tel Aviv - Graphs in Action

Neo4j

So, you finally have a data ecosystem with Kafka and Hadoop both deployed and operating correctly at scale. Congratulations. Are you done? Far from it. As the birthplace of Kafka and an early adopter of Hadoop, LinkedIn has 13 years of combined experience using Kafka and Hadoop at scale to run a data-driven company. Both Kafka and Hadoop are flexible, scalable infrastructure pieces, but using these technologies without a clear idea of what the higher-level data ecosystem should be is perilous. Shirshanka Das and Yael Garten share best practices around data models and formats, choosing the right level of granularity of Kafka topics and Hadoop tables, and moving data efficiently and correctly between Kafka and Hadoop and explore a data abstraction layer, Dali, that can help you to process data seamlessly across Kafka and Hadoop. Beyond pure technology, Shirshanka and Yael outline the three components of a great data culture and ecosystem and explain how to create maintainable data contracts between data producers and data consumers (like data scientists and data analysts) and how to standardize data effectively in a growing organization to enable (and not slow down) innovation and agility. They then look to the future, envisioning a world where you can successfully deploy a data abstraction of views on Hadoop data, like a data API as a protective and enabling shield. Along the way, Shirshanka and Yael discuss observations on how to enable teams to be good data citizens in producing, consuming, and owning datasets and offer an overview of LinkedIn’s governance model: the tools, process and teams that ensure that its data ecosystem can handle change and sustain #datasciencehappiness.

Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...

Shirshanka Das

2017 StrataHadoop SJC conference talk. https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/56047 Description: So, you finally have a data ecosystem with Kafka and Hadoop both deployed and operating correctly at scale. Congratulations. Are you done? Far from it. As the birthplace of Kafka and an early adopter of Hadoop, LinkedIn has 13 years of combined experience using Kafka and Hadoop at scale to run a data-driven company. Both Kafka and Hadoop are flexible, scalable infrastructure pieces, but using these technologies without a clear idea of what the higher-level data ecosystem should be is perilous. Shirshanka Das and Yael Garten share best practices around data models and formats, choosing the right level of granularity of Kafka topics and Hadoop tables, and moving data efficiently and correctly between Kafka and Hadoop and explore a data abstraction layer, Dali, that can help you to process data seamlessly across Kafka and Hadoop. Beyond pure technology, Shirshanka and Yael outline the three components of a great data culture and ecosystem and explain how to create maintainable data contracts between data producers and data consumers (like data scientists and data analysts) and how to standardize data effectively in a growing organization to enable (and not slow down) innovation and agility. They then look to the future, envisioning a world where you can successfully deploy a data abstraction of views on Hadoop data, like a data API as a protective and enabling shield. Along the way, Shirshanka and Yael discuss observations on how to enable teams to be good data citizens in producing, consuming, and owning datasets and offer an overview of LinkedIn’s governance model: the tools, process and teams that ensure that its data ecosystem can handle change and sustain #DataScienceHappiness.

Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...

Yael Garten

Neo4j GraphTalks - Einführung in Graphdatenbanken

Neo4j

Neo4j GraphTalk Basel - Building intelligent Software with Graphs

Neo4j

https://bit.ly/2Aa0I9P Im Zeitalter von Big Data, künstlicher Intelligenz und Cloud Computing nimmt die Menge und Heterogenität der Daten rasant zu. Unternehmen brauchen klare Strategien, um ihre digitalen Informationen effektiv zu nutzen und die datenbasierte Wertschöpfung voranzutreiben. In diesem Webinar stellen wir Ihnen zukunftsweisende Technologietrends vor, die die Datenstrategien von Unternehmen global prägen: - Wie richten große Organisationen ihre Architekturen zur konformen Daten-Bereitstellung strategisch aus? Welche Rolle spielt hierbei das Konzept der „Data Fabric“? - Wie lässt sich eine konsistente Daten-Integration in Echtzeit innerhalb einer heterogenen Landschaft aus on-prem und diversen Clouds sicherstellen? - Wie können Unternehmen eine Self-Service Infrastruktur aufbauen, um ihre Datenbestände zugänglich zu machen und zu monetarisieren? - Welche Rolle kann sprachgesteuerte AI in der Zukunft der Datenanalyse spielen?

Datenstrategie der Zukunft - Technologietrends, die Sie kennen müssen

Denodo

GraphTalks Hamburg - Einführung in Graphdatenbanken

Neo4j

Debu Sinha, Sr Specialist Solutions Architect - AI/ML at Databricks AI/ ML/ Data Science 1. What are feature stores. 2. Why are they important? 3. Using Databricks and the feature store offering to streamline ml. This hold true for small companies as well. How we frame our approach to AI initiatives will determine its success. Don't worry, I am not a zealot. I will not tell you AI and ML are the cure-all and will solve all your problems. Some tasks are particularly well suited to these techniques, but not all. What I love about them is the fact that they allow us to tackle difficult problems that might otherwise be too daunting.

Data Con LA 2022 - Pre- Recorded - Simplifying AI/ML using Databricks feature...

Data Con LA

Meetup Toulouse Microsoft Azure : Bâtir une solution IoT

Alex Danvy

Similar a Marvin Platform – Potencializando equipes de Machine Learning (20)

Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j

Neo4j GraphTalk Oslo - Building Intelligent Solutions with Graphs

GraphTalk Wien - Intelligente Lösungen mit Graphen erstellen

Big Data for Data Scientists - Info Session

Neo4j GraphTalk Düsseldorf - Building intelligent solutions with Graphs

Introducing Neo4j

Big Data in Action – Real-World Solution Showcase

Continuum Analytics and Python

Enabling Data centric Teams

What is the future of data strategy?

Build Answer-generating Apps that Users Love: Development best practices for ...

Neo4j GraphDay Tel Aviv - Graphs in Action

Strata 2017 (San Jose): Building a healthy data ecosystem around Kafka and Ha...

Building a healthy data ecosystem around Kafka and Hadoop: Lessons learned at...

Neo4j GraphTalks - Einführung in Graphdatenbanken

Neo4j GraphTalk Basel - Building intelligent Software with Graphs

Datenstrategie der Zukunft - Technologietrends, die Sie kennen müssen

GraphTalks Hamburg - Einführung in Graphdatenbanken

Data Con LA 2022 - Pre- Recorded - Simplifying AI/ML using Databricks feature...

Meetup Toulouse Microsoft Azure : Bâtir une solution IoT

Más de Daniel Takabayashi, MSc

The recently open-sourced platform called Marvin AI is helping B2W Digital data science teams to put some interesting projects into production, such as the dynamic pricing and fraud detection. The platform is helping B2W avoid challenges that are common in machine learning projects, like building a hypothesis prototype and putting it in production, enabling it to handle ever-increasing data sets, and supporting the necessary throughput of the largest e-commerce company in South America. Marvin AI is made up of tools, libraries, an embedded server that exposes microservices to serve and manage models, and a very useful design pattern, DASFE, that simplifies the process of building new machine learning engines. In this 90-minute hands-on tutorial, one of the project architects and main open source contributor will explain all the core features of Marvin.

Marvin AI: An Open Source Platform to Deploy and Manage Machine Learning Models

Daniel Takabayashi, MSc

A plataforma, recentemente aberta, Marvin AI está ajudando equipes de ciência de dados da B2W Digital a colocar alguns projetos interessantes em produção. Como exemplos temos precificação dinâmica e detecção de fraudes. A plataforma auxilia a B2W, evitando desafios que são comuns em projetos de aprendizado de máquina, como construir um protótipo de hipóteses e colocá-lo em produção. Isso permite que a equipe use conjuntos de dados cada vez maiores, e atinja a performance necessária na maior empresa de comércio eletrônico na América do Sul. O Marvin AI é composto por ferramentas, bibliotecas, um servidor embutido que expõe micro-serviços para servir e gerenciar modelos, e um padrão de design altamente útil chamado DASFE, que simplifica o processo de construção de novos mecanismos de aprendizagem de máquina. Nessa palestra, serão apresentadas as principais features da plataforma open source Marvin AI, além disso, serão discutidas todas as decisões arquiteturais implementadas para suportar projetos de machine learning na B2W Digital.

Marvin AI - Uma plataforma de código aberto para implementar e gerenciar mode...

Daniel Takabayashi, MSc

Da Exploração à Produção - Inteligência Artificial com a plataforma Marvin

Daniel Takabayashi, MSc

Proposta de Inovação (1)

Daniel Takabayashi, MSc

Guia de Estudo OCA Java SE 5 - SE6

Daniel Takabayashi, MSc

Apresentação - Defesa Mestrado - v4

Daniel Takabayashi, MSc

TrabalhoDefesa-Completo-vFinal-Impressão

Daniel Takabayashi, MSc

Más de Daniel Takabayashi, MSc (7)

Marvin AI: An Open Source Platform to Deploy and Manage Machine Learning Models

Marvin AI - Uma plataforma de código aberto para implementar e gerenciar mode...

Da Exploração à Produção - Inteligência Artificial com a plataforma Marvin

Proposta de Inovação (1)

Guia de Estudo OCA Java SE 5 - SE6

Apresentação - Defesa Mestrado - v4

TrabalhoDefesa-Completo-vFinal-Impressão

Último

Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...

nirzagarg

Context 1. Housing Agent collected resale prices on HDB apartments in Singapore. Objective 2. To predict resale prices in to advise his potential clients. Strategies 3. Explore & Clean data for analysis. 4. Perform K-Means Clustering, in Orange, to find possible segments in the customer data. 5. Tune the model to improve its performance. 6. Visualise the findings, share conclusions, and give insight-driven recommendations. Author: Anthony mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com

Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange

ThinkInnovation

RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx

ronsairoathenadugay

Discover how TrafficWave Generator can drive targeted, engaging visitors to your online assets and propel your business to new heights. Unlock the power of free, set-and-forget traffic today. Featuring a user-friendly interface, set-and-forget functionality, and the ability to monetize the traffic with your affiliate links or product offers, TrafficWave Generator is a game-changer for online entrepreneurs, bloggers, and e-commerce store owners alike. Backed by a 30-day money-back guarantee and comprehensive bonus offerings, this software has the potential to transform your traffic-generation and content-creation efforts, unlocking new levels of success in the digital marketing landscape.

TrafficWave Generator Will Instantly drive targeted and engaging traffic back...

SOFTTECHHUB

Digital advertising, or paid media, encompasses the strategic deployment of online advertisements to reach target audiences efficiently and effectively. This includes any digital platform that supports advertising to deliver unique messages for any objective. Understanding the mechanics of digital advertising platforms, along with insights into audience behaviors and preferences, allows marketers to optimize their ad spend and achieve significant engagement and conversion rates. This lecture is for Advanced Digital & Social Media Strategy (MGMTX 466.05) at UCLA Extension.

Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...

Valters Lauzums

原版定制【微信:176555708】【圣地亚哥州立大学毕业证（SDSU毕业证书）】【微信:176555708】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。【业务选择办理准则】一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信176555708】文凭即可二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信176555708】即可三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。留信网认证的作用: 1:该专业认证可证明留学生真实身份 2:同时对留学生所学专业登记给予评定 3:国家专业人才认证中心颁发入库证书 4:这个认证书并且可以归档倒地方 5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息 6:个人职称评审加20分 7:个人信誉贷款加10分 8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才留信网服务项目： 1、留学生专业人才库服务（留信分析） 2、国（境）学习人员提供就业推荐信服务 3、留学人员区块链存储服务 → 【关于价格问题（保证一手价格）】我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子我给客户的都是第一手的代理价格，因为我想坦诚对待大家不想跟大家在价格方面浪费时间对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！

怎样办理圣地亚哥州立大学毕业证（SDSU毕业证书）成绩单学校原版复制

vexqp

SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...

Elaine Werffeli

Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now Booking Contact Details :- WhatsApp Chat :- +91-7737669865 We offer all types of girls of your choice with space. Our escorts are fully cooperative and understand your needs. All types of call girls like Housewives, College girls,#K09 Russian girls, Muslim girls, Afghani girls, Bengali girls, Working girls, south Indian girls, Punjabi girls, etc. In-Call: — You Can Reach At Our Place in Bangalore Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call: — Service for Out Call You have To Come Pick The Girl From My Place We Also Provide Door-Step Services Hygienic: — Full Ac Neat And Clean Rooms Available In Hotel 24 * 7 Hrs In Bangalore Our Services and Rates: – One Shot — 2500/in call (time ½ hour), 5000/out call Two shot with one girl — 5000/in call (time 1 hour), 6000/out call Body to body massage with sex- 3000/in call (time 1 hour) full night for one person– 8000/in call, 10000/out call (shot limit 4 shot) We are available 24*7 all days of the year

Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now

gargpaaro

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...

gajnagarg

20240412-SmartCityIndex-2024-Full-Report.pdf

khraisr

Kings of Saudi Arabia, information about them

eitharjee

Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...

nirzagarg

Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort Service Available 24/7 Hire CALL GIRL IN Lucknow 9548273370 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRL IN #j11 We are Providing :- ● – Private independent collage Going girls . ● – independent Models . ● – House Wife’s . ● – Private Independent House Wife’s ● – Corporate M.N.C Working Profiles . ● – Call Center Girls . ● – Live Band Girls . ●- Foreigners & Many More . Service type: 1.In call 2.out call 3. full Lip to Lip kiss 4.69 5.b-job without Condom 6. Hard Core sex & Much More. 7 Body to Body Touch 8 Kissing 9 Sucking Boobs and More 10 Enjoy by Hand 11 Relax By Oral 12 Sex with Happy Ending • In Call and Out Call Service • 3* 5* 7* Hotels Service • 24 Hours Available • Indian, Russian, Punjabi, Kashmiri Escorts • Real Models, College Girls, House Wife, Also Available • Short Time and Full Time Service Available • Hygienic Full AC Neat and Clean Rooms Avail. In Hotel 24 hours • Daily Escorts Staff Available • Minimum to Maximu m Range Available.c

Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...

HyderabadDolls

Aspirational Block Program Block Syaldey District - Almora

GovindSinghDasila

Computer science Sql cheat sheet.pdf.pdf

SayantanBiswas37

Statistics notes ,it includes mean to index numbers

suginr1

Digital Transformation Playbook by Graham Ware

Graham Ware

Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736733 Neha Thakur Our agency presents a selection of young, charming call girls #J11 available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives,#J11 College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Kolkata ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 8005736733 🔝. for details Operating 24/7, we serve various locations in Kolkata, including Green Park, near metro stations. For premium call girl services in Kolkata 🔝 8005736733 🔝. Thank you for considering us!

Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...

HyderabadDolls

Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort Service Available 24/7 Hire Booking Contact Details :- WhatsApp Chat :- +91-6378878445 We offer all types of girls of your choice with space. Our escorts are fully cooperative and understand your needs.#K09 All types of call girls like Housewives, College girls,#K09 Russian girls, Muslim girls, Afghani girls, Bengali girls, Working girls, south Indian girls, Punjabi girls, etc. In-Call: — You Can Reach At Our Place in Bangalore Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call: — Service for Out Call You have To Come Pick The Girl From My Place We Also Provide Door-Step Services Hygienic: — Full Ac Neat And Clean Rooms Available In Hotel 24 * 7 Hrs In Bangalore Our Services and Rates: – One Shot — 2500/in call (time ½ hour), 5000/out call Two shot with one girl — 5000/in call (time 1 hour), 6000/out call Body to body massage with sex- 3000/in call (time 1 hour) full night for one person– 8000/in call, 10000/out call (shot limit 4 shot) We are available 24*7 all days of the year

Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...

gragchanchal546

Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...

gajnagarg

Marvin Platform – Potencializando equipes de Machine Learning

1. Daniel Takabayashi Lucas Bonatto From exploratory models to production BIG DATA WEEK SÃO PAULO 2017

2. Daniel Takabayashi daniel.takabayashi@b2wdigital.com@ linkedin.com/in/danieltakabayashi github.com/takabayashi IT Manager @ B2W Digital

3. Lucas Bonatto lucas.bonatto@b2wdigital.com@ linkedin.com/in/lucasbonatto twitter.com/lucasbonatto github.com/lucasbm88 IT Specialist @ B2W Digital

4. B2W Digital: e-commerce leader in LatAm Source: 2016 Results from ri.b2w.digital Total GMV (R$) 12,458 MM Market share (%) 26,2% The Digital Platform that connects People, Businesses,Products and Services.

5. Outline • Context • Data-driven culture • Artificial Intelligence • Domains of knowledge • Problem Statement • Marvin • Main components • Architecture • DASFE pattern • General features • Case • Roadmap

7. Context: data-driven culture Single source of truth Data dictionary Broad data access Data literacy Decision making Why is it important to be data-driven?

8. Context: data-driven culture Optimize decision-making

9. Context: Artificial Intelligence Machine Learning NLP Computer Vision • Buy Box optimization • Forecast demand • Fraud detection • Adspend optimization • Feature extraction from product description • Product category classification • Image matching to find associated products

10. Context: domains of knowledge

11. Context: model lifecycle Models Management Support & Feedback Model ServingExploration & Development

12.

13. Problem statement How can we abstract the complexity in the creation of an AI application? Building AI projects is not a simple task. One is required to have advanced knowledge in different domains.

14. GitHub.com/marvin-ai Artificial Intelligence Platform

15. Marvin Artificial Intelligence Platform Empowers data science teams to deliver AI applications, simplifying the process of exploitation and modeling.

16. Marvin: main components ENGINE EXECUTOR ENGINE Data acquisitor Prediction preparator Predictor Trainingpreparator Feedback Trainer TOOLBOX Evaluator

17. Marvin: context diagram

18. Marvin: architecture

19. Marvin: DASFE pattern Batch Data Acquisition & Cleaning Training Preparation Model Training Model Evaluation

20. Marvin: DASFE pattern Online Prediction Preparation Model Prediction Batch Data Acquisition & Cleaning Training Preparation Model Training Model Evaluation

21. Marvin: DASFE pattern Online Prediction Feedback Online Prediction Preparation Model Prediction Batch Data Acquisition & Cleaning Training Preparation Model Training Model Evaluation

22. Marvin: walkthrough

23. Marvin: general features • Training pipeline REST interface • Experiment and artifacts versioning • Engine project scaffold generator • Data sampling and import CLI • Engine test framework (unit, functional, dryrun) • Toolbox: Python support • Artifacts persistence layer: HDFS support • Remote provisioning and deployment

24.

25. Case: risk analysis model • XGBoost in python • Dataset: 1,0 M of orders • Training pipeline: 15 min • REST HTTP predictions: 15 ms • Load test: 100 rps w/ 15 ms mrt

26. Case: how marvin helped the team? “... Jupyter notebook integration with Spark through Marvin’s toolbox lib was very helpful during prototyping phase...” “... the data importation utility speeds up data collection and sampling... ” “... it was easy to do feature engineering, feature selection and model choice using the DASFE model... ” “... we automated the training and deployment phase without having a dev/ops in our team...”

27.

28. Marvin: roadmap • Admin module • Toolbox: Java and Scala support • Feedback server • Artifacts persistence layer: S3 and local FS support • Remote provisioning and deployment: Azure, AWS and GCP • Customized notebook kernel • Automate feature engineering • Hyper parameters support • ML for no-data scientists • …

29. Artificial Intelligence Platform Fork me on GitHub.com/marvin-ai and feel free to contribute! Thank you! @ GitHub.com/marvin-ai twitter.com/_marvin_ai marvin-ai@googlegroups.com

Marvin Platform – Potencializando equipes de Machine Learning

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Marvin Platform – Potencializando equipes de Machine Learning

Similar a Marvin Platform – Potencializando equipes de Machine Learning (20)

Más de Daniel Takabayashi, MSc

Más de Daniel Takabayashi, MSc (7)

Último

Último (20)

Marvin Platform – Potencializando equipes de Machine Learning