Big Data Platform at Pinterest

•Download as PPTX, PDF•

129 likes•22,097 views

This document discusses Pinterest's data architecture and use of Pinball for workflow management. Pinterest processes 3 petabytes of data daily from their 60 billion pins and 1 billion boards across a 2000 node Hadoop cluster. They use Kafka, Secor and Singer for ingesting event data. Pinball is used for workflow management to handle their scale of hundreds of workflows, thousands of jobs and 500+ jobs in some workflows. Pinball provides simple abstractions, extensibility, reliability, debuggability and horizontal scalability for workflow execution.

Data & Analytics

Confidentia
l
Mao Ye
Big Data Platform at interest
1

Data Architecture
Design Choices for Hadoop Platform
Pinball for Workflow Management

Data at Pinterest
• 60 Billion Pins
• 1 Billion boards
• 100M MAU
• 60 PB of data on S3
• 3 PB processed every day
• 2000 node Hadoop cluster
• 250 engineers

Pinterest Data Architecture
App
events
Kafka
Secor
Singer

Pinterest Data Architecture
App
events
Kafka
Secor
Skyline
Pinball
Redshift
Pinalytics
Features
Qubole
(Hadoop)
Singer

• Ephemeral clusters
• Access control layer
• Shared data store
• Easy deployment
Hadoop Platform Requirements
• Isolated multi-tenancy
• Elasticity
• Support multiple
clusters

Decoupling compute & storage
Hadoop Cluster 1
Transient
HDFS
Hadoop Cluster 2
Transient
HDFS
S3 Persistent
Store

Centralized Hive Metastore
Hive
Metastore
Pig
Cascading
Hive
HDFS/S3
DataMetadata

Multi-layered Packaging
Mapreduce Jobs
Hadoop Jars/Libs
Job/User level Configs
Software Packages/Libs
Configs (OS/Hadoop)
Misc Sys Admin
OS
Bootstrap Script
Core SW
Runtime Staging
(on S3)
Automated
Configuration
(Masterless Puppet)
Baked AMI

Executor Abstraction Layer
Hive
Metastore
HDFS/S3
Qubole
Managed
Hadoop
EMR
Executor
Pinball
Dev
Server

• API for simplified
executor abstraction
• Advanced support
for spot instances
• Baked AMI
customization
Why Qubole?
• Hadoop & Spark as
managed services
• Tight integration with
Hive
• Graceful cluster
scaling

Confidentia
l
Pinball for Workflow Management

Confidentia
l
● Scale:
o 60 Billion Pins
o Hundreds of workflows
o Thousands of jobs
o 500+ jobs in a workflow
o 3 petabytes processed daily
● Support:
o Hadoop, Cascading, Hive, Spark …
Scale of Processing
job
workflow

Confidentia
l
Why Pinball?
● Requirements
o Simple abstractions
o Extensible in future
o Reliable stateless computing
o Easy to debug
o Scales horizontally
o Can be upgraded w/o aborting workflows
o Rich features like auto-retries, per-job emails, overrun
policies…
● Options
o Apache Oozie, Azkaban, Luigi

Confidentia
l
Pinball Design
Master
Worker
Scheduler
Command
Line Clients
UI

Confidentia
l
● Workflow
o A directed graph of
nodes called jobs
● Edge
o Run after
dependence
● Node
o Job is a node
Workflow Model

Confidentia
l
Job State
● Job state is captured in a token
● Tokens are named hierarchically
Master
Job Token
version: 123
name: /workflow/w1/job
owner: worker_0
expiration: 1234567
data: JobTemplate(....)

Confidentia
l
Job State Machine
RUNNABLE
RUNNINGWAITING

Confidentia
l
● Master keeps the state
● Workers claim and execute tasks
● Horizontally scalable
Master Worker Interaction
Worker Master Persistent Store
1: request 2: update
3: ack

Confidentia
l
Master
● Entire state is kept in memory
● Each state update is synchronously persisted
before master replies to client
● Master runs on a single thread – no
concurrency issues

Confidentia
l
Open Source
Git repo:
https://github.com/pinterest/pinball
Mailing list:
https://groups.google.com/forum/#!forum/
pinball-users

What's hot

Elastic Stack 을 이용한 게임 서비스 통합 로깅 플랫폼 - elastic{on} 2019 Seoul

SeungYong Oh

Data in Hadoop is getting bigger every day, consumers of the data are growing, organizations are now looking at making their Hadoop cluster compliant to federal regulations and commercial demands. Apache Ranger simplifies the management of security policies across all components in Hadoop. Ranger provides granular access controls to data. The deck describes what security tools are available in Hadoop and their purpose then it moves on to discuss in detail Apache Ranger.

Apache Ranger

Rommel Garcia

Elasticsearch in Netflix

Danny Yuan

Kubernetes is a fast growing open-source platform which provides container-centric infrastructure. Conceived by Google in 2014, and leveraging over a decade of experience running containers at scale internally, it is one of the fastest moving projects on GitHub with 1000+ contributors and 40,000+ commits. Kubernetes has first class support on Google Cloud Platform, Amazon Web Services, and Microsoft Azure. Unlike YARN, Kubernetes started as a general purpose orchestration framework with a focus on serving jobs. Support for long-running, data intensive batch workloads required some careful design decisions. Engineers across several organizations have been working on Kubernetes support as a cluster scheduler backend within Spark. During this process, we encountered several challenges in translating Spark considerations into idiomatic Kubernetes constructs. In this talk, we describe the challenges and the ways in which we solved them. This talk will be technical and is aimed at people who are looking to run Spark effectively on their clusters. The talk assumes basic familiarity with cluster orchestration and containers.

Apache Spark on Kubernetes Anirudh Ramanathan and Tim Chen

Databricks

Apache Ranger Hive Metastore Security

DataWorks Summit/Hadoop Summit

Delta lake is widely adopted. There are things to be aware of when dealing with petabytes of data in Delta Lake. These smart decisions can give the best efficiency and increase the adoption of Delta. Best practices like OPTIMIZE, ZORDER have to wisely chosen. We have support stories where we successfully resolved performance issues by applying the right performance strategy. There are a set of common issues or repeated questions from our strategic customers face when using Delta and in this session we cover them and how to address them.

Operating and Supporting Delta Lake in Production

Databricks

Enterprise data architectures usually contain many systems—data lakes, message queues, and data warehouses—that data must pass through before it can be analyzed. Each transfer step between systems adds a delay and a potential source of errors. What if we could remove all these steps? In recent years, cloud storage and new open source systems have enabled a radically new architecture: the lakehouse, an ACID transactional layer over cloud storage that can provide streaming, management features, indexing, and high-performance access similar to a data warehouse. Thousands of organizations including the largest Internet companies are now using lakehouses to replace separate data lake, warehouse and streaming systems and deliver high-quality data faster internally. I’ll discuss the key trends and recent advances in this area based on Delta Lake, the most widely used open source lakehouse platform, which was developed at Databricks.

Making Data Timelier and More Reliable with Lakehouse Technology

Matei Zaharia

Every business today wants to leverage data to drive strategic initiatives with machine learning, data science and analytics — but runs into challenges from siloed teams, proprietary technologies and unreliable data. That’s why enterprises are turning to the lakehouse because it offers a single platform to unify all your data, analytics and AI workloads. Join our How to Build a Lakehouse technical training, where we’ll explore how to use Apache SparkTM, Delta Lake, and other open source technologies to build a better lakehouse. This virtual session will include concepts, architectures and demos. Here’s what you’ll learn in this 2-hour session: How Delta Lake combines the best of data warehouses and data lakes for improved data reliability, performance and security How to use Apache Spark and Delta Lake to perform ETL processing, manage late-arriving data, and repair corrupted data directly on your lakehouse

Free Training: How to Build a Lakehouse

Databricks

Some Iceberg Basics for Beginners (CDP).pdf

Michael Kogan

Databricks Platform.pptx

Alex Ivy

Building Lakehouses on Delta Lake with SQL Analytics Primer

Databricks

Delta Lake is an open-source storage layer that brings reliability to data lakes. Delta Lake offers ACID transactions, scalable metadata handling, and unifies the streaming and batch data processing. It runs on top of your existing data lake and is fully compatible with Apache Spark APIs. In this talk, we will cover: * What data quality problems Delta helps address * How to convert your existing application to Delta Lake * How the Delta Lake transaction protocol works internally * The Delta Lake roadmap for the next few releases * How to get involved!

Making Apache Spark Better with Delta Lake

Databricks

Databricks' founders caused a seismic shift in data analysis community when they created Apache Spark which has become a cornerstone of Big Data processing pipelines and tools in large and small companies all around the world. Now they've built a revolutionary, comprehensive and easy-to-use platform around Apache Spark and their other inventions, such as MLFlow and Koalas frameworks and most importantly the Data Lakehouse: a concept of fusing data warehouse and data lake architectures into a single versatile and fast platform. Technical foundation for Databricks Data Lakehouse is Delta Lake. More than 7000 organizations today rely on Databricks to enable massive-scale data engineering, collaborative data science, full-lifecycle machine learning and business analytics. Come to the talk and see the demo to find out why.

[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic

DataScienceConferenc1

Snowflake Datawarehouse Architecturing

Ishan Bhawantha Hewanayake

Elastic stack Presentation

Amr Alaa Yassen

Presto Summit 2018 - 09 - Netflix Iceberg

kbajda

IgnitionOne, a global marketing technology and services leader, announced a strategic partnership with Amazon Neptune, a fast, reliable, and fully managed graph database service offered by AWS. In this session, we discuss how this partnership further enhances the IgnitionOne Customer Intelligence Platform (CIP) with amplified identity resolution capabilities, allowing for greater cross-device performance and cross-browser audience activations. Learn how the IgnitionOne CIP with Amazon Neptune goes beyond traditional Customer Data Platform capabilities, giving brands deeper insights for better omnichannel engagement.

Using Amazon Neptune to power identity resolution at scale - ADB303 - Atlanta...

Amazon Web Services

Delta: Building Merge on Read

Databricks

Intro to databricks delta lake

Mykola Zerniuk

Apache Iceberg - A Table Format for Hige Analytic Datasets

Alluxio, Inc.

What's hot (20)

Elastic Stack 을 이용한 게임 서비스 통합 로깅 플랫폼 - elastic{on} 2019 Seoul

Apache Ranger

Elasticsearch in Netflix

Apache Spark on Kubernetes Anirudh Ramanathan and Tim Chen

Apache Ranger Hive Metastore Security

Operating and Supporting Delta Lake in Production

Making Data Timelier and More Reliable with Lakehouse Technology

Free Training: How to Build a Lakehouse

Some Iceberg Basics for Beginners (CDP).pdf

Databricks Platform.pptx

Building Lakehouses on Delta Lake with SQL Analytics Primer

Making Apache Spark Better with Delta Lake

[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic

Snowflake Datawarehouse Architecturing

Elastic stack Presentation

Presto Summit 2018 - 09 - Netflix Iceberg

Using Amazon Neptune to power identity resolution at scale - ADB303 - Atlanta...

Delta: Building Merge on Read

Intro to databricks delta lake

Apache Iceberg - A Table Format for Hige Analytic Datasets

Similar to Big Data Platform at Pinterest

50 Billion pins and counting: Using Hadoop to build data driven Products

DataWorks Summit

Pinterest hadoop summit_talk

Krishna Gade

Webinar - DreamObjects/Ceph Case Study

Ceph Community

Over 100 million subscribers from over 190 countries enjoy the Netflix service. This leads to over a trillion events, amounting to 3 PB, flowing through the Keystone infrastructure to help improve customer experience and glean business insights. The self-serve Keystone stream processing service processes these messages in near real-time with at-least once semantics in the cloud. This enables the users to focus on extracting insights, and not worry about building out scalable infrastructure. I’ll share the details about this platform, and our experience building it.

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

Monal Daxini

Openstack India May Meetup

Deepak Garg

Serverless SQL

Torsten Steinbach

Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)

Sascha Wenninger

Michael stack -the state of apache h base

hdhappy001

AWS Big Data Demystified #1: Big data architecture lessons learned . a quick overview of a big data techonoligies, which were selected and disregard in our company The video: https://youtu.be/l5KmaZNQxaU dont forget to subcribe to the youtube channel The website: https://amazon-aws-big-data-demystified.ninja/ The meetup : https://www.meetup.com/AWS-Big-Data-Demystified/ The facebook group : https://www.facebook.com/Amazon-AWS-Big-Data-Demystified-1832900280345700/

AWS Big Data Demystified #1: Big data architecture lessons learned

Omid Vahdaty

Facebook Presto presentation

Cyanny LIANG

AWS (Hadoop) Meetup 30.04.09

Chris Purrington

In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix. The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs. The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis. Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.

Netflix Open Source Meetup Season 4 Episode 2

aspyker

Cloudera Impala - Las Vegas Big Data Meetup Nov 5th 2014

cdmaxime

Data science is fast becoming a complementary approach and process to solve business challenges today. The explosion of frameworks to help data scientists build models bears a testimony to this. However when a model needs to be turned into a production version in very low latency and enterprise grade environments, there are a very few choices with each one having their own strengths and weaknesses. Adding to this is the current disconnect between a data scientists world which is all about modelling and an engineers world which is about SLAs and service guarantees. A framework like Apache Apex can complement each of these roles and provide constructs for both these worlds. This would help enterprises to drastically cut down the cost of model deployment to production environments.

Low Latency Polyglot Model Scoring using Apache Apex

Apache Apex

Sql Start! 2020 - SQL Server Lift & Shift su Azure

Marco Obinu

True streaming is fast becoming a necessity for many business use cases. On the other hand the data set sizes and volumes are also growing exponentially compounding the complexity of data processing pipelines.There exists a need for true low latency streaming coupled with very high throughput data processing. Apache Apex as a low latency and high throughput data processing framework and Apache Kudu as a high throughput store form a nice combination which solves this pattern very efficiently. This session will walk through a use case which involves writing a high throughput stream using Apache Kafka,Apache Apex and Apache Kudu. The session will start with a general overview of Apache Apex and capabilities of Apex that form the foundation for a low latency and high throughput engine with Apache kafka being an example input source of streams. Subsequently we walk through Kudu integration with Apex by walking through various patterns like end to end exactly once, selective column writes and timestamp propagations for out of band data. The session will also cover additional patterns that this integration will cover for enterprise level data processing pipelines. The session will conclude with some metrics for latency and throughput numbers for the use case that is presented. Speaker Ananth Gundabattula, Senior Architect, Commonwealth Bank of Australia

Low latency high throughput streaming using Apache Apex and Apache Kudu

DataWorks Summit

In this session, learn how to quickly supplement your on-premises Hadoop environment with a simple, open, and collaborative cloud architecture that enables you to generate greater value with scaled application of analytics and AI on all your data. You will also learn five critical steps for a successful migration to the Databricks Lakehouse Platform along with the resources available to help you begin to re-skill your data teams.

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

Databricks

YARN: a resource manager for analytic platform

Tsuyoshi OZAWA

Best of re:Invent

Amazon Web Services

Modern MySQL Monitoring and Dashboards.

Mydbops

Similar to Big Data Platform at Pinterest (20)

50 Billion pins and counting: Using Hadoop to build data driven Products

Pinterest hadoop summit_talk

Webinar - DreamObjects/Ceph Case Study

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

Openstack India May Meetup

Serverless SQL

Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)

Michael stack -the state of apache h base

AWS Big Data Demystified #1: Big data architecture lessons learned

Facebook Presto presentation

AWS (Hadoop) Meetup 30.04.09

Netflix Open Source Meetup Season 4 Episode 2

Cloudera Impala - Las Vegas Big Data Meetup Nov 5th 2014

Low Latency Polyglot Model Scoring using Apache Apex

Sql Start! 2020 - SQL Server Lift & Shift su Azure

Low latency high throughput streaming using Apache Apex and Apache Kudu

5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop

YARN: a resource manager for analytic platform

Best of re:Invent

Modern MySQL Monitoring and Dashboards.

More from Qubole

The effective use of big data is the key to gaining a competitive advantage and outperforming the competition. This change demands that companies consume and blend enormous amount of data created from divergent and inherently mismatched sources, which represents a paradigm shift to the traditional data warehouse. Companies need to modernize their data warehouse, augmenting it with a platform that allows storage, processing, exploration and analysis of large and diverse datasets without limiting the ability to deliver the data access, and flexibility responding to the needs of the business. That’s where Oracle Cloud and Qubole work together delivering a new breed of data platform —capable of storing and processing the overwhelming amount of data that on-premises big data deployments cannot handle. Watch this on-demand webinar to understand: - Why deploying big data on-premises is expensive, complex to maintain and limits your ability to scale across new use cases and data sources - How Oracle Bare Metal Cloud's predictable and fast performance compute and network services deliver the foundation of a cost-effective, high-performance big data platform - How Qubole leverages Oracle Bare Metal Cloud to provide a turnkey big data service that optimizes cost, performance, and scale, enabling self-service data exploration. Qubole delivers a cloud-based, turnkey, self-service big data service that removes the complexity and reduces the cost of doing big data. It leverages Oracle Bare Metal Cloud’s next generation of scalable, inexpensive and performant compute, network and storage public cloud infrastructure to provide a solution that accelerates time to market and reduces the risk of your big data initiatives.

Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...

Qubole

7 Big Data Challenges and How to Overcome Them

Qubole

State of Big Data Adoption

Qubole

Big Data at Pinterest - Presented by Qubole

Qubole

5 Factors Impacting Your Big Data Project's Performance

Qubole

Spark on Yarn

Qubole

Atlanta MLConf

Qubole

Running Spark on Cloud

Qubole

Qubole State of the Big Data Industry

Qubole

Atlanta Data Science Meetup | Qubole slides

Qubole

Qubole presentation for the Cleveland Big Data and Hadoop Meetup

Qubole

BIPD Tech Tuesday Presentation - Qubole

Qubole

Harnessing the Hadoop Ecosystem Optimizations in Apache Hive

Qubole

Optimizing Big Data to run in the Public Cloud

Qubole

DataXu sits at the heart of the all-digital world, providing a data platform that manages tens of millions of dollars of digital advertising investments from Global 500 brands. The DataXu data platform evaluates 1.5 million online ad opportunities every second for our customers, allowing them to manage and optimize their marketing investments across all digital channels. DataXu employs a wide range of AWS services: Cloud Front, Cloud Trail, CloudWatch, Data Pipeline, Direct Connect, Dynamo DB, EC2, EMR, Glacier, IAM, Kinesis, RDS, Redshift, Route53, S3, SNS, SQS, and VPC to run various workloads at scale for DataXu data platform. In addition, DataXu also uses Qubole Data Service, QDS, to offer a Unified Analytics Interface tool to DataXu customers. Qubole, a member of APN provides self-managing Big data infrastructure in the Cloud which leverages spot pricing for cost-efficiencies, provides fast performance, and most importantly a streamlined user-interface for ease of use. Attendees will learn how Qubole provided self-managing Hadoop clusters in the AWS Cloud accelerated DataXu’s batch-oriented analysis jobs; and how Qubole integration with Amazon Redshift enabled DataXu to preform low latency and interactive analysis. Further, in the session we'll take a look at how DataXu opened up QDS access to their customers using QDS user interface thereby providing them with a single tool for both batch-oriented and interactive analysis. By using the QDS user interface buyers of the DataXu data service could perform all manner of analysis against the data stored in their AWS S3 bucket. Speakers: Scott Ward Solutions Architect at Amazon Web Services Ashish Dubey Solutions Architect at Qubole Yekesa Kosuru VP Engineering at DataXu

Getting to 1.5M Ads/sec: How DataXu manages Big Data

Expert Big Data Tips

Big dataproposal

Presto in the cloud

Basic Sentiment Analysis using Hive

Qubole

A session from Qubole Best Practice Webinar Series- “Big Data Secrets from the Pros”. Covers how to make Apache Hive queries run faster by a. Better layout of data on HDFS via partitioning and bucketing b. Designing test queries by using block and bucket sampling before running the queries on large datasets c. Using bucket map joins and parallel processing to run queries faster Visit www.qubole.com for more information.

Effective Hive Queries

Qubole

More from Qubole (20)

Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...

7 Big Data Challenges and How to Overcome Them

State of Big Data Adoption

Big Data at Pinterest - Presented by Qubole

5 Factors Impacting Your Big Data Project's Performance

Spark on Yarn

Atlanta MLConf

Running Spark on Cloud

Qubole State of the Big Data Industry

Atlanta Data Science Meetup | Qubole slides

Qubole presentation for the Cleveland Big Data and Hadoop Meetup

BIPD Tech Tuesday Presentation - Qubole

Harnessing the Hadoop Ecosystem Optimizations in Apache Hive

Optimizing Big Data to run in the Public Cloud

Getting to 1.5M Ads/sec: How DataXu manages Big Data

Expert Big Data Tips

Big dataproposal

Presto in the cloud

Basic Sentiment Analysis using Hive

Effective Hive Queries

Recently uploaded

Abortion pills in Kuwait city !🌆+918761049707^) Where to get cytotec pills in salmiyah, WhatsApp '''(+918761049707)''' Explore Discreet and Safe Abortion Pill Options in Dubai, Empowering Women With Confidential and Medically Supervised Choices For Reproductive Health" Abortion Pills In Dubai, Abu Dhabi/Alain/Sharjah/RAK City Satwa-Mifepristone and Misoprostol Available in Dubai/Abu Dhabi - i Pills in Dubai and Cost Of Cytotec in WhatsApp (+918761049707) Abortion Pills in Dubai/Abu Dubai/ Abu Dhabi. Are you stranded with unwanted pregnancy in Dubai, Abu Dhabi , the United Arab Emirates (U.A.E), Qatar, Oman, Saudi Arabia or Kuwait? you can now contact us now on Whatsapp Dr AJ:+918761049707to buy safe abortion pills In Dubai, Abu Dhabi, Sharjah, Al Ain, Ajman, RAK City, Ras Al Khaimah and Fujairah to terminate an unwanted pregnancy in Dubai and the United Arab Emirates. Get your Discreet 100% Safe (+918761049707 )*Effective Abortion Pills For Sale in Dubai, Abu Dhabi, KUWAIT, QATAR, BAHRAIN, DOHA, SALMIYA, Sharjah, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE. BUY Mifepristone and Misoprostol (Cytotec), Mtp Kit In UAE. Abortion pills available in UAE (United Arab Emirates), Saudi Arabia, Kuwait, Oman, Bahrain and Qatar. Contact us today. +918761049707 -The UAE’s leading abortion care service in Dubai. Abortion Treatment. Medical Abortion. Surgical Abortion. Find A Clinic like Dr Maria Abortion clinic in Dubai We have Abortion Pills / Cytotec Tablets Available in Dubai, Abu Dhabi, KUWAIT, QATAR, BAHRAIN, DOHA, SALMIYA, Sharjah, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE., buy cytotec in Dubai abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain We sell original abortion medicine which includes: Cytotec 200mcg (Misoprostol), Mifepristone, Mifegest-kit, Misoclear, Emergency contraceptive pills, Morning after sex pills, ipills, pills to prevent pregnancy 72 hours after sex. All our pills are manufactured by reputable medical manufacturing companies like PFIZER. Medical abortion is easy and effective for everyone to perform in their own privacy. There are very few complications that may arise from medical abortion if one follows the right guidelines as instructed by the obstetrician. Abortion Pills in Dubai Can Now Be Offered at Dr Maria Abortion clinic in Dubai, F.D.A. Mifepristone and Misoprostol, the first of two drugs in medication abortions, previously had to be dispensed only by clinics, doctors or a few mail-order pharmacies like Dr Maria Abortion clinic in Dubai . Now, We can provide it. For the first time, retail pharmacies, like Dr Maria Abortion clinic in Dubai, will be Able to offer abortion pills in Dubai under a regulatory change made Tuesday by the Food and Drug Administration. The action could significantly expand access to abortion through medication. Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg

In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia

ahmedjiabur940

Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...

nirzagarg

Discover Why Less is More in B2B Research

michael115558

Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service Available 24/7 Hire Our agency presents a selection of young, charming call girls #J11 available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives,#J11 College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Kolkata ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 8005736733 🔝. for details Operating 24/7, we serve various locations in Kolkata, including Green Park, near metro stations. For premium call girl services in Kolkata 🔝 8005736733 🔝. Thank you for considering us!

Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...

HyderabadDolls

7. Epi of Chronic respiratory diseases.ppt

ibrahimabdi22

Context 1. Housing Agent collected resale prices on HDB apartments in Singapore. Objective 2. To predict resale prices in to advise his potential clients. Strategies 3. Explore & Clean data for analysis. 4. Perform K-Means Clustering, in Orange, to find possible segments in the customer data. 5. Tune the model to improve its performance. 6. Visualise the findings, share conclusions, and give insight-driven recommendations. Author: Anthony mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com

Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange

ThinkInnovation

Kings of Saudi Arabia, information about them

eitharjee

Aspirational Block Program Block Syaldey District - Almora

GovindSinghDasila

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...

gajnagarg

Saudi Arabia [ Abortion pills) Jeddah/riaydh/dammam/+966572737505☎️] cytotec tablets uses abortion pills 💊💊 How effective is the abortion pill? 💊💊 +966572737505) "Abortion pills in Jeddah" how to get cytotec tablets in Riyadh " Abortion pills in dammam*💊💊 The abortion pill is very effective. If you’re taking mifepristone and misoprostol, it depends on how far along the pregnancy is, and how many doses of medicine you take:💊💊 +966572737505) how to buy cytotec pills At 8 weeks pregnant or less, it works about 94-98% of the time. +966572737505[ 💊💊💊 At 8-9 weeks pregnant, it works about 94-96% of the time. +966572737505) At 9-10 weeks pregnant, it works about 91-93% of the time. +966572737505)💊💊 If you take an extra dose of misoprostol, it works about 99% of the time. At 10-11 weeks pregnant, it works about 87% of the time. +966572737505) If you take an extra dose of misoprostol, it works about 98% of the time. In general, taking both mifepristone and+966572737505 misoprostol works a bit better than taking misoprostol only. +966572737505 Taking misoprostol alone works to end the+966572737505 pregnancy about 85-95% of the time — depending on how far along the+966572737505 pregnancy is and how you take the medicine. +966572737505 The abortion pill usually works, but if it doesn’t, you can take more medicine or have an in-clinic abortion. +966572737505 When can I take the abortion pill?+966572737505 In general, you can have a medication abortion up to 77 days (11 weeks)+966572737505 after the first day of your last period. If it’s been 78 days or more since the first day of your last+966572737505 period, you can have an in-clinic abortion to end your pregnancy.+966572737505 Why do people choose the abortion pill? Which kind of abortion you choose all depends on your personal+966572737505 preference and situation. With+966572737505 medication+966572737505 abortion, some people like that you don’t need to have a procedure in a doctor’s office. You can have your medication abortion on your own+966572737505 schedule, at home or in another comfortable place that you choose.+966572737505 You get to decide who you want to be with during your abortion, or you can go it alone. Because+966572737505 medication abortion is similar to a miscarriage, many people feel like it’s more “natural” and less invasive. And some+966572737505 people may not have an in-clinic abortion provider close by, so abortion pills are more available to+966572737505 them. +966572737505 Your doctor, nurse, or health center staff can help you decide which kind of abortion is best for you. +966572737505 More questions from patients: Saudi Arabia+966572737505 CYTOTEC Misoprostol Tablets. Misoprostol is a medication that can prevent stomach ulcers if you also take NSAID medications. It reduces the amount of acid in your stomach, which protects your stomach lining. The brand name of this medication is Cytotec®.+966573737505) Unwanted Kit is a combination of two medicin

Abortion pills in Jeddah | +966572737505 | Get Cytotec

Abortion pills in Riyadh +966572737505 get cytotec

Building Real-Time Pipelines With FLaNK Timothy Spann, Principal Developer Advocate, Streaming - Cloudera Future of Data meetup, startup grind, AI Camp The combination of Apache Flink, Apache NiFi, and Apache Kafka for building real-time data processing pipelines is extremely powerful, as demonstrated by this case study using the FLaNK-MTA project. The project leverages these technologies to process and analyze real-time data from the New York City Metropolitan Transportation Authority (MTA). FLaNK-MTA demonstrates how to efficiently collect, transform, and analyze high-volume data streams, enabling timely insights and decision-making. Apache NiFi Apache Kafka Apache Flink Apache Iceberg LLM Generative AI Slack Postgresql

DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK

Timothy Spann

Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...

nirzagarg

High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girls Near 5 Star Hotel Booking Now open +91- 9332606886 Pune VIP Call Girls is highly qualified professionals. #S10 They will offer only top quality services and never try to scam or drain you of money; in fact, they strive to give complete satisfaction so that each cent spent was worth its worth. Furthermore, these professionals pride themselves in keeping customer details confidential; any information gleaned will never be shared with any third party.$V15 Two shots with one girl: ₹4500/in-call, ₹7500/out-call Body to body massage with : ₹5500/in-call Full night for one person: ₹7000/in-call, ₹12000/out-call •𝐂𝐨𝐥𝐥𝐞𝐠𝐞 𝐆𝐢𝐫𝐥𝐬 •𝐇𝐨𝐮𝐬𝐞𝐰𝐢𝐯𝐞𝐬 •𝐇𝐢𝐠𝐡 𝐏𝐫𝐨𝐟𝐢𝐥𝐞 𝐆𝐢𝐫𝐥𝐬 •𝐑𝐚𝐦𝐩 𝐌𝐨𝐝𝐞𝐥𝐬 •𝐅𝐨𝐫𝐞𝐢𝐠𝐧𝐞𝐫 𝐆𝐢𝐫𝐥𝐬 𝐎𝐮𝐫 𝐞𝐬𝐜𝐨𝐫𝐭 𝐬𝐞𝐫𝐯𝐢𝐜𝐞𝐬 •𝐇𝐉 (𝐇𝐚𝐧𝐝 𝐉𝐨𝐛) •𝐁𝐉 (𝐁𝐥𝐨𝐰𝐣𝐨𝐛) # Completion # (Oral To Completion) # Special Massage # O-Level (Oral ) # Oral With A Noncondom) # COB (Come On Body) # Extraball (Have Many Times) # All Meetings 100%Safe

High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...

kumargunjan9515

Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort Service Available 24/7 Hire CALL GIRL IN Lucknow 9548273370 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRL IN #j11 We are Providing :- ● – Private independent collage Going girls . ● – independent Models . ● – House Wife’s . ● – Private Independent House Wife’s ● – Corporate M.N.C Working Profiles . ● – Call Center Girls . ● – Live Band Girls . ●- Foreigners & Many More . Service type: 1.In call 2.out call 3. full Lip to Lip kiss 4.69 5.b-job without Condom 6. Hard Core sex & Much More. 7 Body to Body Touch 8 Kissing 9 Sucking Boobs and More 10 Enjoy by Hand 11 Relax By Oral 12 Sex with Happy Ending • In Call and Out Call Service • 3* 5* 7* Hotels Service • 24 Hours Available • Indian, Russian, Punjabi, Kashmiri Escorts • Real Models, College Girls, House Wife, Also Available • Short Time and Full Time Service Available • Hygienic Full AC Neat and Clean Rooms Avail. In Hotel 24 hours • Daily Escorts Staff Available • Minimum to Maximu m Range Available.c

Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...

HyderabadDolls

Computer science Sql cheat sheet.pdf.pdf

SayantanBiswas37

Digital Transformation Playbook by Graham Ware

Graham Ware

Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now Booking Contact Details :- WhatsApp Chat :- +91-7737669865 We offer all types of girls of your choice with space. Our escorts are fully cooperative and understand your needs. All types of call girls like Housewives, College girls,#K09 Russian girls, Muslim girls, Afghani girls, Bengali girls, Working girls, south Indian girls, Punjabi girls, etc. In-Call: — You Can Reach At Our Place in Bangalore Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call: — Service for Out Call You have To Come Pick The Girl From My Place We Also Provide Door-Step Services Hygienic: — Full Ac Neat And Clean Rooms Available In Hotel 24 * 7 Hrs In Bangalore Our Services and Rates: – One Shot — 2500/in call (time ½ hour), 5000/out call Two shot with one girl — 5000/in call (time 1 hour), 6000/out call Body to body massage with sex- 3000/in call (time 1 hour) full night for one person– 8000/in call, 10000/out call (shot limit 4 shot) We are available 24*7 all days of the year

Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now

gargpaaro

Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...

nirzagarg

Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736733 Neha Thakur Our agency presents a selection of young, charming call girls #J11 available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives,#J11 College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Kolkata ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 8005736733 🔝. for details Operating 24/7, we serve various locations in Kolkata, including Green Park, near metro stations. For premium call girl services in Kolkata 🔝 8005736733 🔝. Thank you for considering us!

Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...

HyderabadDolls

Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Can Get The Service Booking Now open +91- 9332606886 Pune VIP Call Girls is highly qualified professionals. #S10 They will offer only top quality services and never try to scam or drain you of money; in fact, they strive to give complete satisfaction so that each cent spent was worth its worth. Furthermore, these professionals pride themselves in keeping customer details confidential; any information gleaned will never be shared with any third party.$V15 Two shots with one girl: ₹4500/in-call, ₹7500/out-call Body to body massage with : ₹5500/in-call Full night for one person: ₹7000/in-call, ₹12000/out-call •𝐂𝐨𝐥𝐥𝐞𝐠𝐞 𝐆𝐢𝐫𝐥𝐬 •𝐇𝐨𝐮𝐬𝐞𝐰𝐢𝐯𝐞𝐬 •𝐇𝐢𝐠𝐡 𝐏𝐫𝐨𝐟𝐢𝐥𝐞 𝐆𝐢𝐫𝐥𝐬 •𝐑𝐚𝐦𝐩 𝐌𝐨𝐝𝐞𝐥𝐬 •𝐅𝐨𝐫𝐞𝐢𝐠𝐧𝐞𝐫 𝐆𝐢𝐫𝐥𝐬 𝐎𝐮𝐫 𝐞𝐬𝐜𝐨𝐫𝐭 𝐬𝐞𝐫𝐯𝐢𝐜𝐞𝐬 •𝐇𝐉 (𝐇𝐚𝐧𝐝 𝐉𝐨𝐛) •𝐁𝐉 (𝐁𝐥𝐨𝐰𝐣𝐨𝐛) # Completion # (Oral To Completion) # Special Massage # O-Level (Oral ) # Oral With A Noncondom) # COB (Come On Body) # Extraball (Have Many Times) # All Meetings 100%Safe

Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...

kumargunjan9515

Recently uploaded (20)

In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia

Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...

Discover Why Less is More in B2B Research

Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...

7. Epi of Chronic respiratory diseases.ppt

Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange

Kings of Saudi Arabia, information about them

Aspirational Block Program Block Syaldey District - Almora

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...

Abortion pills in Jeddah | +966572737505 | Get Cytotec

DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK

Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...

High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...

Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...

Computer science Sql cheat sheet.pdf.pdf

Digital Transformation Playbook by Graham Ware

Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now

Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...

Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...

Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...

Big Data Platform at Pinterest

1. Confidentia l Mao Ye Big Data Platform at interest 1

2. Data Architecture Design Choices for Hadoop Platform Pinball for Workflow Management

3. Data Architecture

4. Data at Pinterest • 60 Billion Pins • 1 Billion boards • 100M MAU • 60 PB of data on S3 • 3 PB processed every day • 2000 node Hadoop cluster • 250 engineers

5. Pinterest Data Architecture App

6. Pinterest Data Architecture App events Kafka Secor Singer

7. Pinterest Data Architecture App events Kafka Secor Singer

8. Pinterest Data Architecture App events Kafka Secor Skyline Pinball Redshift Pinalytics Features Qubole (Hadoop) Singer

9. Design Choices for Hadoop Platform

10. • Ephemeral clusters • Access control layer • Shared data store • Easy deployment Hadoop Platform Requirements • Isolated multi-tenancy • Elasticity • Support multiple clusters

11. Decoupling compute & storage Hadoop Cluster 1 Transient HDFS Hadoop Cluster 2 Transient HDFS S3 Persistent Store

12. Centralized Hive Metastore Hive Metastore Pig Cascading Hive HDFS/S3 DataMetadata

13. Multi-layered Packaging Mapreduce Jobs Hadoop Jars/Libs Job/User level Configs Software Packages/Libs Configs (OS/Hadoop) Misc Sys Admin OS Bootstrap Script Core SW Runtime Staging (on S3) Automated Configuration (Masterless Puppet) Baked AMI

14. Executor Abstraction Layer Hive Metastore HDFS/S3 Qubole Managed Hadoop EMR Executor Pinball Dev Server

15. • API for simplified executor abstraction • Advanced support for spot instances • Baked AMI customization Why Qubole? • Hadoop & Spark as managed services • Tight integration with Hive • Graceful cluster scaling

16. Confidentia l Pinball for Workflow Management

17. Confidentia l ● Scale: o 60 Billion Pins o Hundreds of workflows o Thousands of jobs o 500+ jobs in a workflow o 3 petabytes processed daily ● Support: o Hadoop, Cascading, Hive, Spark … Scale of Processing job workflow

18. Confidentia l Why Pinball? ● Requirements o Simple abstractions o Extensible in future o Reliable stateless computing o Easy to debug o Scales horizontally o Can be upgraded w/o aborting workflows o Rich features like auto-retries, per-job emails, overrun policies… ● Options o Apache Oozie, Azkaban, Luigi

19. Confidentia l Pinball Design Master Worker Scheduler Command Line Clients UI

20. Confidentia l ● Workflow o A directed graph of nodes called jobs ● Edge o Run after dependence ● Node o Job is a node Workflow Model

21. Confidentia l Job State ● Job state is captured in a token ● Tokens are named hierarchically Master Job Token version: 123 name: /workflow/w1/job owner: worker_0 expiration: 1234567 data: JobTemplate(....)

22. Confidentia l Job State Machine RUNNABLE RUNNINGWAITING

23. Confidentia l ● Master keeps the state ● Workers claim and execute tasks ● Horizontally scalable Master Worker Interaction Worker Master Persistent Store 1: request 2: update 3: ack

24. Confidentia l Master ● Entire state is kept in memory ● Each state update is synchronously persisted before master replies to client ● Master runs on a single thread – no concurrency issues

25. Confidentia l Worker

26. Confidentia l Open Source Git repo: https://github.com/pinterest/pinball Mailing list: https://groups.google.com/forum/#!forum/ pinball-users

27. Confidentia l Thank You