Presto Bangalore Meetup1 Event Listeners@qubole

•

1 recomendación•197 vistas

Shubham Tagra

Talk at Presto Bangalore Meetup on Presto's Event Listeners framework

Software

Analysing presto setup using
Event Listener
Oct 13th, 2018
Ankit Dixit

00Copyright 2017 © Qubole
Goal
● What is an event listener
● How to leverage it for analysis

00Copyright 2017 © Qubole
Analysis time, Source

00Copyright 2017 © Qubole
Cluster presto usage, Query distribution

00Copyright 2017 © Qubole
What is an event listener
● Mechanism to get notified at trigger points: events
● Events:
● Query creation
● Query completion
● Split completion
● Query/Task stats, session info

00Copyright 2017 © Qubole
Code
Event Listener github

00Copyright 2017 © Qubole
Setup
● TPC DS scale 1000
● Master r3.2xlarge
● 3 Slave r3.4xlarge
● 19 perf queries ran multiple times

00Copyright 2017 © Qubole
Use Cases
● Resource Accounting
● Configure alarms on system state
○ Query completion rate
○ Queue size
○ Average data read
● Cluster configuration analysis
● Post execution hooks

00Copyright 2017 © Qubole
Limitations
● Triggers only at event boundaries
● Does not provide with a overall system state
● Not suitable for proactive analysis

Más contenido relacionado

La actualidad más candente

Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwiantoidsecconf

Graph Computing with Apache TinkerPopJason Plurad

Graphite, an introductionjamesrwu

DN 2017 | The King is Dead, Long Live the King | Thomas Richter | Swarm64Dataconomy Media

IPTC News Exchange Working Group 2013 Autumn MeetingStuart Myles

Iot data aggrigationdokechin

The Current Messaging Landscape: RabbitMQ, ZeroMQ, nsq, KafkaAll Things Open

Botconf 2013 - DNS-based Botnet C2 Server DetectionSensePost

La actualidad más candente (8)

Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwianto

Graph Computing with Apache TinkerPop

Graphite, an introduction

DN 2017 | The King is Dead, Long Live the King | Thomas Richter | Swarm64

IPTC News Exchange Working Group 2013 Autumn Meeting

Iot data aggrigation

The Current Messaging Landscape: RabbitMQ, ZeroMQ, nsq, Kafka

Botconf 2013 - DNS-based Botnet C2 Server Detection

Similar a Presto Bangalore Meetup1 Event Listeners@qubole

Key considerations in productionizing streaming applicationsKafkaZone

Ridwan Fadjar Septian PyCon ID 2021 Regular Talk - django application monitor...Ridwan Fadjar

Webinar slides: Backup Management for MySQL, MariaDB, PostgreSQL & MongoDB wi...Severalnines

New RelicGene Chuang

Big Query BasicsIdo Green

Cost Effective Presto on AWS with Spot Nodes - Strata SF 2019Shubham Tagra

OpenTelemetry For OperatorsKevin Brockhoff

Presto Summit 2018 - 10 - Qubolekbajda

Security Monitoring for big Infrastructures without a Million Dollar budgetJuan Berner

Log FilesHeinrich Hartmann

Night owl by Boyd Meyer of PROS Mark Kerzner

20180503 kube con eu kubernetes metrics deep diveBob Cotton

Netflix Open Source: Building a Distributed and Automated Open Source Programaspyker

Building a Distributed & Automated Open Source Program at NetflixAll Things Open

Real-time Analytics with Upsert Using Apache Kafka and Apache Pinot | Yupeng ...HostedbyConfluent

Pivotal Greenplum Cloud Marketplaces - Greenplum Summit 2019VMware Tanzu

Eko10 workshop - OPEN SOURCE DATABASE MONITORINGPablo Garbossa

Google Cloud Platform Special TrainingSimon Su

Ceilometer presentation ODS Grizzly.pdfOpenStack Foundation

Webinar slides: How to Automate & Manage PostgreSQL with ClusterControlSeveralnines

Similar a Presto Bangalore Meetup1 Event Listeners@qubole (20)

Key considerations in productionizing streaming applications

Ridwan Fadjar Septian PyCon ID 2021 Regular Talk - django application monitor...

Webinar slides: Backup Management for MySQL, MariaDB, PostgreSQL & MongoDB wi...

New Relic

Big Query Basics

Cost Effective Presto on AWS with Spot Nodes - Strata SF 2019

OpenTelemetry For Operators

Presto Summit 2018 - 10 - Qubole

Security Monitoring for big Infrastructures without a Million Dollar budget

Log Files

Night owl by Boyd Meyer of PROS

20180503 kube con eu kubernetes metrics deep dive

Netflix Open Source: Building a Distributed and Automated Open Source Program

Building a Distributed & Automated Open Source Program at Netflix

Real-time Analytics with Upsert Using Apache Kafka and Apache Pinot | Yupeng ...

Pivotal Greenplum Cloud Marketplaces - Greenplum Summit 2019

Eko10 workshop - OPEN SOURCE DATABASE MONITORING

Google Cloud Platform Special Training

Ceilometer presentation ODS Grizzly.pdf

Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl

Más de Shubham Tagra

Alluxio Data Orchestration Platform for the CloudShubham Tagra

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...Shubham Tagra

Presto best practices for Cluster admins, data engineers and analystsShubham Tagra

Enabling presto to handle massive scale at lightning speedShubham Tagra

Debugging data pipelines @OLA by Karan KumarShubham Tagra

Journey and evolution of Presto@GrabShubham Tagra

Enabling Presto to handle massive scale at lightning speedShubham Tagra

Presto Bangalore Meetup1 Presto Raptor@olaShubham Tagra

Presto Bangalore Meetup1 Ranger+Presto@olaShubham Tagra

RubiXShubham Tagra

Más de Shubham Tagra (10)

Alluxio Data Orchestration Platform for the Cloud

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...

Presto best practices for Cluster admins, data engineers and analysts

Enabling presto to handle massive scale at lightning speed

Debugging data pipelines @OLA by Karan Kumar

Journey and evolution of Presto@Grab

Enabling Presto to handle massive scale at lightning speed

Presto Bangalore Meetup1 Presto Raptor@ola

Presto Bangalore Meetup1 Ranger+Presto@ola

RubiX

Último

Folding Cheat Sheet #4 - fourth in a seriesPhilip Schwarz

Unveiling the Future: Sylius 2.0 New FeaturesŁukasz Chruściel

What is Fashion PLM and Why Do You Need ItWave PLM

Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...OnePlan Solutions

Powering Real-Time Decisions with Continuous Data StreamsSafe Software

KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app

Introduction Computer Science - Software Design.pdfFerryKemperman

How to submit a standout Adobe Champion ApplicationBradBedford3

Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Mater

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...confluent

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions

Sending Calendar Invites on SES and Calendarsnack.pdf31events.com

What is Advanced Excel and what are some best practices for designing and cre...Technogeeks

Precise and Complete Requirements? An Elusive GoalLionel Briand

React Server Component in Next.js by Hanief UtamaHanief Utama

英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0

CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies

Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC

Intelligent Home Wi-Fi Solutions | ThinkPalmSujith Sukumaran

Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig