Cloud World Forum: Large Scale Data Analysis on AWS

•

1 recomendación•740 vistas

In this talk from the Cloud World Forum Big Data event in June this year, I discuss the benefits of using the AWS Cloud for large scale computation and data processing workloads.

Tecnología

ianmas@amazon.com
@IanMmmm
LARGE SCALE DATA
ANALYSIS WITH AWS 
 
Ian Massingham – Technical Evangelist

THE MORE DATA YOU COLLECT
THE MORE VALUE YOU CAN
DERIVE FROM IT!

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Lower cost,
higher throughput

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Lower cost,
higher throughput
Highly
constrained

+ ELASTIC AND HIGHLY SCALABLE
+ NO UPFRONT CAPITAL EXPENSE
+ ONLY PAY FOR WHAT YOU USE
+ AVAILABLE ON-DEMAND
= REMOVE CONSTRAINTS

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
AWS Import / Export
AWS Direct Connect

Inbound data transfer is free
Multipart upload to S3
Physical media
AWS Direct Connect

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Amazon S3,
Amazon Glacier,
Amazon DynamoDB,
Amazon RDS,
Amazon Redshift,
AWS Storage Gateway,
Data on Amazon EC2

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Amazon EC2
Amazon Elastic
MapReduce

Instead of
$20+ MILLIONS
in infrastructure!

Instead of
$68+ MILLIONS
in infrastructure!

GPU INSTANCES"
"
G2"
CG1
1x NVIDIA Kepler GK104 
8 vCPU (Intel Xeon E5-2670)
2x NVIDIA Fermi M2050 
16 vCPU (Intel Xeon X5570)
$0.65/h
$2.10/h

ON A SINGLE INSTANCE
COMPUTE TIME: 4h
COST: 4h x $2.1 = $8.4

ON MULTIPLE INSTANCES
COMPUTE TIME: 1h
COST: 1h x 4 x $2.1 = $8.4

AMAZON ELASTIC
MAPREDUCE 
HADOOP AS A SERVICE!

•  SPLITS DATA INTO PIECES
•  LETS PROCESSING OCCUR
•  GATHERS THE RESULTS!

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Amazon S3,
Amazon DynamoDB,
Amazon RDS,
Amazon Redshift,
Data on Amazon EC2

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
BATCH
PROCESSING

AMAZON KINESIS 
REAL-TIME DATA STREAM PROCESSING!

Real-time response to content
in semi-structured data streams 
 
Relatively simple computations
on data (aggregates, ﬁlters,
sliding window, etc.)

Hourly server logs: how your
systems went wrong an hour ago
Weekly / Monthly Bill: What you
spent this past billing cycle
Daily customer report from your
website: tells you what deal or ad
to try next time
Daily fraud reports: tells you if there
was fraud yesterday
Daily business reports: tells me
how customers used AWS services
yesterday
Real-time metrics: what just went
wrong now
Real-time spending alerts/caps:
guaranteeing you can’t overspend
Real-time analysis: what to offer
the current customer now
Real-time detection: blocks
fraudulent use now
Fast ETL into Amazon Redshift:
how are customers using services
now

GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!
Amazon S3,
Amazon DynamoDB,
Amazon RDS,
Amazon Redshift,
Data on Amazon EC2
Amazon EC2
Amazon Elastic
MapReduce
Amazon S3,
Amazon Glacier,
Amazon DynamoDB,
Amazon RDS,
Amazon Redshift,
AWS Storage Gateway,
Data on Amazon EC2
AWS Import / Export
AWS Direct Connect

GENERATE ➔ ➔ SHARE!
STREAM
PROCESSING
Amazon S3,
Amazon DynamoDB,
Amazon RDS,
Amazon Redshift,
Data on Amazon EC2
Amazon Kinesis
Stream Processing on
Amazon EC2

WANT TO KNOW MORE?
aws.amazon.com/solutions/case-studies/big-data/!

ianmas@amazon.com
@IanMmmm
THANK YOU 
 
Ian Massingham – Technical Evangelist

Further References
Atomic Fiction Case-Study Video!
https://www.youtube.com/watch?v=ljHo1_5sWxo!
Slideshare with full details on the Schrodinger Materials Science case-study!
http://www.slideshare.net/insideHPC/cycle-computing-recordbreaking-peta-scale-hpc-run!
Real-time Streaming and Querying with Amazon Kinesis and Amazon EMR Video!
https://www.youtube.com/watch?v=NIa33ZwFa8E!
!
!

Más contenido relacionado

La actualidad más candente

Aws pricing overview

saifam

Cost Optimization on AWS - Pop-up Loft Tel Aviv

Amazon Web Services

AWS Cost Optimisation Made Easy

Madan Ganesh Velayudham

AWS Batch: Simplifying Batch Computing in the Cloud

Amazon Web Services

E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.

Clustrix

AWS Melbourne CO Meetup - Introduction - 20 Nov 2017

Peter Shi

Scalability: Rdbms Vs Other Data Stores

Ramki Gaddipati

Amazon EC2 provides you with the flexibility to cost optimize your computing portfolio through purchasing models that fit your business needs. With the flexibility of mix-and-match purchasing models, you can grow your compute capacity and throughput and enable new types of cloud computing applications with the lowest TCO. In this session, we will explore combining pay-as-you-go (On-Demand), reserve ahead of time for discounts (Reserved), and high-discount spare capacity (Spot) purchasing models to optimize costs while maintaining high performance and availability for your applications. Common application examples will be used to demonstrate how to best combine EC2’s purchasing models. You will leave the session with best practices you can immediately apply to your application portfolio.

SRV301 Getting the Most Bang for your Buck with #EC2 #Winning

Amazon Web Services

Alliance 2017 - Jet Reports Tips and Trips

Sparkrock

AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community

AWS Germany

Putting Cloud Controls to Work! Threats are evolving, becoming increasingly sophisticated and more numerous as cloud adoption by enterprise is surging. Traditional approaches to securing IT systems have expanded to include new requirements of cloud computing. This webinar outlines how to successfully harden your AWS infrastructure with IAM best practices. This deck covers: 1) Getting started with the basics 2) How organizations balance the 'need to grant access' with the requirement to secure and protect the business 3) Best practices for managing third-party identities and access control

AWS & Infrastructure Hardening - Cloud Infrastructure Security

Nutanix Beam

Being a pioneer in a field most of the time gives you a boost when you are showing your portfolio. It is enriched with lots of fancy and buzz words and when you talk with your friends and colleagues you sound as you are talking rocket science and you look smart. And that is a good thing. On the other side, you have a lot of hidden traps that you need to overcome when working with new architecture technologies. In this talk I will present the architecture that me and my team created on top of AWS. The architecture currently is serving millions of users on daily basis with 90%+ score in every performance test. It consists of 15+ services that are provided by AWS. The presentation will focus on all of the good things and the problems that we are facing while using AWS services.

Aws architecture problems while being fancy

Goran Kopevski

Sometimes it seems like you can hardly go a day without hearing about how “serverless” is going to change the world of backend architecture. But aside from toy proofs of concept, how are you realistically supposed to put it into practice? Most of us work with years-old codebases that are resistant to decoupling, much less easy to transition to serverless. Come hear how Yelp has been moving a 10 year old codebase from tangled spaghetti toward a serverless future using AWS Step Functions (SFN). You’ll gain familiarity with SFN as infrastructure, learn how it can be used to effectively disentangle complicated systems, and understand how to incrementally introduce serverless components into your monolithic application.

From monolithic to serverless with Amazon Step Functions

Scott Triglia

AWS offers many data services, each optimized for a specific set of structure, size, latency, and concurrency requirements. Making the best use of all specialized services has historically required custom, error-prone data transformation and transport. Now, users can use the AWS Data Pipeline service to orchestrate data flows between Amazon S3, Amazon RDS, Amazon DynamoDB, Amazon Redshift, and on-premise data stores, seamlessly and efficiently applying EC2 instances and EMR clusters to process and transform data. In this session, we demonstrate how you can use AWS Data Pipeline to coordinate your Big Data workflows, applying the optimal data storage technology to each part of your data integration architecture. Swipely's Head of Engineering shows how Swipely uses AWS Data Pipeline to build batch analytics, backfilling all their data, while using resources efficiently. Consequently, Swipely launches novel product features with less development time and less operational complexity.

Big Data Integration & Analytics Data Flows with AWS Data Pipeline (BDT207) |...

Amazon Web Services

Azure Web Scalability

Matias Quaranta

Presented at Snowplow London Meetup, 8 February 2017 Dejan Petelin, head of data science at Gousto, gave a presentation about their data journey, explaining how data reflects the customer’s voice and the importance of joining up all data sources. The goal is to delight and retain customers – critical for a subscription business like Gousto’s. Gousto is using Snowplow as a unified log, to scale up its data capabilities, listen to its customer and provide them with a more personalized experience. Finally, Gousto is moving to the real-time pipeline to enable just-in-time personalization.

How Gousto is moving to just-in-time personalization with Snowplow

Giuseppe Gaviani

Value, TCO & Cost Optimisation

Amazon Web Services

AWS Machine Learning Big Data NYC

Alexis Perrier

Voice Powered Analytics: Data Analytics Week at the SF Loft

Amazon Web Services

Azure DocumentDB en Dev@Nights

Matias Quaranta

La actualidad más candente (20)

Aws pricing overview

Cost Optimization on AWS - Pop-up Loft Tel Aviv

AWS Cost Optimisation Made Easy

AWS Batch: Simplifying Batch Computing in the Cloud

E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.

AWS Melbourne CO Meetup - Introduction - 20 Nov 2017

Scalability: Rdbms Vs Other Data Stores

SRV301 Getting the Most Bang for your Buck with #EC2 #Winning

Alliance 2017 - Jet Reports Tips and Trips

AWS Summit Berlin 2013 - Tadaa - HD Camera and Photo Community

AWS & Infrastructure Hardening - Cloud Infrastructure Security

Aws architecture problems while being fancy

From monolithic to serverless with Amazon Step Functions

Big Data Integration & Analytics Data Flows with AWS Data Pipeline (BDT207) |...

Azure Web Scalability

How Gousto is moving to just-in-time personalization with Snowplow

Value, TCO & Cost Optimisation

AWS Machine Learning Big Data NYC

Voice Powered Analytics: Data Analytics Week at the SF Loft

Azure DocumentDB en Dev@Nights

Similar a Cloud World Forum: Large Scale Data Analysis on AWS

2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo

Ian Massingham

Workshop part2 – Big Data

Amazon Web Services

In this webinar AWS Technical Evangelist, Ian Massingham, discusses the role that AWS services can play in helping you to derive value from your data, from stream processing with Amazon Kinesis, techniques for managing ingest of large data sets, through to processing data with Amazon Elastic MapReduce (EMR) and its ecosystem of tools and running large scale data warehouses on AWS with Redshift. View the recording: http://youtu.be/7bkqopn19WY

Journey Through the AWS Cloud - Big Data Analysis

Amazon Web Services

Large Scale Data Analysis with AWS

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes, and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

Build Data Lakes and Analytics on AWS

Amazon Web Services

AWS re:Invent Recap from AWS User Group UK meetup #8

Ian Massingham

Learn best practices for building a real-time streaming data architecture on AWS with Spark Streaming, Amazon Kinesis, and Amazon Elastic MapReduce (EMR). Get a closer look at how to ingest streaming data scalably and durably from data producers like mobile devices, servers, and even web browsers, and design a stream processing application with minimal data duplication and exactly-once processing. Presented by: Guy Ernest, Principal Business Development Manager, Amazon Web Services Customer Guest: Harry Koch, Solutions Architecture, Philips

Real-Time Streaming Data on AWS

Amazon Web Services

As more and more organizations strive to gain real-time insights into their business, streaming data has become ubiquitous. Typical streaming data analytics solutions require specific skills and complex infrastructure. However, with Amazon Kinesis Analytics, you can analyze streaming data in real-time with standard SQL—there is no need to learn new programming languages or processing frameworks. In this session, we dive deep into the capabilities of Amazon Kinesis Analytics using real-world examples. We’ll present an end-to-end streaming data solution using Amazon Kinesis Streams for data ingestion, Amazon Kinesis Analytics for real-time processing, and Amazon Kinesis Firehose for persistence. We review in detail how to write SQL queries using streaming data and discuss best practices to optimize and monitor your Amazon Kinesis Analytics applications. Lastly, we discuss how to estimate the cost of the entire system.

Analyzing Streaming Data in Real-time with Amazon Kinesis

Amazon Web Services

Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information. In this session, we present an end-to-end streaming data solution using Kinesis Streams for data ingestion, Kinesis Analytics for real-time processing, and Kinesis Firehose for persistence. We review in detail how to write SQL queries using streaming data and discuss best practices to optimize and monitor your Kinesis Analytics applications. Lastly, we discuss how to estimate the cost of the entire system.

ABD301-Analyzing Streaming Data in Real Time with Amazon Kinesis

Amazon Web Services

Your customers probably want a better experience with your brand. Your different business teams want and need better insights in their decision making. Almost certainly, your finance and operations teams require this to happen at a fraction of the cost of traditional on-premises options. Modern data architectures on AWS help many of our best customers realize all of those goals. Your business data contains critical information about customer behaviors, operational decisions, and many factors that have financial impact on your organization. Increasingly, this data sits beyond your transactional systems, and is too big, too fast, and too complex for existing systems to handle. AWS Data and Analytics services are designed from our customers' requirements to ingest, store, analyze, and consume information at record-breaking scale. In this session you will learn how these services work together to deliver business automation, enhance customer engagement and intelligence.

Driving Business Insights with a Modern Data Architecture AWS Summit SG 2017

Amazon Web Services

The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

This session, we focus on common use cases and design patterns for predictive analytics using Amazon EMR. We address accessing data from a data lake, extraction and preprocessing with Apache Spark, analytics and machine learning code development with notebooks (Jupyter, Zeppelin), and data visualization using Amazon QuickSight. We cover other operational topics, such as deployment patterns for ad hoc exploration and batch workloads using Spot and multi-user notebooks. The intended audience for this session includes technical users who are building statistical and data analytics models for the business using tools, such as Python, R, Spark, Presto, Amazon EMR, Notebooks.

Best Practices for Distributed Machine Learning and Predictive Analytics Usin...

Amazon Web Services

In this session, we simplify big data processing as a data bus comprising various stages: collect, store, process, analyze, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architectures, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

ABD201-Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

Building your First Big Data Application on AWS

Amazon Web Services

Real Time Data Ingestion and Analysis In this session you will learn how to perform real time data analytics on streaming data using Amazon Kinesis Streams and run prediction algorithms. Learn how to stream your Cloud Trail logs to Amazon Kinesis Streams and identify anomalies using Spark Stream analytics and Amazon Kinesis Data Analytics. Ganesh Raja, Big Data Solutions Architect, Amazon Web Services

Real Time Data Ingestion & Analysis - AWS Summit Sydney 2018

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

Amazon Web Services

Data Analytics Week at the San Francisco Loft Data Warehouses and Data Lakes Organizations use reports, dashboards, and analytics tools to extract insights from their data, monitor performance, and support decision making. To support these tools, data must be collected and prepared for use. We'll look at two approaches: a structured centralized data repository as a Data Warehouse the less-structured repository of a Data Lake. We'll compare these approaches, examine the services that support each, and explore how they work together. Speakers: John Mallory - Principal Business Development Manager, Storage, AWS Hemant Borole - Sr. Big Data Consultant, AWS

Data Warehouses & Data Lakes: Data Analytics Week SF

Amazon Web Services

Analyzing Streams

Amazon Web Services

Business intelligence is often described as a set of methodologies and technologies that transform raw data into meaningful and useful information for business purposes. But this simple description hides many technical challenges IT teams struggle with. This session will show how to build business intelligence applications leveraging AWS, from the raw data import, consumption and storage down to the information production. We will also cover best practices for services such as Amazon Redshift or Amazon RDS, and how to use applications such as SAP Hana, Jaspersoft and others.

B3 - Business intelligence apps on aws

Amazon Web Services

Similar a Cloud World Forum: Large Scale Data Analysis on AWS (20)

2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo

Workshop part2 – Big Data

Journey Through the AWS Cloud - Big Data Analysis

Large Scale Data Analysis with AWS

Build Data Lakes and Analytics on AWS

AWS re:Invent Recap from AWS User Group UK meetup #8

Real-Time Streaming Data on AWS

Analyzing Streaming Data in Real-time with Amazon Kinesis

ABD301-Analyzing Streaming Data in Real Time with Amazon Kinesis

Driving Business Insights with a Modern Data Architecture AWS Summit SG 2017

Big Data Architectural Patterns and Best Practices on AWS

Best Practices for Distributed Machine Learning and Predictive Analytics Usin...

ABD201-Big Data Architectural Patterns and Best Practices on AWS

Building your First Big Data Application on AWS

Real Time Data Ingestion & Analysis - AWS Summit Sydney 2018

Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

Data Warehouses & Data Lakes: Data Analytics Week SF

Analyzing Streams

B3 - Business intelligence apps on aws

Más de Ian Massingham

Some thoughts on measuring the impact of developer relations

Ian Massingham

Leeds IoT Meetup - Nov 2017

Ian Massingham

What's New & What's Next from AWS?

Ian Massingham

DevTalks Romania - Getting Started with AWS Lambda & the Serverless Cloud

Ian Massingham

Getting started with AWS Lambda and the Serverless Cloud

Ian Massingham

AWS AWSome Day - Getting Started Best Practices

Ian Massingham

AWS IoT Workshop Keynote

Ian Massingham

Security Best Practices: AWS AWSome Day Management Track

Ian Massingham

AWS re:Invent 2016 Day 2 Keynote re:Cap

Ian Massingham

AWS re:Invent 2016 Day 1 Keynote re:Cap

Ian Massingham

Getting Started with AWS Lambda & Serverless Cloud

Ian Massingham

Building Better IoT Applications without Servers

Ian Massingham

AWS AWSome Day Roadshow

Ian Massingham

AWS AWSome Day Roadshow Intro

Ian Massingham

Hashiconf AWS Lambda Breakout

Ian Massingham

Getting started with AWS IoT on Raspberry Pi

Ian Massingham

AWSome Day Dublin Intro & Closing Slides

Ian Massingham

GOTO Stockholm - AWS Lambda - Logic in the cloud without a back-end

Ian Massingham

What's New at AWS Update for AWS User Groups

Ian Massingham

Advanced Security Masterclass - Tel Aviv Loft

Ian Massingham

Más de Ian Massingham (20)

Some thoughts on measuring the impact of developer relations

Leeds IoT Meetup - Nov 2017

What's New & What's Next from AWS?

DevTalks Romania - Getting Started with AWS Lambda & the Serverless Cloud

Getting started with AWS Lambda and the Serverless Cloud

AWS AWSome Day - Getting Started Best Practices

AWS IoT Workshop Keynote

Security Best Practices: AWS AWSome Day Management Track

AWS re:Invent 2016 Day 2 Keynote re:Cap

AWS re:Invent 2016 Day 1 Keynote re:Cap

Getting Started with AWS Lambda & Serverless Cloud

Building Better IoT Applications without Servers

AWS AWSome Day Roadshow

AWS AWSome Day Roadshow Intro

Hashiconf AWS Lambda Breakout

Getting started with AWS IoT on Raspberry Pi

AWSome Day Dublin Intro & Closing Slides

GOTO Stockholm - AWS Lambda - Logic in the cloud without a back-end

What's New at AWS Update for AWS User Groups

Advanced Security Masterclass - Tel Aviv Loft

Último

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

[BuildWithAI] Introduction to Gemini.pdf

Sandro Moreira

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Manulife - Insurer Transformation Award 2024

The Digital Insurer

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Orbitshub

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

When you’re building (micro)services, you have lots of framework options. Spring Boot is no doubt a popular choice. But there’s more! Take Quarkus, a framework that’s considered the rising star for Kubernetes-native Java. It always depends on what's best for your situation, but how to choose the best solution if you're comparing 2 frameworks? Both Spring Boot and Quarkus have their positives and negatives. Let us compare the two by live coding a couple of common use cases in Spring Boot and Quarkus. After this talk, you’ll be ready to get started with Quarkus yourself, and know when to select Quarkus or Spring Boot.

Spring Boot vs Quarkus the ultimate battle - DevoxxUK

Jago de Vreede

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Angeliki Cooney has spent over twenty years at the forefront of the life sciences industry, working out of Wynantskill, NY. She is highly regarded for her dedication to advancing the development and accessibility of innovative treatments for chronic diseases, rare disorders, and cancer. Her professional journey has centered on strategic consulting for biopharmaceutical companies, facilitating digital transformation, enhancing omnichannel engagement, and refining strategic commercial practices. Angeliki's innovative contributions include pioneering several software-as-a-service (SaaS) products for the life sciences sector, earning her three patents. As the Senior Vice President of Life Sciences at Avenga, Angeliki orchestrated the firm's strategic entry into the U.S. market. Avenga, a renowned digital engineering and consulting firm, partners with significant entities in the pharmaceutical and biotechnology fields. Her leadership was instrumental in expanding Avenga's client base and establishing its presence in the competitive U.S. market.

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Angeliki Cooney

DBX First Quarter 2024 Investor Presentation

Dropbox

Exploring Multimodal Embeddings with Milvus

Zilliz

Cloud World Forum: Large Scale Data Analysis on AWS

1. ianmas@amazon.com @IanMmmm LARGE SCALE DATA ANALYSIS WITH AWS    Ian Massingham – Technical Evangelist

2. THE MORE DATA YOU COLLECT THE MORE VALUE YOU CAN DERIVE FROM IT!

5. THE COST OF DATA GENERATION IS FALLING!

6. We are constantly producing more data

7. From all types of industries

9. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!

10. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Lower cost, higher throughput

11. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Lower cost, higher throughput Highly constrained

12. + ELASTIC AND HIGHLY SCALABLE + NO UPFRONT CAPITAL EXPENSE + ONLY PAY FOR WHAT YOU USE + AVAILABLE ON-DEMAND = REMOVE CONSTRAINTS

13. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!

14. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! AWS Import / Export AWS Direct Connect

15. Inbound data transfer is free Multipart upload to S3 Physical media AWS Direct Connect

16. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Amazon S3, Amazon Glacier, Amazon DynamoDB, Amazon RDS, Amazon Redshift, AWS Storage Gateway, Data on Amazon EC2

17. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Amazon EC2 Amazon Elastic MapReduce

18. AMAZON EC2  ELASTIC COMPUTE CLOUD!

19.

20.

21.

22.

23.

24. 3 HOURS  FOR $4828.85/hr!

25. Instead of $20+ MILLIONS in infrastructure!

26.

27. SIMULATING  205,000 COMPOUNDS!

28.

29. 18 HOURS  FOR $1833.33/hr!

30. Instead of $68+ MILLIONS in infrastructure!

31. GPU INSTANCES" " G2" CG1 1x NVIDIA Kepler GK104  8 vCPU (Intel Xeon E5-2670) 2x NVIDIA Fermi M2050  16 vCPU (Intel Xeon X5570) $0.65/h $2.10/h

32. ON A SINGLE INSTANCE COMPUTE TIME: 4h COST: 4h x $2.1 = $8.4

33. ON MULTIPLE INSTANCES COMPUTE TIME: 1h COST: 1h x 4 x $2.1 = $8.4

34.

35.

36.

37.

38. AMAZON ELASTIC MAPREDUCE  HADOOP AS A SERVICE!

39. •  SPLITS DATA INTO PIECES •  LETS PROCESSING OCCUR •  GATHERS THE RESULTS!

40. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2

41. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!

42. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! BATCH PROCESSING

43. GENERATE ➔ ➔ SHARE! STREAM PROCESSING

44. AMAZON KINESIS  REAL-TIME DATA STREAM PROCESSING!

45. Real-time response to content in semi-structured data streams    Relatively simple computations on data (aggregates, ﬁlters, sliding window, etc.)

46. Hourly server logs: how your systems went wrong an hour ago Weekly / Monthly Bill: What you spent this past billing cycle Daily customer report from your website: tells you what deal or ad to try next time Daily fraud reports: tells you if there was fraud yesterday Daily business reports: tells me how customers used AWS services yesterday Real-time metrics: what just went wrong now Real-time spending alerts/caps: guaranteeing you can’t overspend Real-time analysis: what to offer the current customer now Real-time detection: blocks fraudulent use now Fast ETL into Amazon Redshift: how are customers using services now

47. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE!

48. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE! Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2 Amazon EC2 Amazon Elastic MapReduce Amazon S3, Amazon Glacier, Amazon DynamoDB, Amazon RDS, Amazon Redshift, AWS Storage Gateway, Data on Amazon EC2 AWS Import / Export AWS Direct Connect

49. GENERATE ➔ ➔ SHARE! STREAM PROCESSING

50. GENERATE ➔ ➔ SHARE! STREAM PROCESSING Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2 Amazon Kinesis Stream Processing on Amazon EC2

51. WANT TO KNOW MORE? aws.amazon.com/solutions/case-studies/big-data/!

52.

53.

54. ianmas@amazon.com @IanMmmm THANK YOU    Ian Massingham – Technical Evangelist

55.

56. Further References Atomic Fiction Case-Study Video! https://www.youtube.com/watch?v=ljHo1_5sWxo! Slideshare with full details on the Schrodinger Materials Science case-study! http://www.slideshare.net/insideHPC/cycle-computing-recordbreaking-peta-scale-hpc-run! Real-time Streaming and Querying with Amazon Kinesis and Amazon EMR Video! https://www.youtube.com/watch?v=NIa33ZwFa8E! ! !

Cloud World Forum: Large Scale Data Analysis on AWS

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Cloud World Forum: Large Scale Data Analysis on AWS

Similar a Cloud World Forum: Large Scale Data Analysis on AWS (20)

Más de Ian Massingham

Más de Ian Massingham (20)

Último

Último (20)

Cloud World Forum: Large Scale Data Analysis on AWS