SlideShare una empresa de Scribd logo
1 de 19
Descargar para leer sin conexión
Dmitri Nesteruk
dmitrinesteruk@gmail.com
A publishes an event
B subscribes to the event
If lifetime of B < lifetime of A
We can get a memory leak

Weak event pattern
Library for composing async/event-based
programs
Event subscription wrapped as IDisposable
Lets you manually unsubscribe
Unsubscription automatic when source runs out
of data to supply
Declarative subscriptions
Pub/sub mechanism

Objects not aware of each other
Typically built on top of a DI container
Each component references the broker
Erlang
F# MailboxProcessor
Axum (dead)
Task Parallel Library (TPL) DataFlow
A library approach 
Fine-grained
Data-level (SIMD) – not in .NET
Autoparallelization

Coarse-grained
Using threads
Weird synchronization primitives (e.g. Pulse&Wait)
TPL DataFlow
A directed graph of data
Message passing
Buffering
Parallel execution
Dependency management
Pipeline
read

process

Network
read

process

write

process
collate

prepare
process

write
Input

Propagate

Calculate
indices

Output
Render to
screen
Save to DB

Read stock
market data
Filter

Save to DB
ISourceBlock<T>
Source of data; read

ITargetBlock<T>
Where you write the data to

IPropagatorBlock<TIn,TOut>
Propagator; performs data processing/filtering

As with Rx, ready-made implementations
provided
TransformBlock
Transforms data
Also: TransformManyBlock

ActionBlock
Processes the data
Post()
Sends a piece of data to the block for processing

LinkTo ()
Links a block with the next block in the graph
Takes an optional predicate
Returns an Idisposable

CancelationTokenSource if you change your
mind
Basic block processing
BatchBlock<T>
Combines sets of blocks into an array

JoinBlock<2> and <3>
Collects 2 or 3 inputs into a Tuple<2/3>

BatchJoinBlock
Merger of the above
Collects Tuple<IList<T>,…>
Provides a data storage buffer
Can be used synchronously
block.Post()
block.Receive()

or asynchronously
await block.SendAsync()
Await block.ReceiveAsync()
Grouping Blocks
Not a competitor to Rx or TPL itself
Course-grained parallelization/sequencing
mechanism
‘Cheap’ multithreading
Questions?
Конференция по .NET технологиям
4 апреля 2014, ПетроКонгресс
http://dotnext.ru
Предлагайте доклады 

Más contenido relacionado

Similar a Tpl DataFlow

Tshark pen testing, very good insight of the pent test
Tshark pen testing, very good insight of the pent testTshark pen testing, very good insight of the pent test
Tshark pen testing, very good insight of the pent test
claudiu59
 
Lec 49 - stream-files
Lec 49 - stream-filesLec 49 - stream-files
Lec 49 - stream-files
Princess Sam
 

Similar a Tpl DataFlow (20)

Streams of RDF Events Derive2015
Streams of RDF Events Derive2015Streams of RDF Events Derive2015
Streams of RDF Events Derive2015
 
Realtime Detection of DDOS attacks using Apache Spark and MLLib
Realtime Detection of DDOS attacks using Apache Spark and MLLibRealtime Detection of DDOS attacks using Apache Spark and MLLib
Realtime Detection of DDOS attacks using Apache Spark and MLLib
 
Razorback slides-1.1
Razorback slides-1.1Razorback slides-1.1
Razorback slides-1.1
 
Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4
 
Getting Data into Splunk
Getting Data into SplunkGetting Data into Splunk
Getting Data into Splunk
 
Tshark pen testing, very good insight of the pent test
Tshark pen testing, very good insight of the pent testTshark pen testing, very good insight of the pent test
Tshark pen testing, very good insight of the pent test
 
Flink Forward SF 2017: Srikanth Satya & Tom Kaitchuck - Pravega: Storage Rei...
Flink Forward SF 2017: Srikanth Satya & Tom Kaitchuck -  Pravega: Storage Rei...Flink Forward SF 2017: Srikanth Satya & Tom Kaitchuck -  Pravega: Storage Rei...
Flink Forward SF 2017: Srikanth Satya & Tom Kaitchuck - Pravega: Storage Rei...
 
ADMS'13 High-Performance Holistic XML Twig Filtering Using GPUs
ADMS'13  High-Performance Holistic XML Twig Filtering Using GPUsADMS'13  High-Performance Holistic XML Twig Filtering Using GPUs
ADMS'13 High-Performance Holistic XML Twig Filtering Using GPUs
 
Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...
Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...
Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...
 
Splunk Stream - Einblicke in Netzwerk Traffic
Splunk Stream - Einblicke in Netzwerk TrafficSplunk Stream - Einblicke in Netzwerk Traffic
Splunk Stream - Einblicke in Netzwerk Traffic
 
Elk stack @inbot
Elk stack @inbotElk stack @inbot
Elk stack @inbot
 
Why You Definitely Don’t Want to Build Your Own Time Series Database
Why You Definitely Don’t Want to Build Your Own Time Series DatabaseWhy You Definitely Don’t Want to Build Your Own Time Series Database
Why You Definitely Don’t Want to Build Your Own Time Series Database
 
Microsoft Dryad
Microsoft DryadMicrosoft Dryad
Microsoft Dryad
 
Open source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysisOpen source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysis
 
Fluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker containerFluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker container
 
The basics of fluentd
The basics of fluentdThe basics of fluentd
The basics of fluentd
 
First Responders Course - Session 6 - Detection Systems [2004]
First Responders Course - Session 6 - Detection Systems [2004]First Responders Course - Session 6 - Detection Systems [2004]
First Responders Course - Session 6 - Detection Systems [2004]
 
Seattle spark-meetup-032317
Seattle spark-meetup-032317Seattle spark-meetup-032317
Seattle spark-meetup-032317
 
from source to solution - building a system for event-oriented data
from source to solution - building a system for event-oriented datafrom source to solution - building a system for event-oriented data
from source to solution - building a system for event-oriented data
 
Lec 49 - stream-files
Lec 49 - stream-filesLec 49 - stream-files
Lec 49 - stream-files
 

Más de Dmitri Nesteruk

Converting Managed Languages to C++
Converting Managed Languages to C++Converting Managed Languages to C++
Converting Managed Languages to C++
Dmitri Nesteruk
 
Dynamics CRM Data Integration
Dynamics CRM Data IntegrationDynamics CRM Data Integration
Dynamics CRM Data Integration
Dmitri Nesteruk
 
ReSharper Architecture & Extensions
ReSharper Architecture & ExtensionsReSharper Architecture & Extensions
ReSharper Architecture & Extensions
Dmitri Nesteruk
 

Más de Dmitri Nesteruk (20)

Good Ideas in Programming Languages
Good Ideas in Programming LanguagesGood Ideas in Programming Languages
Good Ideas in Programming Languages
 
Design Pattern Observations
Design Pattern ObservationsDesign Pattern Observations
Design Pattern Observations
 
CallSharp: Automatic Input/Output Matching in .NET
CallSharp: Automatic Input/Output Matching in .NETCallSharp: Automatic Input/Output Matching in .NET
CallSharp: Automatic Input/Output Matching in .NET
 
Design Patterns in Modern C++
Design Patterns in Modern C++Design Patterns in Modern C++
Design Patterns in Modern C++
 
C# Tricks
C# TricksC# Tricks
C# Tricks
 
Introduction to Programming Bots
Introduction to Programming BotsIntroduction to Programming Bots
Introduction to Programming Bots
 
Converting Managed Languages to C++
Converting Managed Languages to C++Converting Managed Languages to C++
Converting Managed Languages to C++
 
Monte Carlo C++
Monte Carlo C++Monte Carlo C++
Monte Carlo C++
 
YouTrack: Not Just an Issue Tracker
YouTrack: Not Just an Issue TrackerYouTrack: Not Just an Issue Tracker
YouTrack: Not Just an Issue Tracker
 
Проект X2C
Проект X2CПроект X2C
Проект X2C
 
Domain Transformations
Domain TransformationsDomain Transformations
Domain Transformations
 
Victor CG Erofeev - Metro UI
Victor CG Erofeev - Metro UIVictor CG Erofeev - Metro UI
Victor CG Erofeev - Metro UI
 
Developer Efficiency
Developer EfficiencyDeveloper Efficiency
Developer Efficiency
 
Distributed Development
Distributed DevelopmentDistributed Development
Distributed Development
 
Dynamics CRM Data Integration
Dynamics CRM Data IntegrationDynamics CRM Data Integration
Dynamics CRM Data Integration
 
ReSharper Presentation for NUGs
ReSharper Presentation for NUGsReSharper Presentation for NUGs
ReSharper Presentation for NUGs
 
ReSharper Architecture & Extensions
ReSharper Architecture & ExtensionsReSharper Architecture & Extensions
ReSharper Architecture & Extensions
 
Web mining
Web miningWeb mining
Web mining
 
Data mapping tutorial
Data mapping tutorialData mapping tutorial
Data mapping tutorial
 
Reactive Extensions
Reactive ExtensionsReactive Extensions
Reactive Extensions
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Último (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 

Tpl DataFlow