Exploring the Future Potential of AI-Enabled Smartphone Processors
Social Stream Analysis Use Cases
1. Big
Brother
for
Enterprises:
Social
Media
Stream
Use
Cases
Samisa
Abeysinghe
VP
Developer
Evangelism
26
Feb
2014
2. About
the
Presenter
๏
๏
2
Samisa
Abeysinghe
VP
Developer
Evangelism
samisa@wso2.com
Samisa
Abeysinghe,
Vice
President
of
Developer
Evangelism
joined
the
company
in
September
2005.
Prior
to
the
current
role,
Samisa
used
to
be
VP
of
Engineering
and
managed
the
development
of
WSO2
Carbon
based
product
plaOorm.
3. About
WSO2
๏
๏
๏
Global
enterprise,
founded
in
2005
by
acknowledged
leaders
in
XML,
web
services
technologies,
standards
and
open
source
Provides
only
open
source
plaOorm-‐as-‐a-‐service
for
private,
public
and
hybrid
cloud
deployments
๏
๏
3
All
WSO2
products
are
100%
open
source
and
released
under
the
Apache
License
Version
2.0.
Is
an
AcXve
Member
of
OASIS,
Cloud
Security
Alliance,
OSGi
Alliance,
AMQP
Working
Group,
OpenID
FoundaXon
and
W3C.
Driven
by
InnovaXon
๏
Launched
first
open
source
API
Management
soluXon
in
2012
๏
Launched
App
Factory
in
2Q
2013
๏
Launched
Enterprise
Store
and
first
open
source
Mobile
soluXon
in
4Q
2013
5. NSA like Monitoring for
Your Enterprise
๏ Analyze volumes of data
๏ Address correlation complexities in analytics
๏ Off line vs Real time operations implications
๏ Some operations got to be in real time, else the value is lost
๏ Summarized data over time (and other) dimensions for analytics
A “Big Brother” that keeps an eye on
the whole enterprise
5
6. Why should I bother?
๏ Deal with high volume (terra bites) of information
๏ In order to make decisions
๏ Real time & Offline
๏ Take action
6
7. WSO2 :ONLY COMPLETE
& INTEGRATED Platform
๏ Complete and integrated for
๏ Data capture
๏ Analysis: both real-time and batch
๏ Visualization
๏ Action taking business process execution
Making data driven intelligence for
your enterprise easy
7
16. Why
Monitor
Social
Media
?
● To
get
useful
insight
about
the
market
○ CompeXtor
Analysis
○ Customer
Trend
and
Interest
● IdenXfy
and
Compare
Trends
○ Monitoring
the
brand
○ Comparing
the
trends
of
the
public
interest
16
17. WSO2
Business
AcXvity
Monitor
(BAM)
● Can
capture
data
from
any
applicaXon/service
● APIs,
Services
or
source
including
file-‐based
data
○ WSO2
has
lightweight
agents
to
provide
■ raw
data
from
numerous
sources
■ as
well
as
support
for
● gateways
(lightweight
or
heavyweight)
● Service
and
API
proxies
● Data
imports
from
various
sources
(including
file
based
data)
● Any
other
source
(extended
agents)
17
19. WSO2
Complex
Event
Processor
(CEP)
● CEP
process
mulXple
event
streams
generated
within
the
enterprise
to
construct
data
abstracXon
and
idenXfy
meaningful
paaerns
among
those
streams
○ Powered
by
WSO2
Siddhi
■
■
Processes
more
than
2.5M
events/sec
on
single
server
(commodity
hardware)
Has
a
powerful
and
extensible
SQL
Like
Siddhi
query
language
○ Integrates
with
any
enterprise
system
for
event
capture
■
Support
mulXple
transports
&
event
formats:
HTTP,
JMS
&
SMTP
on
XML,
JSON,
Text
&
Map
○ Efficient
large
volume
event
publishing
to
enterprise
systems
and
custom
noXficaXons
■
■
■
19
XML,
JSON,
Text
over
HTTP,
JMS
&
SMTP
SMS
Support
MySQL
DB
and
Cassandra
Support
21. E.g.
Usecases
● Finding
out
the
trend
of
#Cloud
Twiaer
tags
over
Xme
● IdenXfy
the
public
interest
on
cloud
market
leaders
based
on
tweets
on
#Cloud
Twiaer
tags
● Comparing
interest
trends
on
cloud
market
● NoXfying
via
Email
when
an
interesXng
tweet
was
posted.
21
26. Finding
out
the
trend
of
#Cloud
Twiaer
tags
over
Xme
๏
๏
26
insert
overwrite
table
Twi@erDataStreamStatsPerMinute
select
count(1)
as
total_tweet_count,
year(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
))
as
year,
month(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐
MM-‐dd
HH:mm:ss.SSS'
))
as
month,day(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
))
as
day,hour(from_unixXme(cast(created_at/
1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
))
as
hour,
minute(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
))
as
minute,concat(substring(from_unixXme(cast(created_at/1000
as
BIGINT),
'yyyy-‐MM-‐
dd
HH:mm:ss'),0,16),':00')
as
Xme
from
TwiaerDataStream
group
by
year(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
)),
month(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
)),day(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
)),hour(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
)),minute(from_unixXme(cast(created_at/1000
as
BIGINT),'yyyy-‐MM-‐dd
HH:mm:ss.SSS'
)),substring(from_unixXme(cast(created_at/1000
as
BIGINT),
'yyyy-‐MM-‐
dd
HH:mm:ss'),0,16);
28. IdenXfy
the
public
interest
on
cloud
market
leaders
insert
overwrite
table
CountOnCompanyTwi@erDataStream
select
'IBM'
as
company,
count(1)
as
cnt
from
TwiaerDataStream
where
instr(lower(tweet),
'ibm')
>0
;
insert
overwrite
table
CountOnCompanyTwi@erDataStream
select
'Google'
as
company,
count(1)
as
cnt
from
TwiaerDataStream
where
instr(lower(tweet),
'google')
>0
;
insert
overwrite
table
CountOnCompanyTwi@erDataStream
select
'VMware'
as
company,
count(1)
as
cnt
from
TwiaerDataStream
where
instr(lower(tweet),
'vmware')
>0
;
insert
overwrite
table
CountOnCompanyTwi@erDataStream
select
'Azure'
as
company,
count(1)
as
cnt
from
TwiaerDataStream
where
instr(lower(tweet),
'azure')
>0
;
28
