Perficient is a leading IT consulting firm that helps clients implement business-driven technology solutions. It was founded in 1997 and has over 2,000 employees located throughout North America and in global delivery centers. Perficient provides a wide range of services including custom application development, package implementation, architecture planning, and more to support solutions like enterprise portals, SOA, BI, and CRM. The presentation discusses the need for comprehensive application monitoring to prevent costly outages and errors. It covers using logging and complex event processing to monitor systems, transactions, and business events at a high level and in real-time. A demonstration of Perficient's Generic Audit Logger and Generic Exception Handler is also provided to illustrate logging and monitoring solutions

1. Advanced Logging and Analysis for
SOA, Social, Cloud and Big Data

2. About Perficient
Perficient is a leading information technology consulting firm serving clients
throughout North America.
We help clients implement business-driven technology solutions that integrate
business processes, improve worker productivity, increase customer loyalty and create
a more agile enterprise to better respond to new business opportunities.

3. Perficient Profile
 Founded in 1997
 Public, NASDAQ: PRFT
 2011 Revenue of $260 million
 Major market locations throughout North America
— Atlanta, Austin, Charlotte, Chicago, Cincinnati, Cleveland,
Columbus, Dallas, Denver, Detroit, Fairfax, Houston,
Indianapolis, Los Angeles, Minneapolis, New Orleans,
Philadelphia, San Francisco, San Jose, Southern California,
St. Louis and Toronto
 Global delivery centers in China, Europe and India
 2,000+ colleagues
 Dedicated solution practices
 87% repeat business rate
 Alliance partnerships with major technology vendors
 Multiple vendor/industry technology and growth awards

4. Our Solutions Expertise & Services
Business-Driven Solutions Perficient Services
• Enterprise Portals  End-to-End Solution Delivery
• SOA and Business Process Mgmt  IT Strategic Consulting
• Business Intelligence  IT Architecture Planning
• User-Centered Custom Applications  Business Process & Workflow
• Interactive Design Consulting
• CRM Solutions  Usability and UI Consulting
• Enterprise Performance Management  Custom Application Development
• Customer Self-Service  Offshore Development
• eCommerce & Product Information  Package Selection, Implementation
Management and Integration
• Enterprise Content Management  Architecture & Application Migrations
• Enterprise Resource Planning  Education
• Management Consulting  Interactive Design
• Industry-Specific Solutions
• Mobile Technology Perficient brings deep solutions expertise and offers a
complete set of flexible services to help clients
• Security Assessments implement business-driven IT solutions

5. Our Speakers
Eric Roch
• Principal SOA | Mobile | Cloud
• 30+ years of experience in various aspects of
Information Technology including:
• IT executive level management within industry and
consulting
• technical architecture
• application and systems development.
• He has also been an IT industry speaker and
author for many years.
Ben Hahn
• Sr. Technical Architect
• 20+ years of experience in various aspects of
Information Technology.
• Software Solutions Architect
• Enterprise Infrastructure Architect
• Product Management
• Software community contributor

6. Agenda
• Glitches and outages drive the need for
more comprehensive logging and
monitoring
• Big Data and Logging
• Events and Logging
• Logging and exception management as a
service
• Explicit Transaction Monitoring
• Instrumentation for Logging
• Logging and Exception Management
Framework and Demo
6

7. The Need for Application Monitoring
• Systems are distributed, increasingly more complex and we are progressively
more dependent upon them
• Application glitches are costly in dollar terms, customer loyalty and brand image:
– Department of Commerce study found software errors cost U.S. economy
$59.5 billion annually
– Information Week says IT downtime costs $26.5 billion in lost revenue
– The 2003 North America blackout was triggered when a local outage was
not detected by monitoring software
– Glitches in the reporting of prices on the NYSE may have caused the “flash
crash”
– Knight Capital Group Inc. lost $440 million from software trading error –
about $10M a minute for 45 minutes
– Programming error at Visa Debit Processing Services caused a billing error of
$23,148,855,308,184,500
7

8. Humans Monitoring for Glitches
Tourist follows GPS into Bay UK woman follows GPS into river
8

9. Application Monitoring
“Oh yeah, well I’m watching you watch him”
Source: E*TRADE Baby 9

10. Machine Monitoring
• Log analysis is well established for IT operations, security and
compliance
• Enterprise management software standards to detect platform and
network problems
• LogLogic appliance can ingest up to 250,000 events per second with
high speed filtering and routing capabilities
• Splunk provides general-purpose search language for analysis and
reporting for time-series data using the MapReduce framework
10

11. Finding Glitches in the Data
Source: splunk
11

12. Log Analysis vs. Business Analytics
• Ingest – Versus ETL
• Big Data – Bidirectional integration with Hadoop
• Query language – MapReduce function on unstructured data
• Drill anywhere – Investigate on all the data versus a predefined schema
or cube
• Information discovery – Discover relationships based on patterns in the
data
• Ad-hoc versus dimensional – Log analysis is not based a predefined
structure based a point-in-time set of requirements
Source: splunk Implementation 12

13. Business Events and Event Processing
• Event-driven architecture (EDA) is a
software architecture pattern
promoting the production,
detection, consumption of, and
reaction to events.
• Complex event processing (CEP)
consists in processing many events
happening across all the layers of an
organization, identifying the most
meaningful events within the event
cloud, analyzing their impact, and
taking subsequent action in real
time.
13

14. CEP High-Level Use Cases
• Situation awareness is about
"knowing" the state of the
product, person, document, or
entity of interest at any point in
time.
• Sense and respond is about
detecting some significant fact
about the product, person,
document or entity of interest,
and responding accordingly
• Track and trace is about tracking
the product, person, document or
entity of interest over time and
tracing pertinent facts
Source: TIBCO Software
14

15. CEP Architecture
Fine-grain
• Millions of raw events can CEP
Business Events
Engine
be represented in one
complex event Concept
State
• Component status (fine
grain) – outage (logical Rule
/predictive) Bases
• Events are process with Logical Events – Notifications, Consequences Actions
BPMS ESB
declarative rules and Event
Channel(s)
implicit state management
• Events drive agile business
processes
Flexible Workflows
Integration and
SOA
Business Components
System(s) of Record Business Applications
15

16. Business Event Logging with CEP
16

17. Application and Business Event Logging
• Monitoring logical transaction state vs. monitoring transaction through-put
• Exception management vs. error logging
• Explicit application event logging versus machine logging
• Application instrumentation to a framework
• Exception management and log correlation
– Transaction and conversation IDs
– User defined fields
– XML processing - XSD, XSLT and XPath
– Agents and appenders
– Time and dependencies
17 17

18. Logging and Exception Management Design Goals
• Highly configurable and user friendly GUI
• High-speed
• Non-blocking asynchronous calls
• Open architecture / standards support
• Technology agnostic
• Service-oriented
18

19. What is GAL?
Generic Audit Logger
• A configurable message logger to
record, view or reply message
• Fully configurable via a GUI
• Map logs to custom fields
• Filter and query logs
• Supports message query from GEH
• Works standalone and supports our
exception handler
19

20. GAL DEMO

21. What is GEH?
• Generic Exception Handling
• Provides
– Centralized Exception Handling
– Process/Workflow Exception Resolution
– Issue Resolution Knowledge Base
– Centralized Audit Trail
– Centralized Transaction Monitor
• Benefits
– Standardized Issue Resolution
– Minimal Project Based Exception Handling
– Dynamic Views / Dynamic Workflows
– Faster Learning Curve
– External to Internal Exception Mappings
21

22. GEH DEMO

23. Transaction Monitoring via Logging
• Leveraging GAL and GEH to define a transaction
Using recognition rules, each log entry can define a
• Transaction demarcation – begin or end
• Transaction context – using a conversation identifier
• Transactional checkpoints – define completed stages in a transaction.
• Transactional errors
Once a transaction is defined we get
• Monitoring
• Instrumentation
• Error tracking via an error handler like GEH
• State and through-put
23

24. Transaction Capturing via Logs
24

25. Events via Transactions
Now that we have transactions we can also define events
• Each log entry can now also define an event
• Events can be aggregated to make more logical events
• Events can also be generated for “non-events” e.g. if number of
transactions do not reach a certain threshold (sales quotas)
25

26. Transaction Event Generation
And eventually published out to a CEP
26

27. TRANSACTION MONITOR
DEMO

28. Q&A