A unified view across websphere datapower and mq, solace and tibco messaging

A Unified View Across WebSphere
DataPower and MQ, Solace and TIBCO
Messaging

Get a single point of control for: message auditing, tracking,
transformation, routing and
performance

P/2 EXECUTIVE SUMMARY P/2 FINANCIAL SERVICES SCENARIO P/9 THE CASE FOR MESSAGE TRACKING
P/13 PERFORMANCE AND AVAILABILITY OF AUTOPILOT P/14 CONCLUSION

© 2012 Nastel Technologies, Inc.

A Unified View Across WebSphere DataPower
and MQ, Solace and TIBCO Messaging

Get a single point of control for: message auditing, tracking, transformation, routing and performance

Executive Summary
Many companies have a pressing need to establish a single point of control over their
multi-vendor messaging infrastructures. They are frequently confronted with:

• Acquisitions that expand technology diversity and footprint
• Lack of visibility across multiple middleware platforms
• The need for self-service
• Shortage of human capital
• Continuously rising cost for support and maintenance
• Increasing urgency to determine the root-cause of problems

The complexity of one management system per messaging infrastructure is too costly and when
these infrastructures are interconnected, the management systems become ineffective. These
firms require one place to monitor, visualize and analyze the status of applications and services
shared across the enterprise environment. However, the skill set required to manage each
middleware technology is often different and a single business flow can traverse multiple
different transports and support teams. This makes unified monitoring of messages across the
infrastructure and the ability to track them and rapidly identify and correct problems anywhere
in that flow an essential element in improving service and reducing cost—perhaps, even a
necessity for survival

From the business side there is pressure to adjust to the realities of our economy, reduce
operational and capital expenditures — essentially, do more for less. In addition there is a strong
trend toward outsourcing support functions or moving them off shore which puts pressure on
operating procedures. Economic pressure on enterprise organizations to consolidate has resulted
in significant merger and acquisition activity, where companies are forced to use the IT assets
and applications inherited through these consolidations. However, those inherited applications
do not use the same interfaces—essentially, they do not speak the same language as those
already in use at the acquiring organization. Additionally, these pressures have accelerated the
move to the private cloud, where enterprises hope to save additional costs.

In many of these enterprises, appliances such as DataPower and Solace are used to augment
traditional messaging middleware. In addition, tools such as an enterprise service bus or SOA
appliances are used as “universal translators” to transform the content from one service to the
other and route traffic between them. Much of that traffic comes from Message Oriented
Middleware (MOM) as it has been accepted as the most common way of linking systems in
enterprise environments. The combination of multiple brands of traditional middleware,
appliances and an enterprise service bus creates a great deal of complexity to contend with.
While the flexibility and power gained from this approach can be essential, it remains a
significant challenge to manage this complexity without a single point of control.

P/2



A Financial Services Scenario:
ONE UNIFIED GLOBAL POINT OF CONTROL
One of Nastel’s clients, a global banking conglomerate, had the challenge of monitoring and
managing the WebSphere MQ family of products, DataPower, Solace, and TIBCO EMS and
Rendezvous (RV) as well as a homegrown messaging solution.

Historically, their applications ran on WebSphere MQ. However, when the bank acquired another
bank with it’s own IT infrastructure using TIBCO EMS and Rendezvous rather than WebSphere MQ
it realized that it had a requirement to integrate the disparate systems. Many of the new
systems acquired leveraged TIBCO EMS and Rendezvous for the middleware layer. The bank
decided to leverage DataPower appliances to act as “the bridge” to transform and route the
content of messages coming from applications using WebSphere MQ to the format that their
acquired banks’ applications using TIBCO EMS required and then route them to TIBCO.

In order to monitor the performance of the messaging infrastructure end-to-end, they needed
visibility for their custom applications, WebSphere DataPower, Solace Message and Content
Routers, WebSphere MQ and TIBCO EMS and RV from a single monitoring tool. The bank realized
that vendor monitoring products do extend across technology boundaries and their home grown
tools can no longer keep up with their technology evolution. They needed better tooling to
maintain productivity and keep the costs associated with problem resolution at acceptable
levels.

The IT team began searching for a global monitoring solution that unified the common
operational procedures that existing across the multiple technologies, supported them
operationally; including run-time operations, WebSphere MQ administration, rapid problem
detection and isolation and operational review and planning. But it also had to support the in-
creasing interoperability between WebSphere DataPower, Solace, WebSphere MQ and TIBCO.
Plus, they felt it was time to begin message tracking through JMS, WebSphere MQ and Data-
Power.

FIRST STEP: EXPANDING MONITORING AND MANAGEMENT CAPABILITIES
The bank’s IT department had been using a well-known vendor’s event monitoring product to
monitor WebSphere MQ. They replaced this with Nastel AutoPilot® to monitor and manage
WebSphere MQ on distributed and z/OS platforms as AutoPilot gave them the ability to watch the
homegrown message broker and internal business applications as well. They also used TIBCO
Hawk to monitor the TIBCO EMS system. Business requirements soon dictated that integration
across their environment was necessary. But at this point, they had two separate management
systems.

As the bank moved to integrated their messaging systems they decided to do likewise with their
systems for middleware monitoring. They placed responsibility for all middleware under one
team chartered with integrating front-end and back-end applications. It soon became impossible P/3



for this new middleware team to meet the performance service level agreements mandated
by the business.

They were running mission-critical applications that relied on IBM and TIBCO products,
including a high-volume trading application, but couldn’t handle the demands of a full
production workload, which is millions of messages per second over hundreds of queue
managers. In addition they did not have integrated monitoring for DataPower or Solace
appliances. Having to deal with multiple monitoring solutions along with a need for a third
or fourth prevented them from meeting performance standards. The group determined that
they needed a single monitoring solution that met the following requirements:

• Provides real-time visibility across DataPower, Solace, TIBCO, WebSphere MQ and their home-
grown middleware solution
• Finds stuck messages
• Delivers early notification of faults, bottlenecks and slowdowns
• Monitors the performance of messaging and compares to SLAs
• Provides administrative control over WebSphere MQ
• Can provide message tracking through JMS, WebSphere MQ and DataPower
• Easily integrates with the bank's enterprise management systems

AUTOPILOT: End of the Hawk
Nastel’s AutoPilot was already effectively monitoring the bank’s WebSphere MQ and
homegrown MOM systems. However, while the bank trusted TIBCO, it became desirable to
integrate the TIBCO messaging products into the same monitoring framework.

In large enterprise environments like this one, with many interdependent applications
constantly in flux, the speed and ease of adapting to change was a key factor in the
decision process. To configure TIBCO Hawk to monitor a new EMS server or Rendezvous
daemon would typically take the team considerable effort. With AutoPilot, it takes minutes.
With the size of the bank’s environment, AutoPilot emerged as the obvious choice. As an
added benefit and cost saving, training for team members could be limited to just a single
monitoring environment.

The middleware team decided to standardize all application performance monitoring under
Nastel’s AutoPilot. The middleware team felt that AutoPilot would allow them to:

• Unify the monitoring and management procedures across all middleware technologies
• Create a single portal for administration and certification of user permissions and access
• Create valuable rules that can be applied to all middleware transports for end-to-end monitoring

P/4



Figure 1: This screenshot displays AutoPilot’s real-time monitoring for the Solace appliance.

NASTEL’S MONITORING SOLUTION for the Solace Message Appliance
Nastel’s solution for monitoring the Solace messaging appliances was welcomed by the bank as it
was an integrated component within its unified middleware monitoring solution and not a
standalone tool. AutoPilot for Solace can monitor multiple appliances harvesting their events
and analyzing them in correlation with other middleware and system events in the AutoPilot
Complex Event Processing engine. In addition it combines monitoring of software components
of the Solace environment with the hardware inventory and assets (blades, power modules,
fans) to provide a complete monitoring solution for appliance-based technology

The collected analysis includes:
• Appliance specifications
• Appliance Incoming/outgoing data statistics
• Cache and cache-level statistics
• Router clients and queues
• Hardware properties and specifications
• List of VPNs and VPN-level statistics P/5
• Current/max messages spooled



Figure 2: AutoPilot Expert for DataPower: In the situation above it has detected a configuration
error. AutoPilot can create robust sensors which monitor multiple factors within DataPower before
triggering alerts and warnings and offers a single monitoring interface across all of a customer’s
appliances.

Nastel’s DataPower Monitoring Solution
The bank decided to use WebSphere DataPower appliances for message transformation and
routing between the different middleware messaging systems. AutoPilot's ability to provide deep
operational monitoring for DataPower and provide a unified view across all their middleware
messaging solutions was important to the bank.

The AutoPilot Expert for DataPower analyzes:
• DataPower's consumption of system resources over time
• DataPower metrics
• DataPower’s performance of key business activities
• Routing and transformation history
• Correlate DataPower performance data with events from other systems

NASTEL’S TIBCO EMS MONITORING SOLUTION
A major driver of the bank’s decision to standardize on AutoPilot was its ability to monitor all of
its middleware systems. With AutoPilot’s TIBCO EMS plug-in expert, the middleware team can
easily monitor TIBCO EMS servers and components including: queues, topics, consumers,
producers from a single, consolidated vantage point. There is no limit to the number of TIBCO
EMS servers they can simultaneously monitor.

P/6



The TIBCO EMS Plug-in expert is modularized based
on the bank’s EMS servers and their components, for
example: queue expert, topic expert, consumer and
producer experts. The bank’s specific requirements
for performance monitoring activities will
determine how the experts are deployed -- they can
be deployed individually or in any combination of
multiple deployments.

In addition, the TIBCO EMS expert automatically
measures and collects EMS performance metrics. The
expert periodically samples information about TIBCO
EMS servers and their components based on any
interval defined by the bank’s middleware team.
Some of the key performance metrics they can
collect include: inbound, outbound, acknowledged
and pending message volumes, message rates per
second, average response times, message tracking
and age of oldest messages.
Figure 3: TIBCO EMS Metrics Collected by
These metrics are collected for servers, queues, AutoPilot’s TIBCO EMS Experts
routes, groups, consumers, producers, transports,
durables and events. AutoPilot’s powerful Complex Event Processing (CEP) engine uses these
metrics in policies the middleware team defines for prediction, correlation, validation,
automation, notifications, logging and alerts.

AutoPilot’s CEP engine can also combine metrics and data from many diverse systems within
the bank’s infrastructure. One of the most important features for the bank is that the
AutoPilot TIBCO/EMS plug-in monitors multiple TIBCO EMS servers and the AutoPilot CEP
engine relates the EMS metrics with metrics from other infrastructure components such as
DataPower, Solace, WebSphere MQ, Message Broker, ESB and other supported technologies
to dynamically create events and alerts as needed.

P/7



NASTEL’S TIBCO RV MONITORING SOLUTION
An additional requirement the bank had for AutoPilot was to provide performance
monitoring for TIBCO RV and provide a service view of TIBCO multicast services. Using
AutoPilot’s TIBCO RV plug-in expert, the middleware team can monitor TIBCO RV
performance statistics via the plug-in expert’s automatic collection of data about the TIBCO
RV server and its components such as services, routers and transports. The expert collects
TIBRV statistics by subscribing to and processing the advisory messages.

The TIBCO RV expert analyzes the collected data providing performance analysis for TIBCO
RV Server Info, Client Transport, Service, Router Info and Security Info. It was important to
the bank that this was functional in both agent and agent free deployments. The metrics
that the expert provides come from the RV server and this data is used in AutoPilot in
business views for validation, automation, notification, logging and alerts.

A Business View showing the health of the RV Server environment can display RV status
including: Server health state, inbound and outbound Multicast lost messages, inbound and
outbound Point-to-Point lost messages, reports number of clients using a particular service,
reports number of hosts using a particular service, rate of inbound and outbound traffic,
amount of inbound and outbound data and the number of missed inbound messages.

Figure 4: : TIBCO RV Health Business View

P/8



THE CASE FOR MESSAGE TRACKING
Messaging middleware has transformed the way applications communicate and exchange
information. Application integration has brought about significant advantages to the
development of composite applications; however, it has also brought about a need to track
messages, end-to-end through JMS, WebSphere MQ, DataPower, brokers and other middleware.
As IT complexity grows, compliance and integration needs evolve and so does the need for
tracking the exchange between IT services and their corresponding business services. Most
professionals who support critical business applications have dealt with the problematic
consequences of performance degradation or messages that have not made it to their
destination. These problems are difficult to resolve and take countless hours to determine the
root-cause.

Figure 5: AutoPilot tracks WMQ transaction compliance including: number of transactions, in-flight trans-
actions, violations, violation rate, failures, failure rate, maximum and average duration.

P/9



Figure 6: This screenshot shows the results of AutoPilot’s automated discovery
of WebSphere MQ and Message Broker, the bank’s applications and their WMQ
transactions. In real-time AutoPilot tracks the flow of WebSphere MQ messages
and monitors their performance and availability.

P/10



Figure 7: This screenshot shows an alternative view of WebSphere MQ
transactions. In this case we are comparing discovered transactions to customer
specified milestones or progress through a business process

P/11



Real-time, message tracking provides visibility into message flows enabling organizations to
prioritize problems appropriately and work on the issues that have the most impact to the
business, such as: “Where is my message?”, “Where is my order?”, “Are orders flowing as
expected?” Many of these “problems” may turn out to be false alerts. These false alerts can
often be suppressed using AutoPilot CEP which can also provide proactive notification about
disruption in message flows and stuck or hung messages.

Next Step: GIVING THE MIDDLEWARE TEAM MESSAGE TRACKING
The middleware team needed message tracking in order to more rapidly resolve the issues their
newer applications were facing. However, these teams also used middleware monitoring and
administration and required a unified approach to managing all of this.

The team took advantage of AutoPilot’s Message tracking capabilities which provided the
following benefits:

• Real-time, tracking of JMS, WMQ and DataPower middleware message flows (transactions)
• “Dope and Trace” and “Recognize and Trace” technologies are used with unique Transaction IDs for
stitching and inferencing of the message stream
• Instantly detect delayed, compromised or incomplete JMS, WMQ and DataPower middleware message
flows (transactions)
• Provide notification and automated actions based on user-defined criteria
• Can use middleware payload data to stitch messages together based on business policy
• Analyzes WMQ transaction throughput and message volume in real-time
• Stores all of the performance-related information in an easily accessible SQL database
• Provides an audit trail for all transactions and messages
• Generates reports about WMQ transaction activity on a daily, weekly, monthly basis
• Policy-driven monitoring and automated notification
• Identifies the root cause of performance bottlenecks

P/12



PERFORMANCE AND SCALABILITY OF THE AUTOPILOT
SOLUTION
Horizontal scalability:
On the agent level: ability to monitor a large number of applications across thousands of
servers

On the server level: AutoPilot’s Virtual Complex Event Processing engine (CEP) engine can
span multiple servers utilizing its built-in grid

On the user level: AutoPilot can handle thousands of users

Vertical scalability:
Ability to process large volume of data from one source (one server that may have multiple
applications). For example: situations where one Queue manager had more than 20
thousands queues and 10K channels, which translated to hundred of thousands of data
metrics.

Transactional metrics:
The performance impact on applications tested with JSpec with an impact of less than 2.5%

Input Speed:
The capability to process consistently high volumes of KPIs (key performance indicators) and
tens of thousands of transactions per second, per CEP instance. Because CEP engines can be
added on-demand, processing capacity can be elastically increased as needed.

Rules processing speed:
The single CEP engine running on 64 bit quad CPU server with 4 GB of memory can process
2M rules per second. Because CEP is a virtual machine it can scale up linearly. When adding
an additional CEP engine, the throughput can double.

High Availability:
The GRID-based infrastructure provides a smooth failover.

Support for monitoring a wide range of messaging middleware platforms:
Including: WebSphere DataPower, WebSphere MQ, WebSphere ESB, Message Broker, WBI,
TIBCO RV and EMS, Solace Systems Messaging Appliances, CICS and JMS.

P/13



CONCLUSION:
A POWERFUL APPLICATION PERFORMANCE MONITORING SOLUTION FOR LARGE
ENTERPRISES
AutoPilot has proven to be a highly effective, comprehensive and proactive solution for the
bank’s complex, heterogeneous middleware environment. It is the only monitoring solution
that supports traditional software-based messaging technology as well as appliance-based
technology. It also enables the same operating procedures to be applied for all messaging
products. Since standardizing on AutoPilot, the middleware team has real-time, end-to-end
visibility and is now able to pinpoint problem areas and also the source, or root cause, of a
problem anywhere across its various middleware systems.

AutoPilot’s Complex Event Processing engine automatically eliminates false alerts, identifies
potential problems and provides early warning in order to avoid impacting users and disrupt-
ing business processes. These capabilities dramatically reduce the mean time to problem
resolution, increases overall uptime and reduces user support requirements by cutting down
significantly on calls from business users to the middleware team. It is also the only moni-
toring solution that unifies user “permissioning” and certification of access across the
middleware estate to ensure regulatory compliance is met.

Real-time, visibility from message tracking lets organizations not only measure how well
their business services perform at any given moment, but also optimize them end-to-end.
The most striking outcome is that it enhances an organization’s responsiveness by focusing
them at the right time and the right place. This can be clearly demonstrated when tracking
messages across the infrastructure and can pin-point where performance bottlenecks are
and why, so that organizations, which have previously embarked on a wild goose chase, can
focus on problem resolution with precision.

As a result, AutoPilot is now the foundation that enables the monitoring, automation,
visualization and fault recovery for all of the bank’s multi-vendor middleware, SOA and
messaging appliances and brokers end-to-end. AutoPilot lets the middleware team keep
their entire infrastructure governed, visible and manageable from a single pane of glass.

P/14

A unified view across websphere datapower and mq, solace and tibco messaging

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a A unified view across websphere datapower and mq, solace and tibco messaging

Similar a A unified view across websphere datapower and mq, solace and tibco messaging (20)

Más de jKool

Más de jKool (11)

Último

Último (20)

A unified view across websphere datapower and mq, solace and tibco messaging