Apidays New York 2024 - The value of a flexible API Management solution for O...
[Rakuten TechConf2014] [Fukuoka] Case Study of Financial Web Systems Development and Operations with WebLogic 12c
1. Case Study of Financial Web System
Development and Operations
with Oracle WebLogic 12c
Hirofumi Iwasaki
Financial Services Department, Development Unit,
Rakuten, Inc.
2. 2
Speaker Biography
Hirofumi Iwasaki
– Group Manager, Technology Manager
– Financial Service Department, Development Unit,
Rakuten, Inc.
Carrier
– Planning, designing & implementation of huge enterprise systems for financial,
manufacturing and public systems with enterprise middleware, especially Java EE
& .NET in Japan for about 16 years.
Opus, Lectures, etc.
– Conferences: JavaOne 2014, Oracle OpenWorld 2014, Java Day Tokyo 2014,
JJUG CCC Spring (2014), WebLogic roundtable (2012-2013), Rakuten Tech
Conference (2013) etc.
– Magazine: @IT (2005-2010), CIO Magazine (2009), IT Architect (2005-2009),
Web+DB Press (2005), Java World (2001-2004), etc.
3. 3
Agenda
1. Java EE with WebLogic and Exadata for Our
Financial Systems Overview
2. Starting with GlassFish,
Migrated to WebLogic
3. Hard Business Operations, with the Power of
the WebLogic and Exadata
4. 4
Agenda
1. Java EE with WebLogic and Exadata for Our
Financial Systems Overview
2. Starting with GlassFish,
Migrated to WebLogic
3. Hard Business Operations, with the Power of
the WebLogic and Exadata
8. 1997 2014
8
Internet Services
(1997)
Online Hotel
Reservation
Internet
Research
(2000) (2001) (2005)
(2003)
(2006)
(2007)
EC in
France
in USA
(2010)
EC
In Thailand
Internet
Banking
(2009)
(2004)
(2008)
(2008)
EC in
UK
(2011)
EC
In Austria
In Indonesia
EC in
Malaysia
in Brazil
(2005)
(2009)
(2010)
Internet Finance
In Germany
Global Video
streaming Global Video
(2012)
In Spain
(2013)
(2013)
EC
In Singapore
EC
In Japan
Online Books,
DVD Store
Pay-per-view
Video Service
Portal Site
Performance
Marketing
In USA
Internet
Auction
EC
in Taiwan
Global eBook
Streaming
Global Social
Messaging App
(2014)
Online Security
Brokerage
Credit Card E-money
Rakuten Life
Insurance
IP Telephony
Professional
Baseball
Marriage
Consultant
(2014)
Credit Card Payment
By Smartphone
(2012)
Point Service
Phone Service
(2013)
Online Golf
Reservation
Others
Energy Service
Real Café Service
Curation Service
Expanding Business Portfolio
9. 9
5 Financial Services of Rakuten Group in Japan
Life
Insurance
Credit Card
E-Money
Bank
Financial
Securities
10. 10
5 Financial Services of Rakuten Group in Japan
Big 5 Services
Each has Huge Transactions
24/7 Non-Stop Services
Life
Insurance
Credit Card
E-Money
Bank
Financial
Securities
12. 12
3 Big Issues of the Rakuten Card Systems
Credit Card
1. Outdated !
2. Complicated !
3. Difficult!
13. 13
3 Big Targets for New Architecture
Sustaina
bility
Require
ments
Flexibility
System
Transpar
ency
14. 14
Why We Chose the Java EE ?
Portability
Results of
Financial Sys
Vendor
Support
Community
Excellent Excellent Excellent Excellent
Nothing Good Not Bad Mediocre
15. 15
Why We Chose WebLogic 12c
Product Productivity Reliability Scalability Support Price Operation Development
WebLogic
Server 12c
Excellent Excellent Excellent Excellent Not Good Excellent Excellent
Commercial
Product A
Excellent Excellent Excellent Excellent Not Good Not Good Not Good
Commercial
Product B
Excellent Excellent Not Good Not Good Not Good Not Good Excellent
16. 16
Why We Chose the Oracle Exadata?
Product Productivity Data
reliability
Scalability HA PCI DSS Operation
Oracle
Exadata
Excellent Excellent Excellent Excellent Excellent Excellent
Product X - Not Good - - - Not Good
Product Y Excellent Excellent - Excellent Not Good Not Good
17. 17
PCI DSS Certification Requirements
We were supposed to be certified with PCI DSS, the card payment
industry data security standard. Exadata is the solution.
OS audit DBA audit Standard audit Fine grain audit
Audit
target
Instance start and
stop, connect with
admin or listener
DB operation with
admin user
DB operation with
login, object operation
with DDL/DML, data
reference, etc.
CRUD for specific
data
Output OS file, listener log OS file OS file,
(DBA_AUDIT_TRAIL
view in Oracle)
User definition table,
(DBA_FGA_AUDIT_
TRAIL view in
Oracle)
Audit log Time, OS info, DB
instance, action, auth
info, exit code
Time, DB user, action,
auth info, OS user, exit
code
Time, user, action, OS
user, terminal name,
query, etc.
time, DB user, OS
user, accessed
object name, fine
grain audit policy
name, query
19. 19
1. Policies: Case of Rakuten
Internal Development First,
no outsourcing to external SI vendors. (Group All)
– Financial businesses are also the target for
the application of this policy.
Educate NO ORDER
&
Develop
Rare Case for
Financial Systems in Japan
In-House
Development
External Vendors
20. 20
2. Education: Read, Read, Read!
RECOMMENDED
for WebLogic 12c
Good & Only
Japanese
EE 6 book
Start from HERE
4th Edition
Good Pocket
Reference!
For NetBeans 7
with EE 6
21. 21
2. Education: Online Materials
Original Tutorial
for Newbies (Start here!)
NetBeans Java EE docs
for Advanced Information
22. 22
2. Education: Simplify to Learn
Old Architecture
New Architecture
Too difficult to learn many
non-standard old technologies
Simple & Easy!
23. 23
3. Architecture: Apply EE 6 Specs
Rich Clients
(no business logics)
Call
Web Presentation
(no business logics)
Business Logic
(no presentations)
Data Access
JPA
EJB
CMT
JSF
DBs
Container
Managed
automatic
Transaction
Java FX JTA
Messaging
JMS MQ
Connection
RMI-IIOP
Other
Servers
EMail
MTA
JAX
JavaMail
Call
Call
Call
Call
Call
There's no
rich client
24. 24
3. Architecture: Migrate from Old
Front-End (Apache)
Front-End
(WebLogic)
External
Service
Back-End
(WebLogic)
Old App Architecture
Back-End
Database
View
PHP
Action
with
Business
Logic
Web
Service
API
Service
Data
Service
External
Services
DMZ
(Apache)
Static
HTML,
Images,
CSS
View
Facelet
External
Services
Exadata
Backing
Bean
(no
business
logic)
Business
Logic
Entity
External
DAO
Entity
Transaction
Boundary
Transaction
Boundary
BEGIN
COMMIT
WebLogic
Plug-In
BEGIN
COMMIT
New App Architecture
25. 25
3. Architecture: Simplified
Core
L7 Balancer
Front Real-time
Web Site A
Web Site B
Internal Site
Front Batch
Reverse Proxy
Batch Exec
Services (aka APIs)
System B
Gateway
Database
System C
Sub Proc
26. Local WebLogic Server instance
Code and Test with Fast-swap
26
4. Environment: Ease of Dev.
Centralized DEV DB
= X X
27. 27
4. Environment: Easy Startup
2. Download
Code from
Repository
3. Install JDK, IDE,
App servers -> Build -> Run on the local terminal
1. Join a
project.
4. Refer JIRA tickets
for tasks
28. 28
5. Test: Full Auto Testing &Validation.
1. Auto PULL
Management Server
2. Auto
Build
& Test
3. Auto
4. Report Validate
Hourly
ZERO Violations
29. 29
Agenda
1. Java EE with WebLogic and Exadata for Our
Financial Systems Overview
2. Starting with GlassFish,
Migrated to WebLogic
3. Hard Business Operations, with the Power of
the WebLogic and Exadata
30. 30
In Mid 2011, We Didn’t Have WebLogic 12c Yet
Chart of the mid 2011 Java EE app servers
Vendor App Server EE 5 Servers EE 6 Servers
Open Source GlassFish 2.1.1 3.1.1
Oracle WebLogic 10.3 -
IBM WebSphere 7.0 8.0
Red Hat JBoss 6.0 7.0 (partially)
We wanted to apply Java EE 6 for our new system, but not released.
We decided to use GlassFish 3.1.1 until the EE 6 applied WebLogic
(12c) released.
31. 31
Impact of the Migrating within the Project
Non-Interchangeable Development Code
GlassFish WebLogic
– Different container behaviors.
– Non Java EE, different each server special APIs.
Scheduled Impacts for Migrating WebLogic Configurations
– Cluster configurations for high availabilities.
– Other setting adjustments.
– Bug checking and applying patches.
– Connecting Oracle Enterprise Manager.
32. 32
Investigation of the Differences
WebLogic GlassFish
Code Base BEA WebLogic 6.0
based + Improvements
Felix OSGi modular
based kernel
Web Container WebLogic Original Tomcat Container
EJB Container WebLogic Original GlassFish Original
Remote Invocation T3, RMI-IIOP, SOAP RMI-IIOP, SOAP
Transaction Processing WebLogic Original GlassFish Original
Persistence Container WebLogic Original TopLink Based
Runtime JRockit, Oracle JDK Oracle JDK
33. 33
Schedule for Migrating from GlassFish to WebLogic
Java EE 6
Development
with GlassFish
Migrating to
WebLogic
(12c)
Development Operations
Java EE Development
with WebLogic (12c)
Production
Release
Configuration of Java EE 6 applied
WebLogic (12c) and
Enterprise Manager (12c)
(Dec, 2011)
Production
Release
Finally the
new WL was
released at
the end of 2011
34. 34
Non-Interchangeable Point: 1. Container Initialization
Single WAR including JSF and EJBs
– GlassFish
1. EJB initialize (@Startup)
2. JSF (Servlet) initialize (HttpServlet#init())
– WebLogic
1. JSF (Servlet) initialize (HttpServlet#init())
2. EJB initialize (@Startup)
Inverse initialization pattern
– Affected for the server initializations.
– Absorbed with the wrapper classes
35. 35
Non-Interchangeable Point: 2. Remote Invocation
Different EJB remote invocation operations. Wrapped for absorbing.
GlassFish (5 properties, no security)
Properties prop = new Properties();
prop.setProperty(Context.INITIAL_CONTEXT_FACTORY,
“com.sun.enterprise.naming.SerialInitContextFactory”);
prop.setProperty(Context.URL_PKG_PREFXIES,
“com.sun.enterprise.naming”);
prop.setProperty(Context.STATE_FACTORIES
“com.sun.corba.ee.impl.presentation.rmi.JNDIStateFacto
ryImpl”
prop.setProperty("org.omg.CORBA.ORBInitialHost",
“theservername”);
prop.setProperty("org.omg.CORBA.ORBInitialPort",
“3700”);
Context context = new InitialContext(prop);
ARemote remote
= (ARemote) context.lookup(“java:global/…”);
WebLogic (4 properties, with security)
Properties prop = new Properties();
prop.setProperty(Context.INITIAL_CONTEXT_FACTORY,
“weblogic.jndi.WLInitialContextFactory”);
prop.setProperty(Context.PROVIDER_URL,
“t3://theservername:7001”);
prop.setProperty(Context.SECRITY_PRINCIPAL,
“weblogic”);
prop.setProperty(Context.SECURITY_CREDENTIALS,
“thepassword”);
Context context = new InitialContext(prop);
ARemote remote
= (ARemote) context.lookup(“java:global/…”);
36. 36
Agenda
1. Java EE with WebLogic and Exadata for Our
Financial Systems Overview
2. Starting with GlassFish,
Migrated to WebLogic
3. Hard Business Operations, with the Power of
the WebLogic and Exadata
37. 37
Atomic Database Scaling
Old Database New Scaling model (Exadata)
・・・・・・
RT group Batch group
active-active
cluster to avoid
single-point of
failure
Non-stop
failover
Parallel
operation for
high
performance
Stand-by
Real-Time
Batch
SAN
Active
Fibre
Channel
switch
(1~8Gb/s)
InfiniBand
switch
(40Gb/s)
Real-Time
Batch
Batch traffic
adversely
affects online
performance
Single point of
failure for non-stop
service
MySQL
limitation for
update
transaction
performance
Shared storage
limitation with
another
service's bad
performance
affects
5 minutes
in fail over
Divide online /
batch servers
High
performance
networking
Independent
storage for 24
Hrs / 365 days
performance
guarantee
Storage
Scale-out
enabled
architecture
Not scalable
architecture
×
38. 38
Single Database, Single Schema Strategy
Exadata
X3-2
Web Area
Replication
(APB)
Merged to
Single Exadata
INTRA Area
Ultra-huge financial
online transactions
with ACID props.
MS SQL Server
39. 39
Migration of Application
Front-End (Apache)
Front-End
(WebLogic)
External
Service
Back-End
(WebLogic)
Old App Architecture
Back-End
Database
View
PHP
Action
with
Business
Logic
Web
Service
API
Service
Data
Service
External
Services
DMZ
(Apache)
Static
HTML,
Images,
CSS
View
Facelet
External
Services
Exadata
Backing
Bean
(no
business
logic)
Business
Logic
Entity
External
DAO
Entity
Transaction
Boundary
Transaction
Boundary
BEGIN
COMMIT
WebLogic
Plug-In
BEGIN
COMMIT
New App Architecture
40. 40
Fast Deployment Operations
WebLogic Server
Single
WAR
for API
WebLogic
Managed Server
Real-Time
Batch
For Management
WebLogic Management Console
Same WAR, for
different servers.
Automatic
multi server
deploying
41. 41
Non-stop “Production Redeployment”
WebLogic Server
Auto versioning,
Non-stop redeployment
WebLogic New Module
Request
Dispatcher
Old Module
Requests
WebLogic
Managed
Server
WAR
Automatic multi
versioning, parallel
operation
Old modules will be un-deployed
gracefully when all
old sessions are invalidated.
42. 42
Managing Servers by Oracle Enterprise Manager (EM)
Introduced
Oracle Enterprise
Manager
Easy to Find
Performance &
Status
43. 43
Our Requests for WebLogic and Exadata
For WebLogic,
– Appliance of latest Java EE specs ASAP!!
We know the WebLogic is the basement of the Oracle Fusion
Middleware, but we want the latest EE for our products.
Yes, we’re waiting the next WebLogic version supporting JEE7.
For Exadata
– Make patches easy to operate
Huge costs for updating quarterly update patch.
Complicated procedures for non-stop upgrading.
Hoping for the next generation updating technology.
Hello. Let’s get started.
Let me share about the result of our financial systems using Java EE 6.
This is Hirofumi Iwasaki speaking.
I'm a financial system group manager of Rakuten.
And a professional of enterprise financial system management, planning and development.
Today’s agenda. Firstly, overview of our renewed systems.
Secondly, about the development process using GlassFish and WebLogic.
The last is the operation process using the WebLogic and Exadata.
Started from the overview.
The Rakuten group has many services around the world.
And we’re the Japan team for financial services groups.
And the rapidly expanding worldwide from 2010. 14 countries for e-commerce, 28 countries fro all serves.
The Rakuten Group Consolidated Transaction Volume worldwide
In details, the Rakuten group was started from 1997, and expanding internet finance business as shown.
We have 5 big services of financial in Japan.
Rakuten Card, Rakuten Bank, Rakuten Edy, Rakuten Security, Rakuten Life, Rakuten Insurance.
Each service has huge financial transactions.
Additionally, the systems require the 24/7, non-stop services.
These requirements are tons of heavy implementation points for stable operations.
Let's dig a little deeper. The big 3 requirement of Rakuten financial systems.
First, rapid changeable logics, Second, huge request expansion capacity, and last, transactional.
Very hard requirements for systems.
And our credit card company, the Rakuten Card’s systems were very serious situation. 3 big issues.
Firstly, outdated. Used very special old technologies, and met the EOL. We cannot fix anymore.
Secondly, complicated. One action with many system relations and bucket relays.
Thirdly, difficult to change. There are many systems, files, and databases, and tightly related, mutual dependencies.
We decided to change them all.
We have 3 big targets for the new architecture.
Sustainability, flexibility, and system transparency.
These are for the concrete, long life cycle of the financial system
The platform comparison of our future systems. We selected Java EE and dot NET framework.
Portability. Of course Java EE is excellent. Actually, .NET is poor environment limitation, only for Windows except for MONO project.
Result of financial systems. Both are good, but Java EE is excellent for its long running results of the world.
Vendor support. Java EE systems are supported by many platform vendors.
And the excellent community. We decided to chose the Java EE for our next systems.
Next is platform. We selected 3 major commercial platform for the next systems.
Finally we chose the WebLogic Server, because of its stability and huge result for the financial systems.
Especially we focused the ease of development and operations. The key factors are the “Fast Swap” and “Production Redeployment” functions of WebLogic, the most suitable for our system development and operations.
The last was the database. We chose Oracle Exadata for our new systems.
Exadata is the engineered systems of Oracle, with Oracle Database and Real Application Cluster set.
We focused the scalability, PCI DSS and ease of operations, and lead to this selection.
Actually, the general financial systems require the transactional operations with ACID property for its data keeping, and the Exadata was the best solutions for our credit systems.
As a reliable financial services, we must be certified the PCI DSS.
PCI DSS means the data security standard, so we must think the concrete repository of our credit data.
And we recognize that the Exadata is the most secure relational database for our usage.
Next, architecture policies.
I planned the Java EE to real financial systems, with these 5 big issues.
1st, policies, 2nd, education, 3rd, architecture, 4th, environment, and 5th, test.
1st, policies. In the case of Rakuten, we have a policy, "internal development first".
Of course, financial systems also. No basic policies to throw external vendors.
We must clear this core policies, and consider the next solutions to run.
2nd, educational issues. Read, read, and read.
There's many good Java EE books in book stores, but English book only, not Japanese.
Fortunately, we already changed our standard language to English, and many programmer can read them.
And there are many useful articles in world wide web.
Thanks to the NetBeans team, nice tutorial are still in the web site.
Refer to this site if you want to start Java EE development.
In old architecture, we must learn many non-standard old technologies to develop.
These causes many resource management issues & high operational costs. In new architecture, we simplified to Java EE 6.
3rd, I designed to apply Java EE 6 specs to the new architecture.
Basic structure is obeyed to standard architecture, and applied front-end to JSF.
And due to the no rich client requirement, I skipped JavaFX spec.
We had some older systems to integrate to the new architecture.
In the PHP case, we designed each from PHP business logics to the EJB API codes.
Full rewriting, 100% API-nized for re-use & collaborating services in the future.
We also re-designed the application module blocks. Center API-nized logics, with many front-ends.
All business logic designed as API, with SOAP, REST, and IIOP protocol access
enabled for future service-oriented architectures to simplify.
For the ease of development, we adopt the new IDE,
NetBeans 7 with Apache maven automatic building systems.
And we build full local programming environment to easy coding & run for rapid programming.
We made the easy startup environment to reduce startup costs.
If some programmer attends the project,
2. just download from git server,
3. install tools,
and 4. refer to JIRA for his or her for today's task management.
And to educate the accurate programming manners,
we introduced Jenkins auto-building server with static security analyzers, Sonar & VeraCode.
And we achieved zero violations before the new system release.
In the mid of 2011, we decided to apply the WebLogic server for our systems, but we didn’t have EE 6 applied one yet.
And there was the GlassFish 3.1.1 was already released as a EE 6 reference implementation.
So we decided to use GlassFish 3.1.1, until the next WebLogic 12c released.
As the migrating from GlassFish to WebLogic, there is the differences.
First, different container behaviors, and non Java EE, different each server special APIs.
So we scheduled impact for migrating as shown.
Investigation of the differences. Almost all basement is different.
The schedule. We started our system development with GlassFish, and migrated to WebLogic 12c on December 2011, the 12c unveiled time.
And started the configuration of the WebLogic Enterprise Manager 12c.
We had some older systems to integrate to the new architecture.
In the PHP case, we designed each from PHP business logics to the EJB API codes.
Full rewriting, 100% API-nized for re-use & collaborating services in the future.