Trading Show Chicago Non-functional testing for trading systems 19 May 2016 Iosif Itkin, Exactpro CEO & Co-Founder London Stock Exchange Group

1. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com1
Non-functional testing
for trading systems
19 May 2016
Iosif Itkin, Exactpro CEO & Co-Founder
London Stock Exchange Group
Open Access Quality Assurance & Related Software Development for Financial Markets
Tel: +7 495 640 2460, +1 415 830 38 49
www.exactpro.com

2. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com2
London Stock Exchange Group
Trading 23.3%
Clearing 33.8%
Information 36.5%
Technology 5.7%

3. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com3
London Stock Exchange Group
• A leading diversified international exchange and infrastructure group with assets across the entire exchange value chain
• The Group can trace its history back to 1698
• Headquartered in the United Kingdom, with significant global operations, the Group employs approximately 5,500 people
• Post trade and risk management services are a significant part of the Group’s business operations
• Through FTSE Russell, the Group is a global leader in financial indexing, benchmarking and analytic services with
approximately $10 trillion benchmarked to its indexes

4. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com4
Open Access Software Testing
Exactpro is:
• A specialist firm operating an open access model with focus on
functional and operational testing of securities data distribution,
trading systems, risk management, market surveillance and post-
trade infrastructures.
• A UK company with operations in the US and four QA &
software development centres in Russia
• Part of London Stock Exchange Group as of May 29, 2015
• Incorporated in 2009 with 10 people, our company has
experienced significant growth as satisfied clients require
more services; now employing over 350 specialists
Clients Worldwide:

5. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com5
Trading Systems Types
Proprietary
Trading & HFT
Brokerage
Execution
Venue

6. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com6
Performance Testing Cost Structure
System
Under Test
Production
Connectivity
Test
Tools
Infrastructure:
≈ 10 - 100 servers
infrastructure:
thousands of servers
infrastructure:
1 - 4 servers

7. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com7
The Idea behind Efficient Load Generation

8. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com8
The Idea behind Efficient Load Generation

9. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com9
Latency Measurement

10. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com10
Latency Distribution

11. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com11
Question Everything
Do not Trust:
• Test Tools
• Average Metrics and Percentiles
• Load Shape is Right
• System Requirements
• Anything

12. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com12
Keep Calm
Have No Fear:
• Annihilate the System Under Test
• Repeat the Same Tests Many Times
• Load Levels that Exceed Presumed Production
• Make no Assumptions About Extrapolating Test Results
into More Scalable Environments

13. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com13
Two Load Generator Types
1.
2.

14. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com14
Achieving Diverse and Relevant Load
1. Random load “Pace Maker” 2. Variety of Passive Liquidity
3. “Agressors” try to interact with
the market to increase the number
of test cases happening in the
environment
4. Analyse the situation in retrospective. Make
sure that what happened is correct. Tick
checkboxes against tests that were actually
executed
Updated
test
library

15. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com15
Non-Functional Test Types

16. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com16
Use Passive Testing Tools to
Collect Audit Trails

17. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com17
Questions & Answers
https://ru.linkedin.com/in/iosifitkin http://extentconf.com