Algorithmic Trading has changed the world the way the Traders trade and Trade Support supports. There is a Brave New World happening with the "hands on" Trading evolving into "hands off" Algo Trading. Not all trades need to be made in ultra low latency timing. Future trading will rely on a broader set of data which will be mined for relevance. For example, an important series of XBRL Financial Reporting events are happening throughout the world and especially in the USA. A critical mass of financial data will be ready for mining which will be a boon for transparent "low touch" fundamental style algorithmic trading.
2. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Financial Data Mining with Algorithmic Trading
Robert Golan
DBmind Technologies, Inc.
Please Note: This is the view of DBmind only which may not
pertain to DBmind’s Client Views
3. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Financial Data Mining with Algorithmic Trading
Algorithmic Trading has changed the world the way the
Traders trade and Trade Support supports. There is a
Brave New World happening with the "hands on" Trading
evolving into "hands off" Algo Trading. Not all trades need
to be made in ultra low latency timing. Future trading will
rely on a broader set of data which will be mined for
relevance. An important series of XBRL Financial
Reporting events are happening throughout the world and
especially in the USA. A critical mass of financial data will be ready for mining which will be a boon for
transparent "low touch" fundamental style algorithmic trading. Also, "low touch" trading such as program trading & direct market access (DMA)
will evolve into advanced Algo Trading strategies. Stock and economic indicators combined with XBRL will add value for Algo Trading. This is
about a well thought out strategic high latency trading strategy with data mining discovering the governing rules while adding the expert rules
with validation. Yes, the trader is still the key to making this all happen. Both fundamental and technical trading rules need to be combined with
the expert rules, the data mined rules, and most importantly the regulatory environment rules. RegNMS in the USA and MiFID in Europe have
indirectly helped the adoption of electronic trading and it is important to integrate the GRC related rules in an agile way. Agility is the key and
thus the rules need to be placed into a rules engine and managed by the experts for proper compliance, risk management, and governance.
Japan, China, and the Netherlands with regards to XBRL are ready to be data mined with Algo Trading now. A XBRL US survey is indicating at
least 340 of the estimated 500 public companies that the SEC requires to begin filing in XBRL format in June 2009, have already converted
their financial statements into XBRL. XBRL US, is the non-profit XML standard setter that developed and maintains the US GAAP taxonomy
used by filers to comply with the SEC mandate. Almost $7 trillion in market capitalization will be represented by this XBRL financial data which
is over 50% of the total market cap for all publicly traded companies reporting to the SEC. As this XBRL Financial Data ripens, a wonderful
harvest awaits us data miners which will enhance the current Algo Trading strategies which use this data.
4. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Algorithmic trading
http://en.wikipedia.org/wiki/Algorithmic_trading
• In electronic financial markets, algorithmic trading, also known as algo, automated, black
box, or robo trading, is the use of computer programs for entering trading orders with the
computer algorithm deciding on certain aspects of the order such as the timing, price, or type
(market vs. limit, or buy vs. sell) of the order. It is widely used by pension funds, mutual
funds, and other institutional traders to divide up a large trade into several smaller trades in
order to avoid market impact costs or otherwise reduce transaction costs. It is also used by
hedge funds and similar traders to make the decision to initiate orders based on information
that is received electronically, before human traders are even aware of the information.
• Algorithmic trading may be used in any market strategy, including market making,
intermarket spreading, arbitrage, or pure speculation (including trend following) to make the
complete decision on entering trades and electronically executing the trade with no human
intervention, other than in writing the computer program.
• In 2006 at the London Stock Exchange, over 40% of all orders were entered by algo traders,
with 60% predicted for 2007. American markets and equity markets generally have a higher
proportion of algo trades than other markets, and estimates for 2008 range as high as an
80% proportion in some markets. Foreign exchange markets also have active algo trading
(about 25% of orders in 2006). Futures and options markets are considered to be fairly easily
integrated into algorithmic trading, and bond markets are moving toward more access to
algorithmic traders.
5. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Citadel Investment Group
Interactive Brokers
Credit Suisse
Deutsche Bank
Goldman Sachs
Lehman Bros.
Morgan Stanley
Susquehanna Investment Group
UBS
Some of the Algo Players
11. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Statistical Arbitrage in the U.S. Equities Market
Marco Avellaneda† and Jeong-Hyun Lee
First draft: July 11, 2008
This version: June 15, 2009
Abstract
We study model-driven statistical arbitrage in U.S. equities. The trading signals are generated in two
ways: using Principal Component Analysis and using sector ETFs. In both cases, we consider the
residuals, or idiosyncratic components of stock returns, and model them as mean-reverting
processes.
This leads naturally to “contrarian” trading signals.The main contribution of the paper is the construction, back-testing
and comparison of market-neutral PCA- and ETF- based strategies applied to the broad universe of U.S. stocks. Back-testing shows that, after
accounting for transaction costs, PCA-based strategies have an average annual Sharpe ratio of 1.44 over the period 1997 to 2007, with
much stronger performances prior to 2003. During 2003-2007, the averageSharpe ratio of PCA-based strategies was only 0.9. Strategies based
on ETFs achieved a Sharpe ratio of 1.1 from 1997 to 2007, experiencing a similar degradation after 2002. We also introduce a method to
account for daily trading volume information in the signals (which is akin to using “trading time” as opposed to calendar time), and observe
significant improvement in performance in the case of ETF-based signals. ETF strategies which use volume information
achieve a Sharpe ratio of 1.51 from 2003 to 2007. The paper also relates the performance of mean-reversion statistical arbitrage strategies with
the stock market cycle. In particular, we study in detail the performance of the strategies during the liquidity crisis of the
summer of 2007. We obtain results which are consistent with Khandani and Lo (2007) and validate their “unwinding” theory for the quant fund
drawdown of August 2007.
Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, N.Y. 10012
USA
†Finance Concepts, 49-51 Avenue Victor-Hugo, 75116 Paris, France.
12. Copyright ® 2009, SAS Institute Inc. All right s reserved.
XBRL Basics
XBRL is XML
It is Extensible
There is an XBRL specification – tells you how to
use XBRL
Hinges on taxonomies – the dictionary of terms
for business reporting – which includes financial
statements
13. Copyright ® 2009, SAS Institute Inc. All right s reserved.
XBRL Taxonomy
Created by XBRL
Consortium
Consumed
Rendered
XBRL
Creation
XBRL Document
Created by Preparer
TAGGING
14. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Validation
Standardization
Calculation
Cash = Currency +
Deposits
Calculation
Cash = Currency +
Deposits
Formulas
Cash ≥ 0
Formulas
Cash ≥ 0
Contexts
US $
FY2004
Budgeted
Contexts
US $
FY2004
Budgeted
Label
cashCashEquivalentsAn
dShortTermInvestments
Label
cashCashEquivalentsAn
dShortTermInvestments
References
GAAP I.2.(a)
Instructions
Ad Hoc disclosures
References
GAAP I.2.(a)
Instructions
Ad Hoc disclosures
Presentation
Cash & Cash Equivalents
Presentation
Cash & Cash Equivalents
XBRL
Item
XBRL
Item
XML
Item
XML
Item
XBRL
Item
XBRL
Item
Presentation
Comptant et Comptant
Equivalents
Presentation
Comptant et Comptant
Equivalents
Presentation
Geld & Geld nahe Mittel
Presentation
Geld & Geld nahe Mittel
Presentation
Kas en Geldmiddelen
Presentation
Kas en Geldmiddelen
Presentation
现金与现金等价物
Presentation
现金与现金等价物
Presentation
現金及び現金等価物
Presentation
現金及び現金等価物
Presentation
Деньги и их эквиваленты
Presentation
Деньги и их эквиваленты
Presentation
Гроші та їх еквіваленти
Presentation
Гроші та їх еквіваленти
15. Copyright ® 2009, SAS Institute Inc. All right s reserved.
The Business Reporting Supply Chain
External
Financial
Reporting
Business
Operations
Internal
Financial
Reporting
Investment,
Lending, and
Regulation
Processes
Participants
Auditors
Trading
Partners
Investors
Financial
Publishers
and Data
Aggregators
Regulators
Software VendorsSoftware Vendors
Management
Accountants
Companies
XBRL XBRLXBRL
XBRL
Financial StatementsXBRL-GL
The
Journal
Standard
Transaction
Standards
Collaboration is
KEY!!!
16. Copyright ® 2009, SAS Institute Inc. All right s reserved.
USUS
UKUK
JPJPESES
SESE
CNCN
ZAZA
AUAU
DEDEDKDK
Financial
Banking Regulators Pilot
Pilot Committe
d
Committe
d
KRKR
SGSG
FRFR
NZNZ
NLNL
LuXLuX
Portuga
l
Portuga
l
BEBE
EU CEBSEU CEBS
17. Copyright ® 2009, SAS Institute Inc. All right s reserved.
XBRL Jurisdictions
UK
CA
SPUS
AU
NZ
IR
JP
KR
BE
VZ
CO
BR
AR
PT
RU
SG
HK
NO
SE
PL
FI
IT
CN
IN
LB
CZ
UA
LU
IAS
B
AE
NL
TR
GR
MT
CH
FR
SI
HU
AT
Established
Jurisdictions
Provisional
Jurisdictions
in Construction &
in Project
DE
DK
ZA
20. Copyright ® 2009, SAS Institute Inc. All right s reserved.
CACA UKUK
IEIE
AUAU
NONO
JPJP
NZNZ
NLNL DEDE
CNCN
Tax Authorities Pilot
Pilot Committe
d
Committe
d
Tax XML Technical Committee
recommends use of XBRL
(Oasis-OECD)
29 Tax authorities
21. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Exchanges
& Equity Regulators
Sao Paulo
Sao Paulo
NZSE
NZSE
ASX
ASXJohannesburg
Johannesburg
Shenzen
Shenzen
EuroNext
EuroNext
KOSDAQ
KOSDAQ
Tokyo
Tokyo
Singapore
Singapore
SWX
SWXLux
Lux
Pilot LiveEval
TSX
TSX
OBX
OBX
LSE
LSE
CSE
CSE
Deutsche
Börse
Deutsche
Börse
Taipei
SEC
SEC
Kor
ea
Kor
ea
Shanghai
Shanghai
22. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Grant Boyd, CA
gboyd@aicpa.org
Technical Manager – XBRL, AICPA
•http://www.icgfm.org/XBRLPresentations.htm
23. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Data Mining Approaches
Three Basic Approaches to Data Mining
• Mathematical-based methods,
• Distance-based methods, and
• Logic-based methods
Methods may use supervised or unsupervised variable
• Supervised – induction rules for predefined
classifications
• Unsupervised – rules and classifications determined by
data mining method
24. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Mathematical-based Methods
Neural Network
• Network of nodes modeled after a neuron or neural circuit
• Supervised learning
• Weighted values at different nodes
• Mimics the processing of the human brain
• Form of Artificial Intelligence
25. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Mathematical-based Methods
Discriminant Analysis
• Similar to multiple regression analysis uses a non-
continuous dependent variable
• Approach identifies the variables (features or cases)
that best explain the classification
• Supervisory learning approach
• Loses effectiveness with large complex data sets
26. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Logic-Based Approach
Tree and Rule Induction
• Supervised Learning
− Uses an algorithm to induce a decision tree from a file of
individual cases
− Case has set of attributes and the class to which it belongs
• Decision tree can be converted to a rule-based view.
• Major advantage is ability to communicate and understand
information derived from this approach.
• Prior research addressed audit areas of:
− bankruptcy, bank failure, and credit risk
27. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Distance-Based Method
Clustering
• Data mining approach that partitions large sets of data
objects into homogeneous groups
• Uses unsupervised classification where little manual
pre-screening of data is necessary –
− useful in situations where there is no predefined
knowledge of categories
• Classifications based on an object’s attributes
• Most commonly used in field of marketing but could be
used in auditing
28. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Selecting Data Mining Approach
Criteria:
• Scalability - how well data mining method works
regardless of data set size
• Accuracy - how well information extracted remains
stable and constant beyond the boundaries of the data
from which it was extracted, or trained
• Robustness - how well the data mining method works
in a wide variety of domains
• Interpretability - how well data mining method provides
understandable information and valuable insight to user
29. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Continuous Auditing, XBRL and Data Mining
Presenters:
Jennifer Moore, Lumsden & McCormick, LLP
Karina Barton, Canisius College
Dr. Joseph O’Donnell, Canisius College
New York State Society of Certified Public Accountants
Technology Assurance Committee
June 15, 2004
30. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Prime Services
http://en.wikipedia.org/wiki/Prime_brokeragehttp://en.wikipedia.org/wiki/Prime_brokerage
• Prime Brokerage is the generic name for a bundled package of
services offered by investment banks to hedge funds. The business
advantage to a hedge fund of using a Prime Broker is that the Prime
Broker provides a centralized securities clearing facility for the hedge
fund, and the hedge fund's collateral requirements are netted across
all deals handled by the Prime Broker. The Prime Broker benefits by
earning fees ("spreads") on financing the client's long and short cash
and security positions, and by charging, in some cases, fees for
clearing and/or other services.
• The following "core services" are typically bundled into the Prime
Brokerage package:
− Global custody (including clearing, custody, and asset servicing)
− Securities lending
− Financing (to facilitate leverage of client assets)
− Customized Technology (provide hedge fund managers with
portfolio reporting needed to effectively manage money)
− Operational Support (prime brokers act as a hedge fund's primary
operations contact with all other broker dealers)
31. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Hedge Fund
Hedge Fund
Hedge Fund
Exchange Clearing Settlement
Trading
Position
keeping
Clearing
And
Settlement
Prime
Services
32. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Hedge Fund
Mutual Fund
Hedge Fund
Exchange Exchange Exchange
Algorithmic
Trading
System
Prime Service Provider
Ice Berg
VWAP
TWAP
33. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Client
Client
Client
Exchanges Clearing Settlement
Equities
Derivatives
Fixed
Income
Reference
Data
Validation
and
Enrichment
Risk
Management
Financial
Control
Clearing
and
Settlement
34. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Date: 15 May 2007
Produced by: Chris Swan
35. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Overview of stock exchanges
The main stock exchanges in the world include:
America
• American Stock Exchange
• NASDAQ
• New York Stock Exchange
• São Paulo Stock Exchange
Europe
• Euronext
• Frankfurt Stock Exchange
• London Stock Exchange
• Madrid Stock Exchange
• Milan Stock Exchange
• Zurich Stock Exchange
• Stockholm Stock Exchange
36. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Listing requirements
LSE — main market has requirements for a minimum market
capitalization of £700,000, three years of audited financial statements,
minimum public float of 25 % and sufficient working capital for at least 12
months from the date of listing
NASDAQ — to be listed a company must have issued at least 1.25
million shares of stock worth at least $70 million and must have earned
more than $11 million over the last three years
NYSE — a company must have issued at least a million shares of stock
worth $100 million and must have earned more than $10 million over the
last three years
37. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Overview of stock exchanges
Australia/Asia/Africa
• Australian Stock Exchange
• Bombay Stock Exchange
• Hong Kong Stock Exchange
• Johannesburg Securities Exchange
• Korea Stock Exchange
• Shanghai Stock Exchange
• Taiwan Stock Exchange
• Tokyo Stock Exchange
• Toronto Stock Exchange
38. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Participants
Broker — an individual or firm which operates between a buyer and a seller and usually
charge a commission. For most products a licence is required.
Dealer — an individual or firm which buys and sells for its own account.
Broker/dealer — an individual or firm buying and selling for itself and others. A
registration is required.
Principal — a role of broker/dealer when buying or selling securities for its own account.
Market maker — a brokerage or bank that maintains a firm bid and ask price in a given
security by standing ready, willing, and able to buy or sell at publicly quoted prices (called
making a market). These firms display bid and offer prices for specific numbers of specific
securities, and if these prices are met, they will immediately buy for or sell from their own
accounts.
Specialist — a stock exchange member who makes a market for certain exchange-traded
securities, maintaining an inventory of those securities and standing ready to buy and sell
shares as necessary to maintain an orderly market for those shares. Can be an individual,
partnership, corporation or group of firms.
39. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Prototypical trading systems
Call (periodic) auction — selling stocks by bid at intervals throughout
the day. The orders are stored for execution at a single market clearing price.
Continuous auction — buyers enter competitive bids and sellers place
competitive offers simultaneously. Continuous, since orders are executed upon
arrival.
Dealership market — trading occur between principals buying and
selling to their own accounts. Firm price quotations are available prior to order
submission.
Auction markets are concentrated and order-driven
Dealership markets are fragmented and quote-driven
40. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Examples
NYSE — opens with a periodic auction market and then
switches to a continuous auction. Same for Tokyo Stock
Exchange.
NASDAQ and International Stock Exchange
(London) are quote-driven systems (continuous dealership
market).
41. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Examples
Euronext Paris — the market is segmented into a number of
different groups of stocks based on size and liquidity. The trading
mechanisms vary depending on the segment.
Euronext 100, Next 150 ,CAC40 indices and stocks which have
more than 2,500 order book transactions per year — continuous
auction.
Other stocks — call auction twice a day.
42. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Order types
Market order — immediate execution at the best price
available when the order reaches the marketplace
Limit order — to execute a transaction only at a specified price
(the limit) or better
Stop order
Good till cancelled
Fill-or-kill
All or None
Day order
43. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Algorithmic Trading: An Overview ofAlgorithmic Trading: An Overview of
Applications And Models.Applications And Models.
Ekaterina Kochieva
Gautam Mitra
Cormac Lucas
44. Copyright ® 2009, SAS Institute Inc. All right s reserved.
Summary
Definitions and examples have been given about XBRL &
Financial Data Mining.
Key components to make this work are the global adoption
of a Business Reporting language. This includes
consistent standards being set by both XBRL US & XBRL
International.
All companies need to be held responsible for their
reporting with the current Reg rules adjusted. A global
certification process is needed with the proper GRC
engagement model followed. The good news is that there
will be plenty of data to be mined.
How we mine and integrate this into our Trading Strategies
is up to us. This is the "special sauce" which will make or
break how our Trader trades & Trade Support supports.
Editor's Notes
XBRL provides more than just XML Schema for addressing these problems; it provides XML Schema in connection with XML Link (or XLink) to provide multidimensionality. The XBRL Item is connected to its label or machine definition via a schema concept; then XBRL goes beyond the use of Schema to link the element to a range of additional concepts:
Presentation – providing a human readable label…..in this case with English, French, German, Dutch, Chinese, Japan, Russian, and Ukrainian presentation alternatives for this item label.
Reference – can be any other concept such as reference materials, business process guidance, a Web service or even another business rule.
Calculation – provides an internal mathematical check or validation on the item.
Context – provides the item with transferable context around currency, period, nature (actual, budgeted, forecasted, etc.).
The combination of XML Schema and XML Link provide the data with significant standardization and validation context that can be shared across applications, platforms and entities via domain level taxonomies.
Market maker — a brokerage or bank that maintains a firm bid and ask price in a given security by standing ready, willing, and able to buy or sell at publicly quoted prices (called making a market). These firms display bid and offer prices for specific numbers of specific securities, and if these prices are met, they will immediately buy for or sell from their own accounts. Market makers are very important for maintaining liquidity and efficiency for the particular securities that they make markets in. At most firms, there is a strict separation of the market-making side and the brokerage side, since otherwise there might be an incentive for brokers to recommend securities simply because the firm makes a market in that security.
A stop order (sometimes known as a stop loss order) is an order to buy or sell a security once the price of the security reaches a specified price, known as the stop price. When the specified price is reached, the stop order is entered as a market order. Stop orders are used to try to limit an investor's exposure in the market. With a stop order, the customer does not have to actively monitor how a stock is performing. However because the order is triggered automatically when the stop price is reached, the stop price could be activated by a short-term fluctuation in a security's price. Once the stop price is reached, the stop order becomes a market order. In a fast-moving market, the price at which the trade is executed may be much different from the stop price.
The use of stop orders is much more frequent for stocks, and futures, that trade on an exchange than in the over-the-counter (OTC) market.