This white paper describes TimeScape QL+, a new query language designed for financial markets in order to bring users, data and decision-making closer together. This language is easy to understand, easy to extend, provides powerful support for vector arithmetic of intraday and historic data and deals with data issues that are specific to the financial markets.

View complete TimeScape QL+ white paper PDF. (This document requires the free Adobe/Acrobat Reader.) 16 pages, 420KB

Introduction

More Data, Less Time

Why do we need another query language? Good question. The usage of SQL is both prevalent and effective throughout financial markets IT and the software industry as a whole. However, there are many people who need to access, manipulate and analyse data who are not technical experts in SQL but are experts in their own particular field of business.

Nowhere is this more apparent than within the financial markets - where traders, fund managers, quants, research analysts, risk managers and other business staff are under constant competitive pressure to analyse larger and larger volumes of increasingly complex data in less and less time.

Users and Technologists Think About Data Differently

Business users in financial markets tend to think about the data they need in terms of the financial instruments they trade - such as equities, bonds, options etc. Traders do not think of real-time data, time series data, tick data, static data and calculated data very differently - to them all of this data is needed and relevant to the business decisions that are being made.

However, this same data is generally split out across many tables when stored in a typical relational database implementation, rendering the data as less than user-friendly to access. The implementation gets more complex if different asset classes are stored in different databases. Even within one asset class, static terms and conditions data may be stored in different systems from time series data, tick data and calculated/derived data.

In the absence of further end-user tools to analyse the data, the trader or risk manager must understand both the SQL programming language and the table structure/architecture implemented for the databases in question. Even if the business user is capable of achieving these two things, they often do not have the time to do anything more than avoid using the database by downloading equivalent data from Open Bloomberg into Microsoft Excel.

Technologists can, of course, go some way to addressing the above issue through the provision of read-only views and wrapper functions (in the form of stored procedures) to hide some of the complexity from the end-user. However, as data demands from the business expand, and table complexity increases, these procedures can become computationally expensive and increasingly difficult to manage, maintain and change. As a result, business users continue to be heavily reliant on technologists to deliver and so the unproductive cycle continues.

White paper contents

• More Data, Less Time
• Users and Technologists Think About Data Differently
• TimeScape QL+: Designed for Financial Markets
• Functional Overview of TimeScape QL+
• Some Examples of TimeScape QL+ in Action
  • Example #1 – Loading a Price History for an Equity
  • Example #2 – Calculating Historic Volatility of an Underlying
  • Example #3 – The ‘.?’ Statement and Context Sensitive Help
  • Example #4 – Data Rules and Bond Spread Analysis
  • Example #5 – Tick Data Analysis, Data Frequency and VWAP
  • Example #6 – Multiple Instrument Analysis
  • Example #7 – Vector Arithmetic, Spread Analysis and VAR
  • Example #8 – Adding Your Own Functions
  • Example #9 – Adding Your Own Objects
• Future Directions
• Summary

This paper illustrates how Xenomorph’s real-time analytics and data management system, TimeScape, enables extremely rapid and extensible analysis of tick and intraday timeseries data delivering competitive advantage in pre- and post-trade decision support.

View complete High Frequency Data Analysis white paper PDF. (This document requires the free Adobe/Acrobat Reader.) 20 pages, 860KB

Introduction

A Time of Change and Opportunity

Data management in financial markets are being driven through a period of fundamental change. Trade volumes are increasing exponentially as electronic execution delivers faster trading with ever-tighter margins. The proprietary algorithms used in algorithmic and statistical arbitrage trading are becoming more complex. Developments in areas such as credit theory are establishing market relationships that motivate more complex cross-asset trading strategies. Regulations such as MiFID and Regulation NMS are pushing the whole industry towards better and more transparent execution, but are also fundamental drivers behind both huge business change and dramatically increased data volumes. All of these factors are combining to provide both profit and cost incentives to move away from single asset class data silos.

Looking at data management from a trader’s perspective, then a decade ago many practitioners were content with analysing end of day historic data for strategy back-testing and instrument pricing purposes. It should be said that they quite possibly had no choice from a technological perspective; the capture, storage and analysis of intraday data volumes even then was challenging, especially at a time when the relational database was still a relatively new technology. Given however that derivative pricing margins were wider and statistical arbitrage was profitable using end of day prices, then there was also little incentive to store and analyse intraday tick and high frequency data. Much tighter trading margins, cross-asset trading and improved technology have changed traders’ perceptions of what is required and what kind of analysis is possible with high frequency intraday data.

Risk management has previously not been greatly concerned with intraday data. As many risk managers will confirm, obtaining clean data for end of day risk measurement is challenging enough. Risk measurement techniques such as monte-carlo or historical simulation VaR require large amounts of historical data and are calculation intensive. Large data universes or poor implementation may mean that it is challenging to attempt to run these techniques as an overnight batch, let alone perform the calculations in real or near real-time. Increased intraday trading exposure, better understanding of intraday market behaviour and recent regulatory requirements concerning data transparency and data quality are driving risk managers towards more analysis of tick by tick and intraday data.

The changes above require systems that can adapt to the pace of change, delivering high performance analysis even when the quantity of intraday data being analysed is massive. This paper describes how Xenomorph’s real-time analytics and data management system, TimeScape, has been designed to meet these current challenges and to deliver competitive advantage in pre- and post-trade decision support.

White paper contents

• A Time of Change and Opportunity
• Data Transparency – Increased Productivity for All
• Tick Capture – Flexibility and Ease of Use Together
• Tick Storage – Real-Time, Intraday and History All in One
• Data Access - for Both Traders and Technologists
  • Loading Intraday Data with Functions
  • Loading Intraday Data with TimeScape QL+
• Data Validation – Why Waste Half Your Time?
  • Data Frequency and Time Snapping
  • Data Filling, Aligning and Loading Rules
  • Real or Interpolated Data?
• Data Analysis – From Backtesting to Best Execution
  • Chaining Analytical Functions
  • Intraday Time Period Analysis
  • Beyond the Spreadsheet
• Event Processing – Real-Time Analysis Automation
• Your Proprietary Advantage
• Summary