Time Series Analysis
Time series analysis is a specialized sub-domain of data management, with special rules. Applications that crunch time series data (including market data) benefit from a DBMS tailored to the task.
eXtremeDB Financial Edition performs time series analysis with record-setting performance through two capabilities: columnar data layout and pipelining vector-based statistical functions.
Columnar Data Layout
Traditional DBMSs bring rows of data from a database table into CPU cache for processing. But a time series, such as a security’s price over time, typically occupies a single column within a table. Much of the rest of the row is irrelevant for time series analysis, making it inefficient to fetching entire rows.
With its sequence data type, eXtremeDB Financial Edition implements columnar data layout. This means data is transferred into CPU cache column-by-column, and the CPU cache is not “flooded” with unneeded data.
Traditional DBMSs bring rows of data into the CPU cache for processing. But financial data – such as trades and quotes – are better handled by a column-based layout that avoids flooding the cache with unwanted data.
Throughput between main memory and CPU cache is 2x to 4x slower than the CPU can process data. Traditional DBMSs traverse this bottleneck frequently, with the CPU typically handing off results of each step of a multi-step calculation to temporary tables in main memory.
In contrast, pipelining vector-based statistical functions with eXtremeDB Financial Edition avoids these handoffs, keeping interim results in CPU cache.
Record-Setting Time Series Analysis
The Securities Technology Analysis Center’s STAC-M3 benchmark presents the ideal test of time series analysis performance, with queries provided by trading firms. Using the features described above, eXtremeDB Financial Edition has repeatedly delivered record-setting STAC-M3 results.
Hybrid Data Layout
While columnar data layout accelerates time series analysis, conventional layout using rows is often faster for data that is not sequential. eXtremeDB Financial Edition implements row-based layout for all data types other than sequences. Row and columnar layout can be combined in hybrid data designs to optimize performance managing mixed data.