Testing and Tuning Market Trading Systems: Algorithms in C++
by Timothy Masters  (Author)

About the author
Timothy Masters received a PhD in mathematical statistics with a specialization in numerical computing. Since then he has continuously worked as an independent consultant for government and industry. His early research involved automated feature detection in high-altitude photographs while he developed applications for flood and drought prediction, detection of hidden missile silos, and identification of threatening military vehicles. Later he worked with medical researchers in the development of computer algorithms for distinguishing between benign and malignant cells in needle biopsies. For the last twenty years he has focused primarily on methods for evaluating automated financial market trading systems. He has authored five books on practical applications of predictive modeling: Practical Neural Network Recipes in C++ (Academic Press, 1993); Signal and Image Processing with Neural Networks (Wiley, 1994); Advanced Algorithms for Neural Networks (Wiley, 1995); Neural, Novel, and Hybrid Algorithms for Time Series Prediction (Wiley, 1995); Data Mining Algorithms in C++ (Apress, 2018); Assessing and Improving Prediction and Classification (Apress, 2018); Deep Belief Nets in C++ and CUDA C: Volume 1 (Apress, 2018); and Deep Belief Nets in C++ and CUDA C: Volume 2 (Apress, 2018).

About this book
Build, test, and tune financial, insurance or other market trading systems using C++ algorithms and statistics. You’ve had an idea and have done some preliminary experiments, and it looks promising. Where do you go from here?  Well, this book discusses and dissects this case study approach.
Seemingly good backtest performance isn't enough to justify trading real money. You need to perform rigorous statistical tests of the system's validity. Then, if basic tests confirm the quality of your idea, you need to tune your system, not just for best performance, but also for robust behavior in the face of inevitable market changes. Next, you need to quantify its expected future behavior, assessing how bad its real-life performance might actually be, and whether you can live with that. Finally, you need to find its theoretical performance limits so you know if its actual trades conform to this theoretical expectation, enabling you to dump the system if it does not live up to expectations.
This book does not contain any sure-fire, guaranteed-riches trading systems. Those are a dime a dozen... But if you have a trading system, this book will provide you with a set of tools that will help you evaluate the potential value of your system, tweak it to improve its profitability, and monitor its on-going performance to detect deterioration before it fails catastrophically. Any serious market trader would do well to employ the methods described in this book.
What You Will Learn

• See how the 'spaghetti-on-the-wall' approach to trading system development can be done legitimately
• Detect overfitting early in development
• Estimate the probability that your system's backtest results could have been due to just good luck
• Regularize a predictive model so it automatically selects an optimal subset of indicator candidates
• Rapidly find the global optimum for any type of parameterized trading system
• Assess the ruggedness of your trading system against market changes
• Enhance the stationarity and information content of your proprietary indicators
• Nest one layer of walkforward analysis inside another layer to account for selection bias in complex trading systems
• Compute a lower bound on your system's mean future performance
• Bound expected periodic returns to detect on-going system deterioration before it becomes severe
• Estimate the probability of catastrophic drawdown
  

Who This Book Is For
Experienced C++ programmers, developers, and software engineers.  Prior experience with rigorous statistical procedures to evaluate and maximize the quality of systems is recommended as well.  

Table of contents
Chapter 1: Introduction 1
    The Target Audience, and Overview of Contents 1
        What’s in This Book 1
        What’s Not in This Book 3
    About Trading Systems 4
        Market Prices and Returns 5
        Two Types of Automated Trading Systems 6
        The Agony of Believing the Computer 7
        Future Leak Is More Dangerous Than You May Think 7
        The Percent Wins Fallacy 8
Chapter 2: Pre-optimization Issues 11
    Assessing and Improving Stationarity 11
        The STATN Program 13
        Improving Location Stationarity by Oscillating 17
        Extreme Stationarity Induction 19
    Measuring Indicator Information with Entropy 20
        Computing the Relative Entropy of an Indicator 22
        Entropy Impacts Predictive Model Quality 24
        Improving the Entropy of an Indicator 25
        Monotonic Tail-Only Cleaning 29
Chapter 3: Optimization Issues 35
    Regularizing a Linear Model 35
        Overview of the Regularized Model 36
        Beta Adjustment with Guaranteed Convergence 40
        Differential Case Weighting 41
        Rapid Computation with Covariance Updates 42
        Outline of the Beta Optimization Process 46
        Code for Beta Optimization 48
        Descending a Lambda Path 55
        Optimizing Lambda with Cross Validation 59
        The CD_ MA Program 63
    Making a Linear Model Nonlinear 67
    Differential Evolution: A Universal Nonlinear Optimizer 69
        The DIFF_ EV. CPP Routine for Differential Evolution 75
Chapter 4: Post-optimization Issues 91
    Cheap Bias Estimates 91
        The StocBias Class 92
    Cheap Parameter Relationships 96
    Parameter Sensitivity Curves 108
        Putting It All Together Trading OEX 112
Chapter 5: Estimating Future Performance I: Unbiased Trade Simulation 121
    In-Sample and Out-of-Sample Performance 121
        The TrnBias Program to Demonstrate Training Bias 123
        Selection Bias 124
        Walkforward Analysis 129
        Future Leak by Unobvious IS/ OOS Overlap 131
        Cross-Validation Analysis 143
        Special Precautions for Algorithmic Trading 151
    Comparing Cross Validation with Walkforward: XVW 156
        Computationally Symmetric Cross Validation 158
        What Does This Test Actually Measure?  163
    Nested Walkforward Analysis 172
        The Nested Walkforward Algorithm 174
        A Practical Application of Nested Walkforward 179
    An Example Using S& P 100 Components 187
    Cross Validation Nested Inside Walkforward 188
Chapter 6: Estimating Future Performance II: Trade Analysis 193
    Handling Dynamic Trading Systems 193
        Unknown Lookahead to Single Bars, Revisited 194
    Profit per Bar? Per Trade? Per Time?  195
        Analyzing Completed Trade Returns Is Problematic 196
        The PER_ WHAT Program 198
    A Lower Bound for Mean Future Returns 209
        Brief Digression: Hypothesis Tests 210
        So, How Do We Use This Probability?  212
        Parametric P-Values 216
        Parametric Confidence Intervals 218
        Lower Confidence Bounds and Hypothesis Tests 222
    Bootstrap Confidence Intervals 222
        The Pivot and Percentile Methods 223
        The BC a Bootstrap Algorithm  225
        The BOOT_ CONF. CPP Subroutines 227
        The BOUND_ MEAN Program and Results with SPX 232
        Beware of Bootstrapping Ratios 238
    Bounding Future Returns 241
        Deriving a Lower Bound from Empirical Quantiles 242
        Confidence in the Computed Lower Bound 244
        What About an Upper Bound on Future Returns?  247
        The CONFTEST Program: Overview 248
        The CONFTEST Program: Code 251
        The BND_ RET Program 257
    Bounding Drawdown 262
        Intuition Gone Wrong 263
        Bootstrapping Drawdown Bounds 265
        The DRAWDOWN Program 267
        Experiments with the DRAWDOWN Program 277
        The CHOOSER_ DD Program 279
Chapter 7: Permutation Tests 283
    Overview of Permutation Testing 283
    Testing a Fully Specified Trading System 285
        Testing the Training Process 286
        Walkforward Testing a Trading System Factory 287
        Permutation Testing of Predictive Models 289
    The Permutation Testing Algorithm 291
        Extending the Algorithm for Selection Bias 292
    Partitioning Total Return of a Trading System 294
        Essential Permutation Algorithms and Code 298
        Permuting Simple Market Prices 299
        Permuting Multiple Markets with an Offset 301
    Example: P-Value and Partitioning 310
        Example: Training with Next Bar Returns 312
    Example: Permuting Multiple Markets 316
Index 319

Length: 321 pages
Publisher: Apress; 1st ed. edition (December 26, 2018)
Language: English
ISBN-10: 148424172X
ISBN-13: 978-1484241721

PDF version


EPUB version