In this paper, we propose a new hybrid model of asymmetric
volatility by using treed Gaussian process with jumps to the limiting
linear model (TGPLLM) of Gramacy and Lee combined with the
volatility switching ARCH (VS-ARCH) developed by Fornari and
Mele to model and predict stock market volatility. Nonparametric
sensitivity analysis based on the TGPLLM is applied to check the
relevance level of five input variables in the model. Meanwhile,
support vector machine is also employed to obtain another new
hybrid model for making a comparison with the former. Empirical
analysis of NASDAQ index reveals that the five input variables are
all significant; the hybrid model based on TGPLLM yields better
predictive performance than the ones based on SVM, the parametric
models of VS-ARCH, ARMA-GARCH and ARMA-GJR models