This is a paper on arXive.
Abstract:
                                                                                                                                               
                                                                                                                                                [size=10.000000pt]With the rapid development of artificial intelligence, long short term memory (LSTM), onekind of recurrent neural network (RNN), has been widely applied in time series prediction.
                                                [size=10.000000pt]Like RNN, Transformer is designed to handle the sequential data. As Transformer achievedgreat success in Natural Language Processing (NLP), researchers got interested in Trans-former’s performance on time series prediction, and plenty of Transformer-based solutionson long time series forecasting have come out recently. However, when it comes to financialtime series prediction, LSTM is still a dominant architecture. Therefore, the question thisstudy wants to answer is: whether the Transformer-based model can be applied in financialtime series prediction and beat LSTM.
                                                [size=10.000000pt]To answer this question, various LSTM-based and Transformer-based models are comparedon multiple financial prediction tasks based on high-frequency limit order book data. Anew LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction. The experiment result reflectsthat the Transformer-based model only has the limited advantage in absolute price sequenceprediction. The LSTM-based models show better and more robust performance on differencesequence prediction, such as price difference and price movement.