stocknet-dataset This repository releases a comprehensive dataset for stock movement prediction from tweets and historical stock prices. Please cite the following paper [bib] if you use this dataset, Yumo Xu and Shay B. Cohen. 2018. Stock Movement Prediction from Tweets and Historical Prices. In Proceedings of the 56st Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia, volume 1. > Stock movement prediction is a challenging problem: the market is highly stochastic, and we make temporally-dependent predictions from chaotic data. We treat these three complexities and present a novel deep generative model jointly exploiting text and price signals for this task. Unlike the case with discriminative or topic modeling, our model introduces recurrent, continuous latent variables for a better treatment of stochasticity, and uses neural variational inference to address the intractable posterior inference. We also provide a hybrid objective with temporal auxiliary to flexibly capture predictive dependencies. We demonstrate the state-of-the-art performance of our proposed model on a new stock movement prediction dataset which we collected. You might also be interested in our code for stock movement prediction. Should you have any query please contact me at yumo.xu@ed.ac.uk. ## Dataset Overview Two-year price movements from 01/01/2014 to 01/01/2016 of 88 stocks are selected to target, coming from all the 8 stocks in the Conglomerates sector and the top 10 stocks in capital size in each of the other 8 sectors. The full list of 88 stocks and their companies selected from 9 sectors is available in StockTable, a facsimile of the paper appendix appendix_table_of_target_stocks.pdf. ## Data Component This dataset comprises two main components, * ./tweet: tweet data from Twitter * ./price: price data from [Yahoo Finance](http://nance.yahoo.com/) Each component contains their raw data and preprocessed data organized by stocks, * ./tweet/raw * ./tweet/preprocessed and * ./price/raw * ./price/preprocessed ## Data Format ### Raw Tweet Data Format: JSON Keys: see Introduction to Tweet JSON ### Preprocessed Tweet Data Format: JSON Keys: 'text', 'user_id_str', 'created_at' ### Raw Price Data Format: CSV Entries: date, open price, high price, low price, close price, adjust close price, volume ### Preprocessed Price Data Format: TXT Entries: date, movement percent, open price, high price, low price, close price, volume Note: open, high, low, close prices are normalized values.

(1) provide financial news for each specific stock. (2) provide various stock technical factors and fundamental factors for each stock.

The EDT dataset is designed for corporate event detection and text-based stock prediction (trading strategy) benchmark. 1. Corporate Event Detection It includes 9721​ news articles with token-level event labels. Including 11 event types: Acquisitions, Clinical Trials, Guidance Changes, New Contracts, Stock Repurchases, Stock Split, Reverse Stock Split/ADS Ratio Change, Regular Dividend, Special Dividend, Dividend Cut, Dividend Increase 2. Text-Based Stock Prediction Benchmark It includes 303893​ first-hand news articles from high-quality sources. Each news article is assigned a minute-level timestamp and comprehensive stock price labels. Please see this Github Link and paper for more details.