stocknet-dataset This repository releases a comprehensive dataset for stock movement prediction from tweets and historical stock prices. Please cite the following paper [bib] if you use this dataset, Yumo Xu and Shay B. Cohen. 2018. Stock Movement Prediction from Tweets and Historical Prices. In Proceedings of the 56st Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia, volume 1. > Stock movement prediction is a challenging problem: the market is highly stochastic, and we make temporally-dependent predictions from chaotic data. We treat these three complexities and present a novel deep generative model jointly exploiting text and price signals for this task. Unlike the case with discriminative or topic modeling, our model introduces recurrent, continuous latent variables for a better treatment of stochasticity, and uses neural variational inference to address the intractable posterior inference. We also provide a hybrid objective with temporal auxiliary to flexibly capture predictive dependencies. We demonstrate the state-of-the-art performance of our proposed model on a new stock movement prediction dataset which we collected. You might also be interested in our code for stock movement prediction. Should you have any query please contact me at yumo.xu@ed.ac.uk. ## Dataset Overview Two-year price movements from 01/01/2014 to 01/01/2016 of 88 stocks are selected to target, coming from all the 8 stocks in the Conglomerates sector and the top 10 stocks in capital size in each of the other 8 sectors. The full list of 88 stocks and their companies selected from 9 sectors is available in StockTable, a facsimile of the paper appendix appendix_table_of_target_stocks.pdf. ## Data Component This dataset comprises two main components, * ./tweet: tweet data from Twitter * ./price: price data from [Yahoo Finance](http://nance.yahoo.com/) Each component contains their raw data and preprocessed data organized by stocks, * ./tweet/raw * ./tweet/preprocessed and * ./price/raw * ./price/preprocessed ## Data Format ### Raw Tweet Data Format: JSON Keys: see Introduction to Tweet JSON ### Preprocessed Tweet Data Format: JSON Keys: 'text', 'user_id_str', 'created_at' ### Raw Price Data Format: CSV Entries: date, open price, high price, low price, close price, adjust close price, volume ### Preprocessed Price Data Format: TXT Entries: date, movement percent, open price, high price, low price, close price, volume Note: open, high, low, close prices are normalized values.

(1) provide financial news for each specific stock. (2) provide various stock technical factors and fundamental factors for each stock.

The EDT dataset is designed for corporate event detection and text-based stock prediction (trading strategy) benchmark. 1. Corporate Event Detection It includes 9721​ news articles with token-level event labels. Including 11 event types: Acquisitions, Clinical Trials, Guidance Changes, New Contracts, Stock Repurchases, Stock Split, Reverse Stock Split/ADS Ratio Change, Regular Dividend, Special Dividend, Dividend Cut, Dividend Increase 2. Text-Based Stock Prediction Benchmark It includes 303893​ first-hand news articles from high-quality sources. Each news article is assigned a minute-level timestamp and comprehensive stock price labels. Please see this Github Link and paper for more details.

The dataset contains historical technical data of Dhaka Stock Exchange (DSE). The data was collected from different sources found in the internet where the data was publicly available. The data available here are used for information and research purposes and though to the best of our knowledge, it does not contain any mistakes, there might still be some mistakes. It is not encourages to use this dataset for portfolio management purposes and use this dataset out of your own interest. The contributors do not hold any liability if it is used for any purposes.

A pool of real stocks from S&P 500 for recent 21 years from 01/02/2000 to 12/31/2020. We filter stocks that have missing data throughout the whole period, resulting in 150 stocks with 5,284 trading days.

A daily emerging stock market dataset (Chinese CSI 300 dataset) including 300 stocks and 5,088 time steps from the CSMAR database. We construct our stock dataset using a pool of stocks from the CSI 300 index for the last 21 years, from 01/02/2000 to 12/31/2020. Instead of all stocks in the market, we select the stocks that used to belong to the major market index CSI 300, and filter out stocks that have missing price data over the period. For each trading day, we use the fundamental price features as the features of stocks, including open price, close price, and volume. Additionally, we normalize price features such as open price and close price with logarithm. The dataset randomly splits stocks into five non-overlapping sub-datasets. For each subset, the first 90% of trading days are used as train data, the following 5% as validation data, and the rest 5% as test data.

Enhancing Financial Market Predictions: Causality-Driven Feature Selection This paper introduces FinSen dataset that revolutionizes financial market analysis by integrating economic and financial news articles from 197 countries with stock market data. The dataset’s extensive coverage spans 15 years from 2007 to 2023 with temporal information, offering a rich, global perspective 160,000 records on financial market news. Our study leverages causally validated sentiment scores and LSTM models to enhance market forecast accuracy and reliability. # Our FinSen Dataset arXiv Pytorch 1.5 License: MIT This repository contains the dataset for Enhancing Financial Market Predictions: Causality-Driven Feature Selection, which has been accepted in ADMA 2024. If the dataset or the paper has been useful in your research, please add a citation to our work: @article{liang2024enhancing, title={Enhancing Financial Market Predictions: Causality-Driven Feature Selection}, author={Liang, Wenhao and Li, Zhengyang and Chen, Weitong}, journal={arXiv e-prints}, pages={arXiv--2408}, year={2024} } ### Datasets [FinSen] can be downloaded manually from the repository as csv file. Sentiment and its score are generated by FinBert model from the Hugging Face Transformers library under the identifier "ProsusAI/finbert". (Araci, Dogu. "Finbert: Financial sentiment analysis with pre-trained language models." arXiv preprint arXiv:1908.10063 (2019).) We only provide US for research purpose usage, please contact w.liang@adelaide.edu.au for other countries (total 197 included) if necessary. We also provide other NLP datasets for text classification tasks here, please cite them correspondingly once you used them in your research if any. 1. 20Newsgroups. Joachims, T., et al.: A probabilistic analysis of the rocchio algorithm with tfidf for text categorization. In: ICML. vol. 97, pp. 143–151. Citeseer (1997) 2. AG News. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015) 3. Financial PhraseBank. Malo, P., Sinha, A., Korhonen, P., Wallenius, J., Takala, P.: Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology 65(4), 782–796 (2014) ### Dataloader for FinSen We provide the preprocessing file finsen.py for our FinSen dataset under dataloaders directory for more convienient usage. ### Models - Text Classification 1. DAN-3. 2. Gobal Pooling CNN. ### Models - Regression Prediction 1. LSTM ### Using Sentiment Score from FinSen Predict Result on S&P500 ### Dependencies The code is based on PyTorch under code frame of https://github.com/torrvision/focal_calibration, please cite their work if you found it is useful. :smiley: ☺ Happy Research !

TRADES-LOB comprises simulated TRADES market data for Tesla and Intel, for 29/01 and 30/01. Specifically, the dataset is structured into four CSV files, each containing 50 columns. The initial six columns delineate the order features, followed by 40 columns that represent a snapshot of the LOB across the top 10 levels. The concluding four columns provide key financial metrics: mid-price, spread, order volume imbalance, and Volume-Weighted Average Price (VWAP), which can be useful for downstream financial tasks, such as stock price prediction. In total, the dataset is composed of 265,986 rows and 13,299,300 cells, which is similar in size to the benchmark FI-2010 dataset.