STOCK MARKET ANALYSIS USING MAPREDUCE AND PYSPARK

Author

Dr. X. Stellamary


Abstract

 Stock market analysis has become increasingly important for understanding market movements, investment behaviour and financial risk. The rapid growth of financial data has created a need for scalable computational methods capable of processing large datasets efficiently. This study presents an analytical framework that combines Hadoop MapReduce and PySpark for stock volatility analysis, sentiment analysis and stock price prediction. A large stock dataset is examined to identify securities with relatively high and low volatility. MapReduce is used to calculate monthly returns and volatility, while PySpark and machine-learning techniques are applied to examine price movements. News information is also incorporated to study the relationship between market sentiment and stock behaviour. The framework demonstrates how big-data technologies can support financial analysis by processing historical market information and unstructured news data. The study highlights the usefulness of distributed computing and machine learning for analysing complex stock-market information, while recognising that market behaviour is influenced by numerous changing economic and financial factors.


Keywords

Stock Market, Big Data Analytics, Hadoop MapReduce, PySpark, Stock Volatility, Sentiment Analysis, Machine Learning, Stock Price Prediction.



Full Text:

Download Paper PDF


References


  • Kavya, P., Saagarika, S., Subavarsshini, R. T., Nivetheni, C. G., Marimuthu, M., & Velvadivu, P. (2021). Stock Market Analysis Using MapReduce and PySpark. International Research Journal of Modernization in Engineering Technology and Science, 3(7).
  • Apache Hadoop documentation. MapReduce framework and distributed data processing concepts.
  • Apache Spark documentation. PySpark and Spark ML documentation.
  • Stock market and financial-market concepts.
  • Relevant academic literature on financial sentiment analysis, stock-market prediction and machine learning.

Share your valuable work from Social Media Buttons