This project focuses on predicting housing prices in Calgary, utilizing big data technologies such as PySpark for scalability. The team of five members processed data using resilient distributed datasets (RDD) and performed exploratory data analysis (EDA) and regression analysis with PySpark's MLlib.
The project involves three main categories of features: Economic Indicators, Group Price Indicators, and Individual Price Indicators, each contributing to understanding and predicting housing prices. Economic indicators include factors like average home price and unemployment rate, while group price indicators encompass community-based factors like crime rate and resident count. Individual price indicators focus on specific property characteristics like land size and dwelling type.
The team's presentations document the project's progress, including introductions, data preparation, EDA, modeling techniques, and results. The final report provides a comprehensive overview of data acquisition, processing, and modeling.
Data was sourced from various sources including The City of Calgary, Statistics Canada, and other relevant sources to ensure a robust analysis of housing price prediction in Calgary.
Overall, the project demonstrates a thorough approach to leveraging big data technologies for predicting housing prices, with a focus on scalability and accuracy.
Using resilient distributed dataset (RDD) to pre-process and perform basic exploratory data analysis big data using PySpark and a Google Colab notebook.