Spark框架下交通流数据高效处理方法及其应用
Efficient Traffic Flow Data Processing Method and Its Application Based on Spark Framework

摘要 设计并实现基于Spark的交通流数据处理与预测分析应用框架, 可以完成交通流数据的高效清洗、统计、存储和查询。利用基于多阶空间权重矩阵的STARIMA模型进行交通流预测分析, 可以验证数据处理效率及对预测应用的支撑作用。对比实验结果表明: 1) 交通流数据处理框架运行效率高, 适用于复杂的数据清洗和挖掘算法, 为预测模型建立数据支撑; 2) 交通流预测模型对空间权重矩阵进行了多阶优化, 兼顾高效性和准确性, 预测分析结果可以为交通诱导提供参考。
Abstract A traffic flow data processing and forecasting framework based on Spark is designed, and it can complete the efficient cleaning, statistics, storage and query of traffic flow data. A multi-order spatial weight matrix STARIMA model is used to predict the traffic flow, and it can verify the efficiency of data processing and the support for the prediction. By comparative experiments, the results show that the traffic flow data processing framework is efficient, and it is suitable for realizing complex data cleaning and mining algorithms and establishing data support for the prediction model. The traffic flow prediction model optimizes the multi-order spatial weight matrix, and it takes both efficiency and accuracy into consideration. The prediction results can provide reference for traffic guidance.