• 首页
  • 期刊简介
  • 编委会
  • 投稿指南
  • 收录情况
  • 杂志订阅
  • 联系我们
引用本文:刘 念,蔡春花.基于Spark的电影推荐系统的设计与实现[J].软件工程,2023,26(6):59-62.【点击复制】
【打印本页】   【下载PDF全文】   【查看/发表评论】  【下载PDF阅读器】  
←前一篇|后一篇→ 过刊浏览
分享到: 微信 更多
基于Spark的电影推荐系统的设计与实现
刘 念, 蔡春花
(黄河交通学院, 河南 焦作 454000)
n135691@foxmail.com; 786803383@qq.com
摘 要: 针对大数据时代电影资源快速增长而出现的信息过载问题,采用Spark分布式计算平台,结合MongoDB(分布式文档存储数据库)业务数据库、Redis(远程字典服务)缓存数据库,以及Flume(日志收集系统)、Kafka(开源流处理平台)和Spring开源框架等工具设计并实现了一个电影推荐系统。系统主要包括数据采集模块、离线推荐模块、实时推荐模块和综合业务模块,其中离线推荐模块采用交替最小二乘(ALS)算法训练隐语义模型求得预测评分,实时推荐模块通过Spark Streaming流实时处理用户在可视化界面对电影的评分操作。经功能测试及兼容性测试表明,系统可依据用户行为偏好,快速准确地完成电影推荐,可用性较高。
关键词: 离线推荐;电影推荐;Spark
中图分类号: TP391    文献标识码: A
基金项目: 黄河交通学院校级一流课程建设项目“Linux原理及应用”(HHJTXY-2021ylkc13);黄河交通学院校级教学资源库建设项目“NoSQL数据库原理”(HHJTXY-2022kczyk103).
Design and Implementation of Spark-based Movie Recommendation System
LIU Nian, CAI Chunhua
(Huanghe Jiaotong University, Jiaozuo 454000, China)
n135691@foxmail.com; 786803383@qq.com
Abstract: To address the problem of information overload caused by the rapid growth of movie resources in the era of big data, this paper proposes to design and implement a movie recommendation system by using Spark distributed computing platform, combined with MongoDB (distributed document storage database) business database, Redis (remote dictionary service) cache database, Flume (log collection system), Kafka (open source stream processing platform) and Spring open source framework. The system mainly includes data collection module, offline recommendation module, real-time recommendation module and integrated business module. The offline recommendation module uses the Alternating Least Squares (ALS) algorithm to train the implicit semantic model for predicting ratings, and the realtime recommendation module processes users' movie ratings in the visual interface in real time through Spark Streaming. Function and compatibility tests show that the proposed system can quickly and accurately complete movie recommendations based on users' behavior preferences, with high usability.
Keywords: offline recommendation; movie recommendation; Spark


版权所有:软件工程杂志社
地址:辽宁省沈阳市浑南区新秀街2号 邮政编码:110179
电话:0411-84767887 传真:0411-84835089 Email:semagazine@neusoft.edu.cn
备案号:辽ICP备17007376号-1
技术支持:北京勤云科技发展有限公司

用微信扫一扫

用微信扫一扫