GraphDuo: A Dual-Model Graph Processing Framework
Xinhui Tian, Jianfeng Zhan
IEEE Access · 2018
doi:10.1109/access.2018.2848291 (opens in a new tab)
We email you when a new notice or a question about this paper appears.
Abstract
Algorithms for large-scale natural graph processing can be categorized into two types based on their value propagation behaviors: the unidirectional value propagation algorithms and the bidirectional value propagation algorithms. The graph computation in different types exhibits different properties on how vertices interact with their neighbors and has different requirements on system design. Current distributed graph processing systems usually try to support both types in one general-purpose computing model, which can result in suboptimial performance, high communication overhead, and high resource consumption. In this paper, we propose GraphDuo, a new Spark-based graph processing framework that provides two efficient computing models for unidirectional and bidirectional value propagation algorithms, respectively. Combined with a degree-based graph partitioning scheme, a locality-aware graph layout, and other optimization techniques, GraphDuo achieves low computation cost, low communication overhead, small memory footprint, and good communication balance for two types of algorithms. According to the experimental results, GraphDuo can outperform GraphX from 1.76× to 6.56× and from 1.11× to 1.26× for two types of algorithms, respectively, with much less memory consumption and communication cost. The source code of GraphDuo is publicly available from http://prof.ict.ac.cn/GraphDuo.
Abstract licence: CC BY-NC-ND
Reads
Read by people on this page
None yet.
Checked or read by AI through naulon
None yet.
A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.
Are you an author?
Is this your paper?
Claim it to add your note and be paid when agents cite it. Free.