Optimizing data center traffic of Online Social Networks

Abstract

With a huge number of users and a very large scale of data, an Online Social Network (OSN) service has to partition its data among multiple servers inside a data center. As data are often partitioned randomly, the response time in accessing the data is however unpredictable. Researchers have proposed social locality to address this concern: if a server hosts the master replica of a user's data, it must also host a replica (either master or slave) of every friend of this user, thus enabling convenient access of all of them on the same server. However, doing so comes with two overheads: the replication storage and the traffic of maintaining replica consistency. Existing work focuses on the former, but overlooks the latter that can consume considerable network resources. In this paper, we study social-locality-aware partitioning of the OSN data while meeting diverse performance goals of data center networks. We formulate the traffic optimization problem and propose a new traffic-aware data partitioning algorithm. Through the evaluations with a large-scale, real-world Twitter trace, we further show that, compared with state-of-the-art algorithms, our algorithm significantly reduces traffic without deteriorating the load balance among servers and causing extra replication storage.