Abstract

Big Data processing applications have become popular in the last few years. One of the main reasons is that the data generated by current systems and applications is more complex, have a higher speed and its volume increases exponentially. Another reason would be that the traditional methods for data processing and storage are obsolete and the new tools and frameworks brought a lot of advantages. Various social networks need to process big volumes of data, and users take into consideration the speed and quality of the process. We propose an initial approach for processing data from Twitter social network, in a system which allows a real-time classification of tweets based on topics and user location. With this approach we argue that in a dynamic world, were data increases exponentially and the processing needs to be very fast, the proposed system is capable to determine trending topics in real time.

References

• Jonathan Stuart Ward and Adam Barker, Undefined By Data: A Survey of Big Data Definitions, University of St Andrews, School of Computer Science, 2013. (accessed 24 april 2016)
• Thibaud Chardonnens, Big Data analytics on high velocity streams, University of Fribourg (Switzerland), 2013. (accessed 25 april 2016)
• Aftabl A. Chandio, Nikos Tziritas, Cheng-Zhong Xu, Big-Data Processing Techniques and Their Challenges in Transport Domain, Research Gate, februarie 2015. DOI: 10.3969/j.issn.1673-5188.2015.01.007 (accessed 24 april 2016)
• C.L. Philip Chen, Chun-Yang Zhang. Data-intensive applications, challenges, techniques and technologies: A survey on Big Data, Information Sciences Volume 275, pages 314-347, august 2014. [Online]: http://dx.doi.org/10.1016/j.ins.2014.01.015 (accessed 25 April 2016)
• Benoît Perroud. A hybrid approach to enabling real-time queries to end-users. Software Developer’s Journal, 2013. (access 24 April 2016)
• Nathan Marz and James Warren, Big Data Principles and best practices of scalable real time data systems, Manning, aprilie 2015. (accessed 24 April 2016)
• Boyang Peng, Elasticity and Resource Aware Scheduling in Distributed Data Stream Processing Systems, Master Thesis, University of Illinois at Urbana-Champaign, 2015. (access 24 april 2016)
• Karan Patel, Yash Sakaria and Chetashri Bhadane, Real Time Data Processing Frameworks, International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.5, No.5, septembrie 2015, DOI: 10.5121/ijdkp.2015.5504. (accessed 24 may 2016)
• Martin Illecker, Real-time Twitter Sentiment Classification based on Apache Storm, Master Thesis, Innsbruck, 2015.
• Bc. Dávid Katuščák, Dynamic Processing of Event Streams Using Java Tools, Master’s thesis, Brno, 2015.
• Sanjeev Kulkarni, Nikunj Bhagat, Maosong Fu, Vikas Kedigehalli, Christopher Kellogg, Sailesh Mittal, Jignesh M. Patel, Karthik Ramasamy, Siddarth Taneja, Twitter Heron: Stream Processing at Scale, in Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, 2015. [Online]: http://dx.doi.org/10.1145/2723372.2742788
• Arkaitz Zubiaga, Damiano Spina, Raquel Martinez, Victor Fresno, Real-Time Classification of Twitter Trends, Journal of the American Society for Information Science and Technology, March 2014. [Online]: http://arxiv.org/abs/1403.1451v1
• Doug Laney, 3D Data Management: Controlling Data Volume, Veocity, and Variety, META Group, February 2001. [Online]: https://blogs.gartner.com/doug-laney/files/2012/01/ad949-3D-Data Management-Controlling-Data-Volume-Velocity-and-Variety.pdf