Abstract

In this massive technological atmosphere the number of information generated is increasing at an awfully high rate. Distributed storage system will be used for storing this immense quantity of information. Massive information will be handled by victimization hadoop and map reduces. The multiple node clusters will discover victimization the hadoop framework. Hadoop comes with default distributed file system that is hadoop distributed file system. The mapped is enforced by map technique and that we can see the performance of map reduce task on the bases of bytes written. The amount of bytes written by map task doesn't increase with the amount of files increasing. The explanation is that once the reduce function perform reduces the map output it simply combines the output of the map function. Thus so as to enhance the performance of map reduce task we are going to setup the cluster of nodes in heterogeneous atmosphere and analyze the behavior of map reduce task victimization the perfecting mechanism.

References

• J. Dean and S. Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters”, Communications of the ACM, vol. 51, no. 1, (2008).
• S. Seo, I. Jang, K. Woo, I. Kim, J. S. Kim and S. Maeng, “HPMR: Prefetching and Pre-shuffling in Shared Map Reduce Computation Environment”, IEEE International Conference on Cluster Computing and Workshops, (2009) August 31-September 4: New Orleans, USA.
• S. Khalil, S. A. Salem, S. Nassar and E. M. Saad, “Mapreduce Performance in Heterogeneous Environments: A Review”, International Journal of Scientific & Engineering Research, vol. 4, no. 4, (2013).
• J. Xie, S. Yin, X. Ruan, Z. Ding, Y. Tian, J. Majors, A. Manzanares and X. Qin, “Improving MapReduce Performance through Data Placement in Heterogeneous Hadoop Clusters”, IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), (2010) April 19-23: Arlanta, USA.
• M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker and I. Stoica, “Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling”, Proceedings of the 5th European conference on Computer systems, (2010) April 13-16: Paris, France.
• Z. Tang, J. Q. Zhou, K. L. Li and R. X. Li, “MTSD: A task scheduling algorithm for MapReduce base on deadline constraints”, IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum (IPDPSW), (2012) May 21-25: Shanghai, China.
• X. Zhang, Z. Zhong, S. Feng and B. Tu, “Improving Data Locality of MapReduce by Scheduling in Homogeneous Computing Environments”, IEEE 9th International Symposium on Parallel and Distributed Processing with Applications (ISPA), (2011) May 26-28: Busan, Korea.
• C. Abad, Y. Lu and R. Campbell, “DARE: Adaptive Data Replication for Efficient Cluster Scheduling”, IEEE International Conference on Cluster Computing (CLUSTER), (2011) September 26-30: Austin, USA.