Secure data deduplication mechanism based on Rabin CDC and MD5 in cloud computing environment

Chunking is a technique that splits a whole file or data into separate chunks. Chunking is applied to detect duplication in any remote processes such as data deduplication and data compression. Content defined chunking is a process of splitting the file into variable length chunks as per the cut points defined initially. Also these are more challenging as byte shifting is involved which results to be expected to increase the deduplication process. While processing out content defined Chunking onto cloud based Dedup App for the encrypted data stored in cloud results in improvement in efficiency. In this paper, we have proposed a technique which includes applying Rabin CDC for purpose of making variable size chunks and further processing that data chunks for encrypted data on cloud. This works on same data being uploaded by different users on same cloud storage (Hadoop Distributed File System).