Author : Abhishek Pandhare, 1
Date of Publication :13th April 2017
Abstract: Nowadays, cloud-based storage services are rapidly growing and becoming an emerging trend in data storage field. There are many problems when designing an efficient storage engine for cloud-based systems with some requirements such as big-file processing, lightweight meta-data, low latency, parallel I/O, deduplication, distributed, high scalability. Keyvalue stores played an important role and showed many advantages when solving those problems. This paper presents about Big File Cloud (BFC) with its algorithms and architecture to handle most of problems in a big-file cloud storage system based on key value store. It is done by proposing low-complicated, fixed-size meta-data design, which supports fast and highly-concurrent, distributed file I/O, several algorithms for resumable upload, download and simple data deduplication method for static data. This research applied the advantages of ZDB - an in-house key value store which was optimized with auto-increment integer keys for solving big-file storage problems efficiently. The results can be used for building scalable distributed data cloud storage that support big-file with size up to several terabytes
Reference :
-
- Dropbox tech blog. https://tech.dropbox.com/. Accessed October 28, 2014.
- Zing me. http://me.zing.vn. Accessed October 28,2014
- Facebook. http://facebook.com, 2014
- Federal Information Processing Standards Publication 197 ADVANCED ENCRYPTION STANDARD (AES) November 26, 2001
- I. Drago, E. Bocchi, M. Mellia, H. Slatman, and A. Pras. Benchmarking personal cloud storage. In Proceedings of the 2013 conference on Internet measurement conference, pages 205–212. ACM, 2013.
- I.Drago, M.Mellia, M. M Munafo, A.Sperotto, R.Sadre, and A. Pras. Inside dropbox: understanding personal cloud storage services. In Proceedings of the 2012 ACM conference on Internet measurement conference, pages 481494. ACM, 2012.
- S. Ghemawat, H. Gobioff, and ST. Leung. The google file system. In ACM SIGOPS Operating Systems Review, volume 37, pages 2943. ACM, 2003.
- Thanh Trung Nguyen, Tin Khac Vu, Minh HieuNguyen ,LeQuy Don Technical University, Ha Noi, Distributed Big File Cloud Storage based on key value store Viet Nam †VNG Research, Viet Nam
- International Journal of Advanced Research in Computer Science and Software Engineering Volume 5, Issue 12, December 2015
- Yan Kit Li, Xialofeng Chen, Patrick P.C.Lee Secured Authorized Deduplication Based Hybrid Cloud Approach
- Oraclehttp: //www.oracle.com/technetwork/art icles/java/compress-1565076.html
- SDLC http: // www.veracode.com/ security/ softwaredevelopmentlifecycle