Nt1310 Unit 3 Storage System

DE-DUPLICATION STORAGE SYSTEM
There are few deduplication storage systems that are being used with respect to the different storage purposes. Some of them have mentioned below:
1. Duplicate Data Elimination (DDE):
Duplicate Data Elimination supports both the combination of the content hashing then copy on write and lazy updates to identify and merge the indistinguishable data blocks in the SAN (storage area network system).The core difference between DDE storage system and other storage systems is that it accurately deduplicate and analyzes the analogous hash values of the blocks of the data right at the source side itself, before the actual transmission of the data. This kind of system always works in background.
In Deduplication technology,

Extreme Binning:
This kind of storage systems focus on non conventional workload. This storage system provides scalability in support of corresponding deduplication approach. It includes composition of low-locality individual files. The similarity of files gets priority over locality and permits one disk access for blocks lookup for file. This technique includes sorting of similar data into bins & removing duplicate chunks inside each bins. In this way deduplication achieved with respect to the different bins. In addition to this extreme binning keeps only primary index in memory that reduces RAM consumption.
3. HYDRAstor:
This is secondary storage solution, which offers the decentralized hash index for the grid of storage nodes, which proceed as the back end and a conventional file interface as the front end. The back end HYDRAstor is brilliant to manage huge scale, changeable size and static addressed unchallengeable and extremely flexible data chunks with the help of the Directed Acyclic Graph. The deduplication is detected based on the hash table. The eventual target of this type of storage system is to form a backup system
Disadvantage:
1. It ignored multiple users sharing files request.
4.

Minimum and Maximum chunk size limit used for splitting file into chunks for searching for duplicates.
Drawback:
As larger chunks produced might escape duplicate detection are more likely to be related than smaller ones.
TTTD-S Algorithm:
This algorithm overcomes the drawback of TTTD. It computes average threshold as a new parameter of regular and backup divisor and uses it as benchmark of chunking algorithm. When average threshold is hit, primary and secondary divisors shrink to half once chunking is completed, both divisors get restored.
CONTENT BASED CHUNKING:
In this chunking, file format is known. Depending on the section of the file, it provides option to change the chunk size. File section or some formats will not change anytime. Using this method we can assume how often the section or file is going to change in future and size of chunk accordingly.
Advantage:
This algorithm is aware of the file format. When compared with other algorithms it sets the boundaries more natural. So it facilitates more space