Five data deduplication technology considerations


Five data deduplication technology considerations

Data deduplication technology is a method of reducing storage needs by eliminating redundant data. The data deduplication

Continue Reading This Article

Enjoy this article as well as all of our content, including E-Guides, news, tips and more.

By submitting your email address, you agree to receive emails regarding relevant topic offers from TechTarget and its partners. You can withdraw your consent at any time. Contact TechTarget at 275 Grove Street, Newton, MA.

You also agree that your personal information may be transferred and processed in the United States, and that you have read and agree to the Terms of Use and the Privacy Policy.

Safe Harbor

phenomenon has caught a lot of attention with not only humungous data growth but also increased adoption of virtualization. Data deduplication technology will eventually become an integrated part of backup application, due to the nature of data growth.

Data deduplication technology is very important in virtual environments, which are very quick to create data. Data deduplication technology allows you to backup only the data that has changed. It takes a file, creates segments, and every time a change is made, the segment in the database is not sent for backup, thereby reducing the amount of data sent from the client to the media server. However, an organization needs to consider some important parameters before deciding to adopt data deduplication technology:

  • Although it can be implemented in any storage environment, an organization with a high data retention rate is more prone to duplicated data and thus, data deduplication technology yields more benefits.
  • Data deduplication can be done on the source, the media server or the target. Deduplicating the data at the source allows you to send only the segments, thereby reducing bandwidth usage and enabling encryption of data. The central processing unit (CPU) is used at the source to do dedupe. So, it works relatively well where lot of CPU or rather significant amount will be available. However, if you are an information technology organization and are backing up data for another department, you might consider using target or media server data deduplication technology.
  • The nature of data for data deduplication technology ought to be considered as some data is more dedupable. For instance, a set of word files will have a high dedupe ratio, whereas databases are not very dedupeable as updating even one record messes up the file. Hence, it should be decided whether to back up data as it is or to delete some of it at the source. Temporary data can be deleted and permanent data can be backed up.
  • Another consideration is the availability of deduplicability hardware such as the virtual tape library, which makes data deduplication technology more conducive.
  • The bandwidth is another important factor, as it is expensive. Hence, it is important to determine where you want to implement data deduplication technology (front end or back end), what data is conducive for deduplication, and other such issues. In primary data, the duplication rate is not as high as secondary data. When you are talking about suitability of data in a traditional environment, you are doing full backups every 12 weeks or so. While doing it at the front end, it becomes a little more difficult to justify it, since the frequency of duplication is relatively low.

About the author: Oussama El-Hilali is the vice president for NetBackup Engineering at Symantec. He is responsible for planning, development and release of Symantec’s NetBackup range, including PureDisk and RealTime Protection. He has also led the Product Management team for NetBackup.

(As told to Jasmine Desai)

This was first published in December 2010

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.