Nice to see Dave Simpson from InfoStor getting interested in the subject of dedupe for primary. Today, he had a new post on the topic, along with a plug for a webcast he’s doing on the topic with Noemi Greyzdorf of IDC. In the post, Dave makes the point that people are starting to muddy the waters when it comes to terms such as “dedupe for primary”–something that NetApp and others are popularizing at the moment. He notes that quite often, the term covers a lot more than just dedupe, and can include compression, single instancing, and other capacity optimization methods.
Dave makes some great points. For example, there are a lot of possible trade-offs you can make between performance and space savings when you are looking at data reduction technologies. My company Ocarina gives you the choice of superfast lightweight compression, subfile dedupe (fast), object dedupe (medium fast, but better results) and content-aware compression (slow, but great results).
You can turn any of these things on or off, and you can pick the optimizations you want by policy not only by volume, but right down to the individual file or file type.
For example, you could say, “I want wire speed lightweight compression only for my MS Word docs, but use object dedupe and content-aware compression on any PDFs you find in this homeshare volume; don’t do anything to this database volume, and do everything you know how to do to shrink this archive volume.”
There’s no one right answer. The hotter your data, the more likely it is to be true primary data with lots of users reading it and writing it in real time. That means you have to be that much more careful about what data reduction algorithms you apply to it. The colder the data, the more aggressive you can be. Some data reduction vendors can only deploy in one tier of storage because they only have one kind of tool. If all you have is wire speed compression, you can do hot database data, but you won’t be able to shrink most kinds of data at all. If you have a heavyweight dedupe and compression solution, you may be stuck in archive or backup, because you’re not fast enough for primary. If all you have is a hammer, then all the world looks like a nail.
At Ocarina, we want to give you the toolbox, and the ability through policies to intelligently decide which tools to use for each file, volume or data set. This means you can match your data reduction strategy to a file’s performance requirements and get the best fit for each part of your storage – primary, nearline, archive, etc. Multiple tools, multiple options, and a far more customized solution. At the end of the day, having a really really great hammer is only marginally useful if what you need to do is cut something in half … like your storage budget.
