Online Storage Optimization

In what seems to be a theme this week in Online Storage Optimization–we’re all about what’s next, and what’s “out there” in the stratosphere. We’re also not beneath making as many pop cultural sci fi references as we can. That’s just the way we roll.

Analyst George Crump had a nice piece in Networking Computing (the new umbrella publication that has eaten Byte and Switch for breakfast). His article, “Deduplication Moves Beyond Deduplication” gives a good overview of the state of dedupe and where it might be headed. For those of you who have been reading this blog regularly, we’re predicting a major leap in innovation around dedupe.

We see an end-to-end or “global dedupe” as the obvious next step for this groundbreaking technology. As Carter George, our lead writer noted a recent post: we see a future in which dedupe will have post process architecture, and will find duplicates across multiple nodes and multiple storage pools. It will also integrate a combination of technologies–most likely the best will be those that have both dedupe and content aware compression. This has been the direction that this blog’s parent Ocarina has taken, and it has proven a very wise strategy.

As Crump notes, Ocarina differs from other solutions in that it “adds deduplication to the process as well as content specific optimizers that provide a greater understanding of the file formats being processed.” The next wave of dedupe, he predicts, will still involve two or three different solutions within an environment. No doubt there will be a path that takes us from here to the ideal of simplification we envision. Meanwhile, they will continue to add features that ensure that the investment in dedupe is amply leveraged.

I’d say that one great example of that is the ability to easily facilitate migration among multiple tiers of storage–in other words, migrating data and tracking its location as part of the optimization process. A solution that can do that in addition to reducing file sizes offers notable savings and simplification on both ends. Oh, did I mention Ocarina already does that?

George Crump has a piece in Byte and Switch today that poses an important question: “Can we get to a single point of deduplication?” This is a question that we have taken up in one form or another in some of our recent posts, such as this one and this one.

In the article, Crump asks the question in another way: “… can you have all your data tiers; primary, archive and backup deduplicated by a single engine?”

In light of the recent focus on deduplication, this in my view is a question that really does need to be raised. For how long will the industry to silo out these different tiers for its deduplication solutions? And how much sense does it make to rehydrate data every time you move it, in order to once again deduplicate it? Not a lot.

Crump writes: “The current deduplication vendors could work on building out their solutions to either scale up into primary storage performance (see Data Domain’s DD880) or they could move their existing data duplication technology into other markets; see the increased speed of Ocarina Networks and Permabit as well as their move into cloud storage.”

At the same time, as we’ve pointed out here, online storage is quite a bit different than backups and so far at least, none of the successful backup dedupe vendors - Data Domain, Diligent, Quantum, etc. have been able to break into it. Rather, it is NetApp and Ocarina who have been the trailblazers.

Crump makes another key point:

“NetApp and Ocarina could continue to enhance and improve the re-hydration speed of their technologies to make read performance a non-issue, making primary storage a viable platform. Ocarina can already maintain the deduplicated format as they move through tiers, so landing on backup or archive disk would simply be another move for them.”

This is an interesting observation, and one that is often missed in reporting on both of these solutions. We look forward to seeing more debate and discussion on this issue, which was well kicked off with this piece.

Even in these recessionary times. Green IT continues to gain momentum. This, according to a recent survey from Symantec, which states: “Virtually all the companies surveyed are discussing their Green strategy. They are not just talking, either. Green budgets are on the rise and IT is more than willing to pay a premium on energy efficient products…”

Citing this survey, analyst and consultant George Crump argues in an article in Byte and Switch today that energy savings are possible in many areas of IT, including storage.

Update: also found this article on TechTarget that’s a little incomplete, but worth a read nevertheless — Green storage best practices control costs, increase energy efficiency.

Writes Crump:

“I admit it, I was wrong. I assumed that green IT initiatives would be put on the back burner as we slogged our way through the current recession but according to a recent survey by Symantec apparently just the opposite is happening. Will green storage be a key part of the green IT effort?”

The answer could well be “yes.” In fact, there is a green side to data reduction, which as many are aware has been a very hot topic lately–especially in light of the recent battle between storage giants NetApp and EMC over archive deduplication leader Data Domain.

Crump argues that one of the greenest forms of storage is tape, which doesn’t require any power or cooling.

He also notes: “While disk archives that leverage clustered storage will have some difficulty in powering down drives since data is distributed across nodes in the cluster, they could power manage the nodes in the cluster itself. Of course, they gain power efficiency through greater density per node - bigger drives, compression and deduplication.”

Crump is discussing archive storage here, and cites Permabit as a solution for that realm. There is no doubt that data reduction for primary storage is a particularly key way to reduce one’s storage footprint. Primary storage, by its very nature doesn’t lend itself to tape, because the data is being kept online. Dedupe is therefore becoming a must-have, with all the leading vendor offering some type of it for their primary storage. For those who would like notably better results, a next-generation solution such as Ocarina is a way to further shrink files. The results–lower power, cooling and space usage. Green indeed.

Summer is almost here, and despite rumors of a recession, the malls are filling up with shoppers seeking bathing suits, sunblock, iPhones, and other de rigueur gear of the season. Here in storage land, the latest industry news continues to amaze, amuse and baffle.

Here are a few headlines that caught our eyes:

Data Domain Board Rejects EMC Takeover Offer - Computerword, Lucas Mearian

HDS Expands Thin Provisioning - Search Storage, Beth Pariseau

VCs and IT Execs Discuss IT’s Brave New World in Boston - Storage Soup, Beth Pariseau

VMWare and HP Announce Co-developed Plan - Byte and Switch, Mike Fratto

And, for a little summertime diversion, one 3Par storage rapper’s response to the strange and wondrous EMC-NetApp-Data Domain tale:

3P’s Open Rap to Data Domain Employees - Storagerap, Marc Farley

Some very interesting posts and stories out there this week. Here are some that caught our attention.

Small businesses find big value in Twitter - Financial Times

Storage Resource Analysis - Gestalt IT/Storagenerve Devang Panchigar

Cloud Storage Options Multiply - Byte & Switch

Woz Staves off Elimination - Computerworld on “Dancing with the Stars” (admit it, you’re following the blow by blow of this story)

Quick Calculations Show that Enterprises Can Save Over a Million Dollars Using Archival Storage - Jerome Wendt, DCIG

We hope you enjoy your reading today!

My company Ocarina Networks is one of Network World’s “10 Storage Companies to Watch.” We’ve been given this honor before, by Byte and Switch in 2007. This is not just to toot our horn, but to mention that more and more, we’re confirming our hunch that primary storage is where there is an immense need for new and innovative optimization solutions.

As I mentioned in an earlier post, the kind of data that’s driving much of today’s storage growth–files–demands a file-aware solution for shrinking them down. It’s clear to me that there is an emerging set of opportunities in this space, and we are only beginning to see where this will lead.

Byte & Switch has once again released its “Top 10 Storage Startups to Watch” for 2008, and it’s definitely worth a read. My company Ocarina Networks was on that same list last year, and so I can say with confidence that they got it right at least once before. 

As reflected in this year’s list, data reduction technologies continue to be hot. Makes sense in a down economy that anything that increases capacity will continue to get budget dollars. As we’re finding, dollars for stuff like Ocarina is already there in every data center’s budget – it’s just listed as disk expense. We’re not only ahead of our revenue goals for our storage optimization product launched in April, but we’re having to triple the size of our sales force to keep up with demand. 

If you have planned to buy 100 TB of disk, and can spend half as much for an optimization solution that shrinks your files that means you don’t have to buy any disk at all. A win all the way around. While Ocarina started out with wins in large web sites – where the fastest year-to-year storage growth is taking place – we’re now seeing installs in life sciences, energy, movie studios, and finance.  

The chief takeaway from what I’ve seen: some nice-to-have new technologies may be facing a tough summer with an economic downturn, but data reduction scores high on both saving money and green IT, and is likely to stay strong, or maybe even move up in priority, during a down cycle in storage spending.