Online Storage Optimization

Bleary-eyed, the storage industry has begun to wake up from its holiday stupor. VMware has decided to go into the email business. EMC continues to vacuum up talent like a Roomba on a tear through the world’s biggest living room. Meanwhile, the jokers over at Gestalt IT are picking up the “Fake Steve Jobs” meme and running with it. Their version is actually funnier than the original — at least to this blogger, perhaps because I know the players and situations.

The increasingly crowded and competitive Storage Monkeys Top Vendor Blogs contest is about to screech to its exciting conclusion. Voting ends Friday. Front runners are EMC bloggers Chuck Hollis and Storagezilla. Third place at the moment is the HP Storageworks blog, helmed by fearless blogger Calvin Zito. This puts early front runner Marc Farley, founder of the vaunted Steering Wheel Camera Society of America in fourth place. Step on it, Marc! In fifth right now is the Storage Anarchist, Barry Burke, who is just barely edging out NetApp’s Val Bercovici. Well, it ain’t over till it’s over–these things can change fast.

So before the week is out–why not VOTE?

Speaking of which, blogger extraordinaire Stephen Foskett has started a series that delves into the whole vendor blogging question. He has two posts up on the topic, “Vendor Bloggers 1: Why Does It Matter?” and “The Spectrum of Vendor Blogs.” Mr. F cites none other than Online Storage Op as an example of a hybrid “independent-seeming official” blog, but credits us for being transparent about the fact that our parent is a company. No doubt Stephen and I will hash this out further when we give a talk on social media to a group of storage industry pros at The BD Event in Palo Alto next Wednesday.

But wait… there’s yet more news, and this is actual news:

Nexsan and FalconStor are teaming up to try to defeat rival Data Domain. It can get really interesting when two vendors come up with a combination product that serves a larger purpose than they would’ve had if they acted alone. Two pieces on the topic caught my eye this week:

Beth Pariseau, SearchDataBackup - Nexsan and FalconStor gun for EMC Data Domain with Dedupe SG 2 data deduplication backup device

Writes Beth: “Analysts say a series of updates to Dedupe SG — comprised of FalconStor’s dedupe software and Nexsan enterprise data storage systems — put it into closer competition with the 800-pound gorilla Data Domain.”

She quotes ESG’s Lauren Whitehouse, who says that the high-availability config on this combo is a poke in Data Domain’s eye. And Dave Vellante of Wikibon calls the bundle the “best of both worlds” due to the fact that it’s compatible with existing home office systems and reduces data over the WAN–though he questions how it will do in real world deployments.

Joseph Kovar, ChannelWeb - Nexsan, FalconStor Join Forces On Newest Backup Appliance

Joe, for his part talked to Greg Knieriemen at Chi Corp., which partners with both Nexsan and FalconStor who is impressed with among other things the potential inherent in its 10-GB-ethernet option. Hmm, where have I heard that name Greg Knieriemen before?

Well, that’s all for now folks. Maybe next time I talk to you I’ll be checking my zMail.

‘Tis the season for holly hocks, eggnog, and pundits predicting industry trends for the coming year. Apparently, in the storage blogo-tweet-osphere, getting in the holiday spirit means poking fun at one another.

One of the most notable entries is a series of “Letters to Father Christmas” penned by none other than one of our favorite bloggers, Storagebod (Martin Glassborow). They are broken into seven — count ‘em! — installments (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, The Last One). Storagebod really kept Santa busy this year.

The letters contain such gems as this jab at storage giant EMC, which he has dubbed “Elven Magic Company”: “there was that V-MAX announcement; boy did that go down well with all of the other Father Christmases. Still, it was great to see you get down with the Intel Gnomes; still a pity that the FAST magic wasn’t ready to go quite yet…”

And this for another company he calls Hippie Pixies (try to guess…) “I think you need a tete-a-tete with your Hitachi partners; they’ve been sleeping on the job! They either come up with a refreshed array or you’re going to have to go acquisitive to get yourself a top of the line Tier 1 array.”

And lest you think that the Elven Magic folk are balking, he got props from the big elf himself, Chuck Hollis. He writes: “He tells all of us vendors what we need to hear, and does it in a gentle and funny manner.  I think Martin has emerged as one of the most insightful and reasonable voices in the storage blogging world, and we all follow him closely here at EMC.”

Martin G. also sends us and our fellows a little gifty: “A special shout out to the smaller vendor bloggers; what you do is very important, more so than the bloggers of the big guys…Most of you do it very well and with a keen sense of humour! Keep it up and keep me smiling.” We’re working on it, Martin, we really are.

But wait, Mr. Bod isn’t the only one getting nipped by Jack frost. Over at his StorageIO blog, Greg Schulz has penned a post for the season: Behind the Scenes, SANta Claus Global Cloud Story. In it, we learn that “The heart or brains of the SANta operation is his global system operations center (SOC) or network operation center (NOC) that rivals those seen at NASA among others with multiple data feeds. The SOC is a 24×365 operations function that covers all aspects from transportation, logistics, distribution, assembly or packaging, financials back office, CRM, IT and communications among other functions…” Wow. For those of us who thought it was just elves turning wooden cranks to pop out Tonkas, this gives us a whole new view of the ops at the North Pole.

And speaking of cranking out wondrous things, Beth Pariseau over at SearchStorage has put together two seasonal posts that provide for highly entertaining reading. What’s great about this is that those of us in the tweetsphere saw her doing at least some of the research in real-time by asking for feedback through Twitter. Here they are:

First, on SearchStorage - What to buy a geek for the holidays

The best quote in this has to be the following from Taneja Group analyst Jeff Boles, “I’ve rejected the Kindle concept. Have you seen the book prices? It’s an entirely disadvantaged arrangement for what is ultimately a compromised experience in the first place. Wait until NEC makes a levitating robot dog that follows you around and holds the Kindle up for you while you do other things…” Quips Pariseau, “We’ll be sure to include this on our “What to Buy a Geek in 2050″ list.” Well, fine but I’m still putting that robot dog on my wish list…

Then this, on Storage Soup: The storage industry’s holiday wish list

In it, is the “wish list” of Wikibon’s Dave Vellante:

• secure cloud
• A dumpster to haul all my backup tapes that I’ve converted to disk-based backup.
• A primary storage device that optimizes capacity without sacrificing performance. (Dave, we may just have a surprise for you under the tree over here at Ocarina….)
• A virtualization performance guru … make it 5 gurus …

Finally, here’s a video showing John Troyer, Community Manager at VMware (and my costar in the Gestalt IT Tech Field Day overview video). In it, we see him open a veritable bonanza of presents from the VMware community. The normally unflappable Troyer cannot help but be blown away by the outpouring of love and truly enviable gifts he received. Those who know John are aware that this is all entirely appropriate to the level of positive energy he brings to that circle of techies. All the way over here at Online Storage Optimization we can feel the glow.

And with that, we wish you all a Merry Merry Christmas, and to all a good night!

It’s very interesting to write for a blog that is focused on a specific topic–in our case, dedupe for primary–and then suddenly see the whole world wake up to the reality of it all at once. There has been quite the pile-on in the storage blogosphere of late.

So, what has been said so far? First, we had Chuck Hollis on his blog talking about primary dedupe and data I/O density. He makes some great points, but he is seeing the problem in a certain way–in essence, he’s thinking of data reduction may impact performance of primary storage. However, in some cases, dedupe can improve performance, where it allows much higher cache hit rates on highly used shared data blocks (virtual machines are the perfect example) and another fact is that a lot of storage on expensive primary tiers today does not need to be there. It started there, but it’s grown cold.  If you don’t want to create another tier and move files, dedupe gives you a way to create a cheaper logical tier on the storage you already have.

In that case, some trade-off in performance is perfectly acceptable. Ocarina’s solution for deduping primary storage  gives you the choice of deduping in-place (creating a logical tier 2) or doing dedupe-and-migrate as a single atomic operation, shrinking colder data and moving it off of Tier 1 storage in one step. In fact we just announced that Ocarina is now part of the EMC Velocity Technology and ISV Program, giving EMC’s Celerra a major edge over NetApp for both in-place dedupe on Celerra for primary, and for dedupe-and-tier.

A string of comments on Chuck’s blog included some heated exchanges between Chuck and arch rival NetApp’s bloggers Vaughn Stewart and Kostadis Roussos.

In response to the post, Hu Yoshida at HDS put in his view, which is that he essentially agrees with Chuck on this question. His main point is that dedupe for primary isn’t a panacea. True enough, but as Hu himself has noted in an earlier post, there’s a great advantage to integrating it when you’re already taking advantage of these other tiering, storage virtualization, and provisioning options.

Then finally, EMC Avamar’s Steve Kenniston covered a great deal of ground , and in fact ended up highlighting two key points that are complimentary parts of Ocarina’s strategy. First, we want to get as many deeply-embedded design wins with NAS and file system vendors as possible - meaning that a common “language of dedupe” would be spoken across multiple vendors. Second, we’re developing an end-to-end dedupe strategy, where a file that is deduped early in its lifecycle can be kept in its most compressed form as it moves throughout storage workflows.

Once deduped, data should never have to be rehydrated unless it is being accessed by users and applications. For all the rest of the classic storage workflows - backup, replication, data distribution, archive - there’s no reason for data to have to be rehydrated as it moves across tiers, platforms, and vendors.

Examples would be supporting replication of optimized (deduped and compressed) volumes, allowing deduped volumes to be backed up without rehydration (regardless of what the backup target is), and seamless integration with NDMP so that NDMP backups and restores can work work transparently with deduped files, without even knowing that they are deduped. The first wave of dedupe products were not only vendor-specific (NetApp Dedupe) but also tier specific (dedupe for backup, dedupe for primary, etc). While there are cases where a customer’s need for data reduction is urgent enough to deploy those point solutions, the real win is when dedupe is common and compelling across vendors and tiers.

Now, some people might say, “I already bought a dedupe appliance for my backup target, do I really need dedupe anywhere else?” But the fact is, if you dedupe upstream from your backup appliance, you not only save money on primary storage,  you still get benefit from your backup appliance. Backups are repetitive - you back up a volume every day, either full or incremental. So even if you have already deduped your primary volume, by backing it up every day, you are creating more duplicates in the backup target. The dedupe appliance will find those and take them out. If the primary volume has already been deduped, though, your backup data set will be smaller, and the work that the backup appliance has to do will be faster. The benefits are cumulative - if you get 5:1 on your primary data, and then back that up every day for a month, you may end up with 100:1 savings in your backups instead of today’s 20:1.

Interestingly, EMC has all the pieces here. And actually we can show how this works in an HDS environment just as well, which we may do in a later post. If you run Ocarina to dedupe your primary file store on Celerra, Ocarina can do the following:

* Optimize some primary files, identified by file type, right where they are on the fastest primary tier.   This may allow those files to see better cache hit rates.
* Optimize other files and move them to another volume on the same Celerra or another Celerra, perhaps a volume with SATA instead of Fibre drives.  Because Ocarina uses EMC FileMover stubs, this means that we can create a much larger global namespace on Celerra than a simple Celerra volume would support.
* Optimize all files in policy and post them to EMC Celerra for archive in an optimized form (deduped and compressed).
* Optimize whole volumes, and then back up those volumes to EMC Data Domain, where additional dedupe will take place as backup after backup creates more dupes in the backup target.

All of this can be done - on EMC, HDS, and other vendors - as true “end to end” dedupe, where data only gets rehydrated where its needed for a live application or user I/O request.

This year’s VMWorld ended with a bang–as rock legend Foreigner played for a crowd of 10,000 virtualization fanatics at the Yerba Buena Center in S.F. This really was one heck of a jampacked and exciting event. It is clear that this is becoming one of the key shows for the virtualization and storage markets. There were two–count ‘em!–EMC acquisitions. The big buzzword was “cloud.” There were more than a few product launches, and even a few new companies on the scene.

On a more personal note, this was a great way to connect with a whole lot of storage bloggers that I’d met online but not in person. The storage “tweetup” on Tuesday evening drew a crowd that could be called a veritable who’s who of the storage blogging community, including Marc Farley, Stephen Foskett, “HPStorageGuy” Calvin Zito, NetApp’s Alex McDonald, Dave Vellante of Wikibon, Ed Saipetch, “Storagenerve” Devang Panchigar, and others. I also met EMC bloggers Dave Graham and Chad Sakac, and NetApp blogger Vaughn Stewart. What a talented community!

One thing that made VMWorld stand out was the diversity of companies represented there–from huge heavyweights such as EMC to newish tech cos. like Veeam to startups just coming out of stealth. Perhaps the hottest of these new entrants was EvoStor–which I mentioned in yesterday’s post. As Storage Switzerland’s George Crump says in a recent VMWorld dispatch on Byte and Switch:

“EvoStor, a VMware only, first that I have seen, storage solution. This is more than focus, this is exclusive support of a single environment … The system is basically a storage grid similar to other scale-out models like LeftHand or Isilon but again, focused on VMware.”

Also this week, EMC went on a shopping spree. Tuesday, we learned it had picked up e-discovery specialist Kazeon. As Stephen Foskett notes in Gestalt IT, “EMC will likely integrate the Kazeon technology with their SourceOne archiving and discovery platform.” He also says that while EMC probably paid a fair price for the technology–just $75 million–it’s not without its challenges.

A day earlier, on the first day of VMWorld, EMC announced it had acquired “application image management” company FastScale. On his blog, Chuck Hollis describes this as the final piece in a puzzle that it has been assembling for some time with Ionix. He calls their method of prepping images for virtualization “predupe,” reducing footprint in memory.

As Beth Pariseau writes in Storage Soup, the FastScale acqui is a sign that EMC and VMWare are acting more like two interrelated companies. The more logical acquirer would be VMWare, since this is a server virtualization company. This could signal a shift in focus away from their purely hands-off approach that meant that it sometimes even signed agreements with EMC arch-competitor NetApp. Time will tell where this is leading.

There’s much more to say about the show, but this will have to do for now.

Storage giants EMC and NetApp have been fierce competitors for some time now, and so it’s not unusual to see dueling blog posts between them. However, nothing beats the back and forth blog posts we’ve been seeing since the bidding war began to heat up between them for deduplication specialist Data Domain. And even in light of today’s news that the Data Domain board is rejecting EMC’s offer in favor of NetApp, the dance might still continue, at least according to eWeek’s Chris Preimesberger.

While some of the bloggers are throwing zingers at each other over which is the better acquirer/place to work, the net effect, I think, has been a very interesting discussion around deduplication–which is fast becoming recognized as the most significant technology in these “do more with less” times.

Here are some of the most notable entries, first from EMC:

Storagezilla - Data Domain Plus Plus

In this post, ‘Zilla argues for the Data Domain acqui by laying out a potential scenario in which its deduplication technology becomes the “cornerstone” of a complete backup and recovery division for EMC.

Chuck’s Blog - Why do I work for EMC?

In response to a letter to Data Domain employees by Joe Tucci, CEO of EMC, Chuck Hollis writes quite eloquently about what makes EMC a great place to work.

The Backup Blog - Putting the Pieces Together: Deduplication Technologies

In this blog, Scott Waterhouse gets into the question of what is needed to get beyond tape backups. Or, as he succinctly puts it, “backup sucks.” (For our take on backups, please read Carter George’s post “Backup to the Future.”)

NetApp, meanwhile, came out swinging with these posts:

Jay’s Blog - Deduplicating Customer Choice

In this, he lays out an argument as to why the merger will make NetApp a stronger player, with an across the board deduplication solution for both nearline and backups. He also argues that EMC would have the overwhelming market share in deduplication for both VTLs and backup appliances if it won Data Domain. An interesting read overall.

Exposed Blog - It’s Always Calmest Before the Storm

In this, NetApp blogger Val Bercovici goes after EMC on the question of which is the better place to work–complete with a comparison table.

Extensible NetApp Blog - Ex-Chain Smokers

In this post, NetApper Kostadis Roussos posts a video that has also been making the rounds on Twitter from an all-hands NetApp meeting immediately following the initial announement that NetApp would be acquiring Data Domain. In it, Frank Slootman, CEO of Data Domain talks about why the acqui makes sense to him. He also makes some disaparaging remarks about EMC.

Exciting news broke this morning in the storage world - EMC has made a $1.87 billion counterbid for Data Domain, against NetApp’s original $1.5 billion offer in late May. We will have some commentary on this soon, but meanwhile, it seems worthwhile to at least aggregate the opinions that are starting to churn out of the real-time blogosphere on the subject.

One very quick piece of analysis: If nothing else, we hope that people take from this news the immense importance that deduplication is starting to take on in the storage industry. It is slowly dawning on everyone that this is the best way to save on storage costs, both in the near-term and over the long haul. It serves to reign in both CapEx for storage and OpEx for power, cooling, people and other costs. In short, dedupe is hot!

As our lead blogger Carter George pointed out when the news of the NetApp bid went public, the acquisition would mean that, “…one vendor, NetApp, has the market leadership position in both dedupe for online with NetApp Dedupe and dedupe for backup with Data Domain.” He also raised the question how EMC and others would respond. Well, now we know!

First out of the gate with the instant analysis was Steve Duplessie.

Steve’s IT Rants - EMC Trying to Outbid Netapp on Data Domain. In this post, Mr. D. argues that EMC doesn’t need what Data Domain has, and that in essence it’s just a way to mess with its prime competitor. Later, Steve spelled out his perspective even more overtly with the following tweet: “new conspiracy theory - EMC trying to make Netapp spend more cash - evil but excellent if true.”

Chuck Hollis of EMC gives his perspective as an insider:

Chuck’s Blog - EMC Makes Surprise Play for Data Domain - In this, Chuck explains that this supports the EMC vision of deduplication everywhere. Beyond that, he doesn’t get into any detail as to where it will integrate or the fate of EMC’s similar products, such as Avamar. EMC’s Joe Tucci, in a conference call today, also declined to give specifics on the fate of that product line.

Stephen Foskett, Gestalt IT - EMC Takes on NetApp for Data Domain. Stephen argues quite compellingly that Data Domain might actually be a better fit for EMC than NetApp, because in the long-run it has a better technology than Quantum, which could still be cut loose.

As he puts it: “Their Quantum-based VTLs are expensive and enterprise-focused, while a new Data Domain-powered line might have broader appeal.”

Marc Farley, StorageRap - EMC versus Netapp for Data Domain - call notes

More than notes, this also includes his perspectives.

More to come. Meanwhile, the question remains - who will prevail in this bidding war?

Chuck Hollis at EMC’s post today is an early Valentine’s Day card to the small but growing storage blogger community. Listing some of the most active and well-respected bloggers, he gives props to all of them, underlining their unique qualities. This post couldn’t have come at a better time, as there has been a small amount of mud slung here and there of late among this rough and tumble crowd. In this post, Chuck hones in on what’s really important, which is that the storage blogging community has become a fertile area of discussion and debate among a group of highly thoughtful and intelligent folks.

Chuck himself is an example of an opinionated and smart blogger who can be harsh at times but who will also back off of a position if it seems appropriate, as he recently did when confronted with some misunderstandings about an argument he made about “Frankenstorage.”

It is an exciting time for the storage industry, which is shedding its image as a place where only the unwashed, die hard techies survive and becoming an attractive career option for a wide variety of engineering students, and many others with skills and enthusiasm for this crucial aspect of IT. (Writers, for example.)  No doubt this is really just the beginning of what will be a large and vibrant social network that shares its knowledge for the greater good of all.