Online Storage Optimization

At Ocarina, we’re having a great deal of success these days with partnerships, and the buzz around this is being seen in the storage press and beyond. What has happened in part is that now that NetApp has made dedupe table stakes, we are the dance partner that many vendors are turning to, as we stand out as the ones who have the best data reduction technology for online data.

Before we go on, we should acknowledge a recent FUD-spreading post by NetApp’s Dr. Dedupe in which he tries to question whether Ocarina is even dedupe technology. Let’s just clarify, Ocarina offers a solution that includes both content-aware compression and a next-generation form of dedupe called object dedupe. What makes him think that Ocarina is just “resizing photos” is frankly a little beyond us, but in any case, that’s not in any way, shape or form what Ocarina does. However, we are pleased to see that NetApp believes the Ocarina technology deserves attention and is pointing their binoculars at us!

W. Curtis Preston makes a good suggestion in the comments field of Dr. Dedupe’s post. The best way to handle this is obviously to run some tests to see which solution offers better results. As it happens, we have already commissioned just such a study, by George Crump at Storage Switzerland. His results will be published soon, but I can reveal that our results are excellent compared to block dedupe in the filer.

Fundamentally, the value proposition of dedupe technology is that it increases storage capacity, resulting in lower CapEx and OpEx. The difference between the more standard, block-level dedupe such as NetApp’s and our technology is that Ocarina can intelligently extract and analyze the natural semantic objects inside virtually any file.

For example, in a PowerPoint file, a slide is a natural object and so are graphics that appear on a slide. Rather than hashing 4k file system blocks, we look for natural objects like the slide or the graphic, and we hash and dedupe those. This is one reason that we are able to achieve such startling results. For more information on this, please see my earlier post in response to another NetApp blogger, Alex McDonald, who also seemed caught up in semantics around the difference between dedupe and compression. We are planning to release updated white papers in May that reflect all of our latest capabilities. But the bottom line always comes down to, how much data can you reduce?

In general, if we are applied to a data set where our content-aware algorithms recognize most of the file types and objects, we’ll get better results than any other approach. Where the data set is something we do not have specific algorithms for, we’ll treat each file as an opaque object, and our results will trend down to about the same as you’d expect from block dedupe. So worst case, we’re the same, best case we’re as much as 50 times better.

A big advantage of a content-aware approach is that you can set policies that define what gets optimized and when. Block dedupe typically processes all of a volume or none. Since block dedupe has no awareness of content, it has no way to decide whether a given block should be deduped, compressed, or left alone. In a content-aware solution, you can say dedupe files like this, compress files like that, dedupe and compress files older than “x,” and leave these other kinds of files alone, because they are very performance sensitive. We see that kind of file and object level policy control being essential to broad adoption.

This is one key reason many of the largest storage vendors are partnering with Ocarina and including its data reduction technology their overall offerings. We have recently announced partnerships with EMC, HDS, HP, BlueArc, and Isilon, as well as cloud storage provider Nirvanix. This, of course, is good news for customers and the industry in general.

Our success will depend on how well we seamlessly integrate and support their storage hardware. So far, this is going well, and in fact some of the partnerships we’ve already announced are looking as if they may escalate to deeper integrations and levels of partnership.

If we execute well on those, then the data reduction for file storage category may eventually become “NetApp dedupe” versus “everyone-else-with-Ocarina.” That is our goal.

Data reduction for online storage is a hot topic for a couple of reasons. It’s been validated in other parts of the data center. We all know dedupe has become the norm for backups, with disk-based backup targets with dedupe built in rapidly replacing tape. Compression and simple dedupe (called dictionary compression) have also been widely adopted (as WAFS solutions) in the network. The next natural frontier is online file data. Just like Data Domain focused their technology on backups and Riverbed did the same for WAFS, we have optimized our data reduction technology for online data. Each of these use cases has a different design optimum, and if you started out building a solution for online file data, you’d make different design decisions than you would for backups and WAN optimization. In our case, that meant building a whole new kind of dedupe to be able to get the best results on the kinds of files that are driving storage growth.

In my view, we’re really just in the early stages of seeing data reduction make its way to online storage.  Over the course of the next year or two, we expect to see dedupe for online data become as widely understood and deployed as dedupe for backups. During that time, we’ll see lots of debate over different approaches, lots of education about how things work, and a market that will gravitate towards the solutions that both deliver the best results, and which deliver the best performance for end-users and applications.

Here’s a sign of the times. While on a conference call to interview Wikibon’s co-founder Dave Vellante this morning, he, several others on the call, and I were all tweeting about the experience in real time. Welcome to today’s socially networked, version 3.0 of business life.

Wikibon is founded on a new and some might say highly disruptive concept: make IT research free and open source, rather than charging the usual $30K per piece or a subscription, the way analyst firms such as IDC, Gartner, and the rest have always done.

Wikibon is at a .org site, but it’s a for-profit company. Vellante says they initially thought of making it nonprofit and quickly changed their minds. They wanted a business. At the same time, it’s modeled on the mother of all wikis, Wikipedia, and in many respects is very similar, though with the added benefit of being a social network. Vellante says he believes his company is the only open source, GNU licensed research and advisory model out there. This means that virtually all the materials posted on the site can be edited, adapted and used for whatever usage participants choose–including their own for-profit ventures. It’s also one of a handful of specifically IT-related social networks (although one could argue that many social networks are appealing to that very crowd, esp. Twitter and Friendfeed).

Wikibon focuses on the following areas: storage, virtualization, information management (i.e. email archiving, discovery), mobile enterprise, and sustainability. A security portal is also in the works. Information is gathered for the site’s content in a variety of ways. One key method they use are open conference calls to discuss specific topics of interest, followed by a meeting of a core group of analysts to distill the results. The notes are then published to the site, where they can be read and edited by anyone who happens by. They’re also sent out as part of an email newsletter to the group’s membership.

This type of open and collaborative approach is the secret sauce the Wikibon folks say they’re bottling.

“When we started we had all been involved with research companies ourselves, such as IDC and Gartner,” Vellante explains. “The processes were onerous. Documents are circulated, go to editing, then production, and so on. It was just terrible. You could never have gotten six pieces out in 24 hours that were collaborative in nature.”

Is Wikibon a game changer for an industry that treats its white papers and other research as propreitary, and is accustomed to charging a great deal for the materials it generates? Vellante’s answer to this is an emphatic “yes.” After all, his is a business in which they’re happy to see others not only use their materials, but even resell them at a profit–something he says he “welcomes” because it’s one more way that Wikibon is using the power of virality to spread its message.

“I think the existing players are going to have to respond to this,” says Vellante.

Another difference, says Vellante, is that Wikibon doesn’t do “pay for play” white papers. That is, if a vendor comes to Wikibon offering to pay for a favorable piece, they’ll be told that this isn’t how Wikibon works. They may do a paper about a company, he says, but the content will be determined by the collaborative process. They may also suggest doing a case study on a customer instead, although they normally don’t accept payment for that either.

How does Wikibon make money? Well, there are “underwriters”- companies sponsor the site a la NPR. So in essence, vendors are supporting the business. However, says Vellante, they always tell their proposed sponsor that the goal is to ensure that materials are helpful to CIOs.

The overhead seems reasonably low. The company does have a brick and mortar office in Marlboro, MA, but only two three paid employees, Vellante, resident CTO David Floyer and David Butler (updated 5/1). The rest of the core group are using Wikibon as a way of increasing their visibility for their own analyst firms. Vellante says a big selling point for Wikibon is its high calibre core staff, many of whom are former analysts with the big firms like Gartner and IDC, and who are now independents. This includes: Dennis Martin (Demartek), Nick Allen (The Tod Point Group), Bill Mottram (Verdictus), Fred Moore (Horison Information Strategies) and Gary MacFadden.

The Wikibon-ers are taking social media and running with it. At SNW this year, they continually tweeted about briefings they were conducting, and their tweets got a lot of attention, even garnering some business right on the spot. (They also got the attention of this blogger.)

It remains to be seen whether this experiment in open source research/analysis will truly disrupt the existing order of things, or simply add a new dimension to it. Either way, it’s worth signing onto the site, which has a treasure trove of materials to choose from, lots of collaboration, and a real sense of possibility.

An interesting day today in that there isn’t too much in the way of hard news, but plenty o’ commentary floating around the old blog-o-tweet-o-news-o-sphere regarding storage.

First, lots of buzz about Ocarina this past week:

We earned a nice mention in the keynote address at BIO IT World, where Ocarina is in attendance and was a “best in show” finalist. Things are going extremely well there, so stay tuned!

Dedupe Team Up: A post by George Crump of Storage Switzerland on InformationWeek that shows how Ocarina is working with major storage vendors to help them compete with NetApp on dedupe.

NetApp’s Dr. Dedupe questions Ocarina about whether it is doing dedupe in this post “When is Dedupe Not Dedupe?” Kind of an odd one–as W. Curtis Preston points out in the comments field, the “Technology” pages on the Ocarina web site–particularly this one which breaks down the Ocarina ECOsystem process–answer his question. I’d also suggest he read this recent post which was actually in response to another NetApp blogger, Alex McDonald.

Another mention of Ocarina and its recent BlueArc partnership announcement in Wikibon’s Bill Mottram’s summary of the scene at NAB last week. He writes:

“BlueArc: The news on the BlueArc booth was the announcement (April 20th) of their partnership with Ocarina. Availability of the Ocarina Optimizer for BlueArc is scheduled for mid-May.”

And in other news…

InfoStor’s Dave Simpson comments on the Oracle-Sun acqui with some thoughtful insights on his newly revved up blog.

Devang Panchigar has a nice round-up and some commentary on EMC’s recent V-Max announcement on Gestalt IT/Storagenerve.

And, as widely reported, EMC has asked its employees to take an across the board pay cut. And, with the kind of speed we’ve come to associate with blogging, EMCer Storagezilla posted on the 5% cut on his blog right after the announcement was made public.

Happy Tuesday everyone!

“Incent people correctly and you will have real success not just sales.”

If a luddite like Maureen Dowd can do it, then there’s no reason why we here at tech-savvy Online Storage Optimization can’t conduct an interview via Twitter. And who better to try this idea out with than Stephen Foskett, one of the most social media savvy and active bloggers and tweeters in the storage world. Stephen recently joined cloud storage provider Nirvanix as Director of Consulting.

In addition to being a great way to elicit some thoughtful responses, the process itself proved to be an interesting experiment in public interviewing. For example, at several points others retweeted Stephen’s responses, and even jumped in with their own answers to my questions. So, here for your reading pleasure is the complete interview, with all of the 140-character-long questions and answers. If he gave secondary responses, I note it by separating them with an ellipsis (…).

@sunshinemug: Let’s start with this–what’s yr new gig at Nirvanix like so far, and what are you hoping to accomplish there?

@sfoskett: It’s an amazing start-up atmosphere: All hands on deck and everyone pushing in the same direction. It’s a breath of fresh air! … I’m so glad to be focused 100% on enterprise storage and enterprise strategies again. … I actually wasn’t much interested in “just some job”, it was the vision of tiered storage including the cloud that hooked me!

@sunshinemug: Sounds fun! And what are they pushing towards? i.e. What kinds of goals are front and center there right now?

@sfoskett: Goal number 1 is building a new kind of enterprise storage: Managed, off-site, scalable, granular, mobile. (Retweeted by Marc Farley @3parfarley).

@sunshinemug: What’s the potential for CloudNAS in yr estimation?

@sfoskett: CloudNAS is a gateway really - it’s a way to get data in and out of the cloud without re-engineering applications. … One of the hurdles to managed storage is having to rewrite applications to make use of even standard APIs.

@sunshinemug: Interesting– so yr saying that eliminating that hurdle is a reason to use them?

@sfoskett: Every product has hurdles, and in storage it’s all about migration. How do I start using your solution? How do I stop? (Also retweeted by @3parfarley.) … Nirvanix is not trying to make CloudNAS a competitor for NetApp or a Microsoft file server, it’s a bridge.

@sunshinemug: What does your role–Dir. of Consulting–entail?

@sfoskett: I’m an evangelist for storage as a service inside enterprise IT. How do they write SLAs? Make a business case? Set up tiers?

@sunshinemug: So you interact directly w/IT mgrs? Or…? And what are the biggest probs they are facing at this point that you help them solve?

@sfoskett: IT management’s biggest challenge is how to continue to innovate in this budget-squeezed environment. Save money and do better! … It’s interesting to be in the fray more instead of in the ivory tower pretending I’m not selling anything.

@sunshinemug: Say more about that–how has the move out of the ivory tower been and what are you learning?

@sfoskett: I wrote on my blog about trust, independence, and credibility and the consulting business equation. This changes that. … My goal is to connect with enterprise customers and help them, not to build a business on billable consulting hours.

@sunshinemug: In what ways? I.e. How are you walking that fine line?

@sfoskett: I don’t work for sales, marketing, or engineering. I work for the CEO. He understands that the best customer is the right one.

@sunshinemug: Seems like this is where many people are headed–thinking in terms of service rather than “what’s in it for me.”

@sfoskett: There’s a point that you get desperate and say how can I push it out the door. This happens in services as well as SW/HW sales! … I think that means the incentives are wrong. Incent people correctly and you will have real success not just sales.

@sunshinemug: OK one final question: what do you predict will be the most notable trend in the coming year for the storage industry?

@sfoskett: I think this is the year of automated tiered storage. Everyone is going in that direction. It’s where v12n gets real!

For this question, several other responses also came in:

@StorageMonkeys: Notable trend?: Consolidation: budget footprint, workforce, energy.

@storagebod: Notable trend this year will be make and mend…but next year, I think we’ll see some new tech along V-MAX lines appearing … This will be game-changing as we move into a nearly fully automated management paradigm.

@3parfarley notable trends:   moving outside comfort zones to make major leaps in performance and efficiency.

@ianhf: this year = rabbits from hats money wise | end year = cloud storage, API storage and automated tiers - raid will be hygiene.

And, it’s not to late if you have something to say. Simply fill in the comment field below.

We hope you enjoyed this experiment in Twitter-viewing.

About our interview subject: Stephen Foskett has provided vendor-independent end user consulting on storage topics for over 10 years. He has been a storage columnist and has authored numerous articles for industry publications. Stephen is a popular presenter at industry events and recently received Microsoft’s MVP award for contributions to the enterprise storage community. As the director of consulting for Nirvanix, Foskett provides strategic consulting to assist Fortune 500 companies in developing strategies for service-based tiered and cloud storage. He holds a bachelor of science in Society/Technology Studies, from Worcester Polytechnic Institute.

Earlier this week on Twitter, we experienced what it’s like to have real time, public discussions with bloggers, analysts, and journalists about our product. It was a great example of why Twitter is such a powerful tool for talking about technology–not to mention one that can be a little hair-raising.

It started when Martin Glassborow, known to us all as Storagebod, tweeted about his results with the Ocarina Networks Simulator, OcaSim. We were of course aware that Martin was running these tests, but somehow none of us realized he would be releasing his results as they came in — all on this highly public forum. In short, this was one of those real rubber hitting the road type moments for a young company. Either Ocarina’s compression solution was going to knock his socks off, or it wasn’t.

As it turned out, his results were impressive. While he was less than amazed by our results on video files (which we make no claims to compress), he saw a 22% reduction on a fileset that was completely made up of JPEGs. As many people know, JPEGs are notoriously difficult to compress, as they are already compressed files. He noted this in his tweet.

Then things got a little crazy.

SearchStorage reporter Beth Pariseau retweeted his original tweet, while Stephen Foskett reported that he had seen 20% savings using the OcaSim on a set of family photos. At this point, a few analysts jumped in. Steve Duplessie from ESG said he’d be interested in running the OcaSim, while Greg Schulz of Storage IO suggested a challenge. He wrote:

“@pariseauTT @storagebod how about run against some PPT/PDF briefing/sales slide decks c which 1s compresses most, blind results of course ;)”

This raised an issue that we are getting more and more familiar with. One of the biggest challenges we face at Ocarina is getting past the disbelief factor. For so long, there have been rules about what can be compressed or deduped.

Customers will often say, “but everyone knows you can’t compress an already compressed file. It’s impossible.” Or, “I’ve tried dedupe for primary storage, and it hardly makes any difference.”

Much of what we are doing is helping people open their minds to the fact that the impossible is now not only possible, but fully operational and installed with major storage vendors such as BlueArc–which also tweeted during this exchange–and Isilon.

Oh, and Steve and Greg, we’re more than happy to send you the OcaSim. This is getting to be fun!

While it’s true that Spring has officially been underway for a month, only now do we start to embrace the change in seasons. The smell of burgers wafts across backyards, birds warble away in the trees, and baseball players start rubbing Bengay into their sore muscles.

In the storage industry, there are also some changes afoot. Here are a few we’ve noted this week:

Stephen Foskett has a new blog, Enterprise Storage Strategies. The blog resides on the Nirvanix site, where Stephen recently took on the role of Director of Consulting. For those who, like me, are starting to get a little confused by his many online presences, he has posted an explanation of what we can expect from this new blog and how it compares with the other two, Packrat and Gestalt IT. Stay tuned for more about the prolific Stephen F. on this blog very soon.

More changes afoot:

Broadcom has made a hostile bid for Emulex. Dave Simpson’s (also somewhat new) blog has a perspective.

My nearby neighbor Robert Scoble has been hired by Rackspace to build a whole big community thing called Building 43. Some interesting commentary by Rich Miller on Data Center Knowledge. In general, this does raise a question about where companies should draw the line when it comes to social networking experiments that may or may not have anything to do with their bottom line.

And speaking of social networking:

Lots of discussion about Ocarina on Twitter over the last few days. Very exciting times for the company. And it seems that not only Storagebod, but some other bloggers and analysts may be testing it, now that there is word on the street about its 22%+ compression rates on JPEGs.

It’s a beautiful, sunny day in San Jose, a fitting reminder of the amazing planet we celebrate today.  To me, every day is earth day, because for now at least, this is our home. We can treat it well, or we can treat it with less than total regard. It’s our choice.

On this blog, we talk a great deal about greening the data center through reducing your storage footprint. This, in fact, is the rallying cry of our parent, Ocarina Networks. True, “Green IT” has become a bit of a buzzword these days, but without it, we are heading into a downward spiral of waste and overuse of natural resources.

Some nice pieces today that take a look at the advancements in Green IT:

Earth Day: The Best of Green IT - ChannelWeb

This slideshow covers the innovations in greener data center equipment from just about all the major vendors, including IBM, HDS, HP, NetApp, and many others. It also discusses the important contribution that deduplication is making to the goal of reducing power, cooling, and materials. (Thanks @SamMoulton for that tip.)

Do Something with Nothing - Storagebod’s Blog

While perhaps not intended as a specifically “green” sentiment, Martin Glassborow’s post today on his Storagebod blog gets into a topic that is sometimes overlooked in discussions: deciding if you can get away with buying less storage. Sound and very simple advice.

Cisco Launches Carbon Emissions Map, First in San Francisco - Earth2Tech

Just in time for Earth Day, Cisco announced it will launch an “Urban EcoMap.” This web-based mapping tool will provide up-to-date data on emissions and other environmental factors for the city of San Francisco. To me, this is more symbolic than anything else, but may increase awareness of some of the more hidden types of emissions - i.e. it’s not just cars, guys.

Hope you’re able to get out there and enjoy a little of what this amazing planet has to offer.

Happy Tuesday. It’s been one doozy of a week. A heatwave hit Silicon Valley, both in reality and in the more metaphorical sense–in other words, the Sun seemed to be shining all over the place. And it’s also been an exciting week right here at Ocarina, as mentioned in an earlier post.

Here are a few posts that caught my eye today:

VMware’s Cloud Strategy - Ed Saipetch

The Real Impact of vSphere - Chuck’s Blog

Storage Changes in the VMware vSphere 4 Family - Stephen Foskett, Packrat

And, on another topic…

Data Storage Can Become Green - Greg Schulz

And on yet another topic that has been mentioned here and there:

Oracle and Sun - Expanded Thoughts - Steve’s IT Rants

Crazy Larry Strikes Again - StorageRap, Marc Farley

What will tomorrow bring, I wonder?

It’s hard to believe it’s only Monday, because already it’s been a slam dunk kind of week at Ocarina. As mentioned in my earlier post today, Ocarina has announced partnerships with major storage vendors, as covered very nicely by Beth Pariseau over at SearchStorage. Also today, InformationWeek has announced the winners of “The InformationWeek Startup 50: Business Technology Companies To Watch,” and Ocarina is a featured company on the list.

To make this list, companies needed to not only be innovative, but also show that they had technologies that were relevant from a business standpoint in these recessionary times. Out of the 50, Ocarina was one of only five that was additionally profiled. Those five companies from the list “represent the innovative ways these startups solve critical IT problems, cut costs, and improve operations.”

We first spoke with IW last spring, when editor John Foley interviewed Carter George, VP Products and the chief writer on this blog. The video interview is on the magazine’s Startup City blog. Last fall, the magazine named Ocarina “Startup of the Week.”

This latest honor is a sign that InformationWeek is remaining a very relevant news source. It is keeping a keen eye on the types of innovations that have real world applications–cutting costs, keeping IT departments lean and mean, and enabling enterprises to make the best use of their resources. We hope you take the time to read and enjoy entire article.

This week, many of our crew are attending NAB Show, the annual mega-convention of the National Association of Broadcasters in Las Vegas. Why do we need to be there? Well, more and more, the business of making shows and movies has become inextricably linked to the need to store immense files.

To add to the excitement, we are announcing partnerships with two major storage vendors, HDS and BlueArc today. Both are making their storage Ocarina Optimized, thus adding our groundbreaking compression and dedupe to their offerings.

Updates: Beth Pariseau has a very well written and researched SearchStorage article up that explains the partnerships in depth. Definitely worth a read!

On Wednesday, Chris Mellor at the UK Register wrote about the BlueArc-Ocarina partnership as well, with this piece.

With BlueArc, we’re announcing the Ocarina Optimizer for BlueArc. This integrates Ocarina’s data reduction technology seamlessly with the BlueArc Titan storage solutions. CGI animation leader Rainmaker Entertainment–makers of kids animation movies, most recently an animated sequel to the original, Jerry Lewis “The Nutty Professor”–finds this combined solution an ideal method of keeping files accessible during the making of an entire CGI film, which can easily run into petabytes of data.

With HDS, we’re joining the Hitachi Data Systems Technical Alliance Partner (TAP) ISV Program. In doing so, Ocarina helps HDS customers “stretch their IT storage dollars to support the phenomenal growth of unstructured data,” as Asim Zaheer, vice president of product and competitive marketing, Hitachi Data Systems put it.

Exciting times for this industry, as well as our company. We hope to see you at NAB if you’re there.