Online Storage Optimization

It’s been a tough year — a worldwide recession, a sluggish housing market, rising unemployment … and on top of all that, the tarnished image of one of sports’ most squeaky clean players. Well, actually, there have been some bright spots. As DCIG blogger and storage analyst Jerome Wendt notes while looking back at the past year, “Deduplication is the Big Success Story of 2009.”

Wendt writes: “Deduplication is arguably one of the most notable trends of 2009 as it has been widely adopted by users after bursting onto the scene just a few years ago and has grown to be included in both software and hardware products.”

Wendt focuses on dedupe for backups, where there has been much publicized activity over the past year. The big storage story of 2009 was of course the battle between storage titans EMC and NetApp over backup dedupe specialist Data Domain. He cites an industry survey from SearchDataBackup that indicates that 41% of enterprises either are or are seriously considering dedupe to control data growth and costs. He also notes that the despite the predicted demise of Quantum, that dedupe company remains strong.

Dedupe for backups is one part of the cost reduction puzzle. Another part is to reduce data at the source, in primary storage. This is of course the specialty of this blog’s parent Ocarina, which implements a unique combination of content-aware dedupe and compression to achieve startling results. It focuses on the very types of unstructured data that are driving storage growth today–emails, images, documents, and so on. The company has been partnering with almost every leading storage provider, including HP, EMC, HDS, BlueArc, and Isilon. Another  leader in this space is NetApp, which has a strong dedupe for primary offering that has also garnered a great deal of attention.

Here’s the thing, the economy might be slowing down, but data growth continues apace. This is one reason that the storage industry has been thriving this year. But rather than standing still, what is spells is a concerted effort to keep that data under control. As Wendt notes, another of the year’s big trends is cloud storage, which offers companies more flexibility for storing some percentage of their data. I would also add that virtualization has taken a huge leap forward, not only in terms of the technology itself, but also in terms of adoption over the past year. Yet another way to attack the problem.

So if 2009 was all about dedupe for backups, I’m going to guess that 2010 will be very much about data reduction at all points on the data life cycle. What do you predict?

Image: Gizmodo

Seems New Year’s is coming early this year. Across the pond in the UK, Ocarina and our partner BlueArc have decided to hand out bottles of bubbly to all and sundry. In short, anyone suffering from a storage hangover–that is, any company that is saddled with more than 20 TB of unstructured data–is eligible for a free bottle of champagne. Talk about the hair of the dog.

The bottle giveaway started last week, and runs through the end of November. It’s open to any companies across EMEA. Here are the basics: send us a sample data set, and we’ll dedupe and compress it by 30-80%, losslessly of course. If we fail, we’ll give you the bottle anyway. (The one thing we don’t accept are cryptologists trying to game the system. This should be a sample data set representative of your company’s main storage repository.) So it’s kind of a no lose situation, unless of course you don’t like champers. If that’s the case, just send the bottle on to this blogger, and she’ll be sure to give it to the needy. (That is, her friends and a few others she’s inviting over for brunch.)

We’re not talking about the cheap stuff here, either. I checked with the team in London, and have been told that it will be Taittinger or better quality.

So, why the sparkly stuff and not, say, a Starbucks gift certificate or some silly piece of shwagola? Because, say our UK marketing team, too much data shouldn’t be a source of misery. It can even (gasp!) be something to celebrate, if you’re equipped with the right tools. In this case, this is a combo of the BlueArc Mercury or Titan and the Ocarina ECOsystem for advanced dedupe and compression. Roll these two together, and you’ve got a delicious mimosa of a storage drink. Slainte!

Here’s how to win: Send an email with “FestiveGiveAway” in the reference line to: ukinfo (AT) ocarinanetworks.com or ukinfo (AT) bluearc.com. EMEA-based companies only, please.

Many organizations are struggling to manage their storage in the face of a massive upsurge in the amount of data that must be stored and made accessible. To save on primary storage costs, it’s imperative to make sure that it is being deployed in the most efficient way possible. As we’ve already been discussing at length on this blog and elsewhere, storage tiering is one key way of approaching this.

A few weeks ago, IDC co-presented a webinar with BlueArc and Ocarina on just this topic. It is now online and available for your viewing/listening pleasure at the link above.

Here are some of the highlights:

Noemi Greyzdorf, Research Manager, Storage Software, IDC discusses the process for putting a successful tiered storage infrastructure in place. She gives information on how significant savings can be gained in both acquisition and operating costs.

Victoria Koepnick, Sr. Manager, Product Management, BlueArc talks about how her company has developed technologies to fulfill these goals. She shows how intelligent tiering that is policy based can save on storage costs and ensure ease of use. Using the example of the recent death of pop star Michael Jackson, she talked about how with dynamic read caching, the many images could have been immediately pulled from the archive so that successive accesses to the files would’ve been immediate under such emergency circumstances. Great example!

Eric Scollard, VP of Sales at Ocarina gets down to brass tacks on the results from storage optimization and tiering. He shows how having a toolbox of data reduction techniques at one’s disposal can make a vast difference when it comes to the amount of money spent on storage.

We hope you enjoy the webinar, and please feel free to comment on it below!

As many people know, Ocarina Networks has been living up to its name lately. It really is becoming a “network”-oriented company, inking partnerships with just about all of the top storage vendors–HP, BlueArc, Isilon, and so on and so forth. This is great news for storage customers, who can now depend on the very best in data reduction, slashing storage costs.

For those who want to get a quick and entertaining hit on how one of these partnerships works–this one with BlueArc, might we suggest this new animated demo on the Ocarina site? This demo offers a case study in how a world-class CGI animation studio, Rainmaker Entertainment, deployed BlueArc storage with Ocarina to achieve astounding compression results. (For more on this, you might also want to take a look at our Q&A with Shmuel Shottan, CTO of BlueArc from last February.)

And for more on how Ocarina is joining forces with the top storage vendors to help media and entertainment companies maximize storage capacity, check out these recent news stories:

Beth Pariseau, SearchStorage - Ocarina deduplication and Isilon clustered NAS help visual effects studio archive images, cut costs

Debra Kaufman, Studio Daily - VFX Companies Lower Storage Costs with Ocarina

Bryant Frazer, Studio Daily - Q&A: Carter George, VP of Products, Ocarina Networks
Downsizing Storage Requirements for Post-Production

Happy Friday everyone!

As I mentioned in a blog post yesterday, the media is beginning to swoop down and take note of yesterday’s announcement that HP is now an official reseller of Ocarina. It’s pretty darn big news.

Here are some of the better articles that have appeared. I’ll add more as they come in, so keep rechecking this post. In fact, if you’re worried about missing something, I suggest you subscribe to the Online Storage Optimization news feed by clicking the RSS symbol above.

New addition: Paul Shread, Enterprise Storage Forum - HP Sees Opportunity in Data Deduplication

Chris Mellor, The Register (UK)  — HP Makes Ocarina Music

As Chris puts it: “Ocarina has similar partnerships with BlueArc, EMC and Isilon. It looks almost inevitable that every other filer supplier must be looking at the Ocarina product and thinking a reseller deal might be a good idea. Otherwise, it could lose sales to the competition when a lot of image-type data is being stored.”

Raju Shanbhag, TMCnet - Ocarina Networks’ Ocarina ECOsystem to be Resold by HP

Says Raju: “According to the company, this solution has the intelligence to extract and analyze the component parts of virtually any file … As the amount of information companies produce on a daily basis are increasing phenomenally, companies are looking for highly scalable storage solutions that efficiently and cost-effectively manage these volumes.“

This week, Ocarina sent a contingent to SIGGRAPH 2009, a conference and exhibition that draws computer graphics, animation, post-production, and imaging specialists from around the globe. Lots of toys to look at, and stories to hear. We are there through the end of this week, spending time in the booths of two of our partners, Isilon Systems and BlueArc. We’re so tired we almost feel like we’re getting wall-eyed, but every time we look up we see more folks who have gotten in their cars and driven here. There are so many new talents here, it’s really a group of incredibles. As far we can tell, the blue sky is the limit to what they can achieve.

If we were ever wondering whether next-generation data reduction technologies are needed in this era of animated, 3-D movies, the stories we’re hearing at this show convinced us that without some type of data reduction we could heading for an ice age. Data growth for this industry is a clear and present danger.

This has been a great week already for BlueArc, which announced that Starz Animation used its storage for a new animated feature, “9″ from Focus Features. For those who are here, we encourage you to come tomorrow, as cinematographer John Hickson, IT Systems Architect at Starz, and the champion of the Studio Sys Admins user group is going to speak in the booth about the installation and the movie itself. *Thanks Julie Herd Goodman for that correction.*

One of the Starz folks, Terry Dale, is quoted in the news release as follows:

“During the production of Focus’ 9, it was critical that we have a storage system in place that would handle the massive demands of the three films we were animating concurrently. 9, in particular, has characters — specifically the machines and the seamstress — that generate enormous data sets during rendering,” said Terry Dale, VP, operations, for Starz Animation Toronto. “The BlueArc Titan storage system gave us the performance we needed, the reliability we required and enabled us to complete our production on 9 without disruption.”

And in case you haven’t seen it, here is the preview for this amazing looking Tim Burton extravaganza:

If you’re trying to figure out how to do more with less when it comes to your storage, I’d strongly suggest you participate in an upcoming Webinar, “How to Use Storage Tiering to Create Cost Efficient Storage of your Online Data.” It will take place on August 5 at 9 a.m. PDT and 12 p.m. EDT.

The time to register for this event is now, and visiting the above link will walk you through the steps to do so.

Sponsored by Ocarina and BlueArc, the webcast will delve into the practicalities involved in achieving storage efficiency. The focus will be on use of intelligent storage tiering and capacity optimization technologies to reduce data footprint and effectively manage data center storage resources.

Featured panelists will be Noemi Greyzdorf, Research Manager, Storage Software at IDC, Victoria Kepnik, Sr Product Manager at BlueArc, and Eric Scollard, VP of Sales at Ocarina Networks.

As we have discussed here in the past, storage tiering can be one important way to reduce disk costs. As Carter George put it in a recent post: To keep up, you have to cut the flab out of your storage. This, too, calls for a two-pronged approach. … This means doing a better job of tiering, and keeping files only as long as you really need them … The second part is the “exercise” element of keeping your storage slim and trim. That is, run a storage efficiency tool –may we suggest Ocarina as one example — that will efficiently trim the fat out of your data. That kind of combination means that you really can tighten your belt on your storage budget.”

And while storage tiering and capacity optimization are frequently discussed in storage publications, this is the first time I’ve seen this particular group of storage experts come together and take a serious and significant look at the details of how this can best be achieved for your enterprise. We look forward to your participation.

At Ocarina, we’re having a great deal of success these days with partnerships, and the buzz around this is being seen in the storage press and beyond. What has happened in part is that now that NetApp has made dedupe table stakes, we are the dance partner that many vendors are turning to, as we stand out as the ones who have the best data reduction technology for online data.

Before we go on, we should acknowledge a recent FUD-spreading post by NetApp’s Dr. Dedupe in which he tries to question whether Ocarina is even dedupe technology. Let’s just clarify, Ocarina offers a solution that includes both content-aware compression and a next-generation form of dedupe called object dedupe. What makes him think that Ocarina is just “resizing photos” is frankly a little beyond us, but in any case, that’s not in any way, shape or form what Ocarina does. However, we are pleased to see that NetApp believes the Ocarina technology deserves attention and is pointing their binoculars at us!

W. Curtis Preston makes a good suggestion in the comments field of Dr. Dedupe’s post. The best way to handle this is obviously to run some tests to see which solution offers better results. As it happens, we have already commissioned just such a study, by George Crump at Storage Switzerland. His results will be published soon, but I can reveal that our results are excellent compared to block dedupe in the filer.

Fundamentally, the value proposition of dedupe technology is that it increases storage capacity, resulting in lower CapEx and OpEx. The difference between the more standard, block-level dedupe such as NetApp’s and our technology is that Ocarina can intelligently extract and analyze the natural semantic objects inside virtually any file.

For example, in a PowerPoint file, a slide is a natural object and so are graphics that appear on a slide. Rather than hashing 4k file system blocks, we look for natural objects like the slide or the graphic, and we hash and dedupe those. This is one reason that we are able to achieve such startling results. For more information on this, please see my earlier post in response to another NetApp blogger, Alex McDonald, who also seemed caught up in semantics around the difference between dedupe and compression. We are planning to release updated white papers in May that reflect all of our latest capabilities. But the bottom line always comes down to, how much data can you reduce?

In general, if we are applied to a data set where our content-aware algorithms recognize most of the file types and objects, we’ll get better results than any other approach. Where the data set is something we do not have specific algorithms for, we’ll treat each file as an opaque object, and our results will trend down to about the same as you’d expect from block dedupe. So worst case, we’re the same, best case we’re as much as 50 times better.

A big advantage of a content-aware approach is that you can set policies that define what gets optimized and when. Block dedupe typically processes all of a volume or none. Since block dedupe has no awareness of content, it has no way to decide whether a given block should be deduped, compressed, or left alone. In a content-aware solution, you can say dedupe files like this, compress files like that, dedupe and compress files older than “x,” and leave these other kinds of files alone, because they are very performance sensitive. We see that kind of file and object level policy control being essential to broad adoption.

This is one key reason many of the largest storage vendors are partnering with Ocarina and including its data reduction technology their overall offerings. We have recently announced partnerships with EMC, HDS, HP, BlueArc, and Isilon, as well as cloud storage provider Nirvanix. This, of course, is good news for customers and the industry in general.

Our success will depend on how well we seamlessly integrate and support their storage hardware. So far, this is going well, and in fact some of the partnerships we’ve already announced are looking as if they may escalate to deeper integrations and levels of partnership.

If we execute well on those, then the data reduction for file storage category may eventually become “NetApp dedupe” versus “everyone-else-with-Ocarina.” That is our goal.

Data reduction for online storage is a hot topic for a couple of reasons. It’s been validated in other parts of the data center. We all know dedupe has become the norm for backups, with disk-based backup targets with dedupe built in rapidly replacing tape. Compression and simple dedupe (called dictionary compression) have also been widely adopted (as WAFS solutions) in the network. The next natural frontier is online file data. Just like Data Domain focused their technology on backups and Riverbed did the same for WAFS, we have optimized our data reduction technology for online data. Each of these use cases has a different design optimum, and if you started out building a solution for online file data, you’d make different design decisions than you would for backups and WAN optimization. In our case, that meant building a whole new kind of dedupe to be able to get the best results on the kinds of files that are driving storage growth.

In my view, we’re really just in the early stages of seeing data reduction make its way to online storage.  Over the course of the next year or two, we expect to see dedupe for online data become as widely understood and deployed as dedupe for backups. During that time, we’ll see lots of debate over different approaches, lots of education about how things work, and a market that will gravitate towards the solutions that both deliver the best results, and which deliver the best performance for end-users and applications.

An interesting day today in that there isn’t too much in the way of hard news, but plenty o’ commentary floating around the old blog-o-tweet-o-news-o-sphere regarding storage.

First, lots of buzz about Ocarina this past week:

We earned a nice mention in the keynote address at BIO IT World, where Ocarina is in attendance and was a “best in show” finalist. Things are going extremely well there, so stay tuned!

Dedupe Team Up: A post by George Crump of Storage Switzerland on InformationWeek that shows how Ocarina is working with major storage vendors to help them compete with NetApp on dedupe.

NetApp’s Dr. Dedupe questions Ocarina about whether it is doing dedupe in this post “When is Dedupe Not Dedupe?” Kind of an odd one–as W. Curtis Preston points out in the comments field, the “Technology” pages on the Ocarina web site–particularly this one which breaks down the Ocarina ECOsystem process–answer his question. I’d also suggest he read this recent post which was actually in response to another NetApp blogger, Alex McDonald.

Another mention of Ocarina and its recent BlueArc partnership announcement in Wikibon’s Bill Mottram’s summary of the scene at NAB last week. He writes:

“BlueArc: The news on the BlueArc booth was the announcement (April 20th) of their partnership with Ocarina. Availability of the Ocarina Optimizer for BlueArc is scheduled for mid-May.”

And in other news…

InfoStor’s Dave Simpson comments on the Oracle-Sun acqui with some thoughtful insights on his newly revved up blog.

Devang Panchigar has a nice round-up and some commentary on EMC’s recent V-Max announcement on Gestalt IT/Storagenerve.

And, as widely reported, EMC has asked its employees to take an across the board pay cut. And, with the kind of speed we’ve come to associate with blogging, EMCer Storagezilla posted on the 5% cut on his blog right after the announcement was made public.

Happy Tuesday everyone!

This week, many of our crew are attending NAB Show, the annual mega-convention of the National Association of Broadcasters in Las Vegas. Why do we need to be there? Well, more and more, the business of making shows and movies has become inextricably linked to the need to store immense files.

To add to the excitement, we are announcing partnerships with two major storage vendors, HDS and BlueArc today. Both are making their storage Ocarina Optimized, thus adding our groundbreaking compression and dedupe to their offerings.

Updates: Beth Pariseau has a very well written and researched SearchStorage article up that explains the partnerships in depth. Definitely worth a read!

On Wednesday, Chris Mellor at the UK Register wrote about the BlueArc-Ocarina partnership as well, with this piece.

With BlueArc, we’re announcing the Ocarina Optimizer for BlueArc. This integrates Ocarina’s data reduction technology seamlessly with the BlueArc Titan storage solutions. CGI animation leader Rainmaker Entertainment–makers of kids animation movies, most recently an animated sequel to the original, Jerry Lewis “The Nutty Professor”–finds this combined solution an ideal method of keeping files accessible during the making of an entire CGI film, which can easily run into petabytes of data.

With HDS, we’re joining the Hitachi Data Systems Technical Alliance Partner (TAP) ISV Program. In doing so, Ocarina helps HDS customers “stretch their IT storage dollars to support the phenomenal growth of unstructured data,” as Asim Zaheer, vice president of product and competitive marketing, Hitachi Data Systems put it.

Exciting times for this industry, as well as our company. We hope to see you at NAB if you’re there.