Online Storage Optimization

The acquisition of Ocarina by Dell has now closed, and we can now say some things about where we are headed.  Mergers and acquisitions can often be a time of confusion, as the new combined company figures out what the new plan will be.

I’d like to use this first blog after the closing of the deal to clear up some of that confusion.    Some of our competitors have tried to take advantage of the period between the announcement of the deal and the closing (a time during which neither company can make forward-looking statements about joint operations) to sow uncertainty and doubt amongst some of Ocarina’s most-valued customers:  our OEM and resale partners.

Now that the deal has closed, I can say that Dell is actually very interested and committed to pursuing OEM partnerships for the Ocarina technology with other storage vendors.

One competitor in particular tried to imply that our OEM’s would have to drop Ocarina and go looking for a new dedupe technology.

This is simply not true.  While each partnership is a unique case, in general, the OEMs that want to continue doing business with us will be able to. What’s more, this is now a stronger solution, both technically and on the business front.  Dell plans to invest in growing the Ocarina engineering team – and provide a richer roadmap and have a faster cadence.    Clearly, onsite support will now be available worldwide for those partnerships that involve reselling an Ocarina-based dedupe appliance. Software support will eventually be available in many more languages, also around the clock. In addition to those obvious technical benefits, some of the risk of doing business with a startup is now also resolved.

Finally, I think that Dell sees value in an ecosystem of compatible and complimentary dedupe products. It’s great, for example, that Ocarina dedupe can work together with Commvault and Symantec dedupe. We think it is good for customers to see that compatible ecosystem grow. What is good for customers is also good for our OEM’s – by participating in an open ecosystem that gives customers not only the best-of-breed technology, but also a rich set of choices for deployment, they are strengthening the solution they offer and the story they have to tell.

As @storagebod (Martin Glassborow) noted today in a blog post, the issue of protecting deduplicated data is an important one for any business.

Deduplication and other data reduction technologies offer the opportunity for increased data protection at a reduced cost.

When you hit a threshold of 50% or greater overall data reduction, it lowers the cost of performing a a full mirror of data. For example, if you have 100 terabytes of data, mirroring today would require 200 terabytes of disk space. But if the data were reduced by 75%, you could store the same data in 25 terabytes, and full mirroring would require on 50 terabytes of disk - a significant savings. In other words, it is possible to fully protect your storage, including full mirroring, with less disk than it takes to store the unprotected data today.

While this is true with any amount of space savings, when that level of savings goes beyond 50% (or a dedupe ratio of 2:1 or better), the storage cost of full mirroring is zero.

There are a number of options that your deduplication provider should be pursuing to compliment your data protection strategy. For example, a vendor should be able to allow you do two key things with your dedupe configuration:

1. Allow a minimum level of duplicate blocks to accumulate prior to starting deduplication. For example, you could allow data to be written twice, and then deduplicate all subsequent occurances. So long as the dedupe solution is aware of both original occurances, you have a form of mirroring without needing to do full mirroring at the disk level. Call this duplicate mirroring.
2. Set a threshold for a maximum number of duplicates. To set a maximum level of exposure on the loss of physical disk, you should be able to ask that once you have found ‘n’ instances of the block, to start over. You could set that number at 8, 32, 128 or whatever frequency makes business sense to you, and therefore, the potential loss of the sector on which the duplicate is stored would only affect a certain number of files.

These two examples are not necessarily a complete answer in and of themselves, but they do provide guidance for tools you can work with as part of an overall data protection strategy. As deduplication becomes a storage fundamental, and is in place across multiple tiers of storage and on multiple products in your data center, understanding the impact of dedupe on your data protection strategy will be key and your dedupe vendor should be providing you the right tools to manage that.

Today is a bit of a sad day for me, as this will be my last post on Online Storage Optimization. For those who are just joining us, I’ve been the regular “newsy” blogger on this site. It’s never been a traditional setup. I’m not an employee of this blog’s parent, Ocarina Networks. Rather, I’m an independent social media consultant who started out as a PR rep for the company. I moved into this role when we relaunched the blog as more of a publication in early 2009. It’s been a wonderfully fruitful arrangement that allowed me all the freedom and breadth to think, talk and learn to my heart’s content.

As much as I’ve loved working on this project, the reality of my life as a consultant has meant that I am being pulled in many directions. I’m now in the process of launching a new business, currently in stealth but soon to be revealed. (And I hope I can count on your support when it does.) This new set of responsibilities makes it impossible for me to continue to follow the daily ups and downs, trials and tribulations and fascinating personalities of the storage industry as an active blogger. I will of course be watching from afar. More than that, I’ll never forget the warm welcome I received in the storage blog-o-tweet-osphere.

With that said, I feel compelled to thank certain specific people who have made the experience of being part of the storage industry particularly enjoyable and enlightening. First, Carter George, VP Products at Ocarina and the lead blogger on this site. Carter took me under his wing from the get-go, sharing his vast wealth of knowledge as a leader in the industry. He took every single one of my questions seriously, no matter how stupid, and answered them in ways that expanded my understanding of this complex and technical subject area. For those who don’t know him personally, Carter is also one of the nicest and most approachable people you could meet. He encouraged me to stretch myself, and the result is that, lo and behold, I became a reasonably well-known and recognized “storage blogger.” Not something I would’ve dreamed of in million years.

Second, Stephen Foskett, publisher of Gestalt IT. A couple months after I started working on this blog, I got a DM on Twitter from him that read, “I hope Ocarina appreciates what you’re doing for them.” It couldn’t have been better timed. I was buried in working on a white paper about Ocarina’s newest release that I honestly didn’t believe I would ever have the technical know how to finish. At the same time, I was struggling to come up with topics for the blog. I also worried, continually, that I had pissed someone or other off by what I said on this blog or on Twitter. To get this message from someone as well-respected as Stephen gave me a much needed sense that somehow or other, I was doing okay.

Third, Murli Thirumale, CEO of Ocarina. Murli doesn’t have the personality you associate with your typical Silicon Valley CEO–he’s about as far from the image we all know of the crazed egomaniac as you can get. He is a thoughtful, respectful, and yet endlessly upbeat person who has built a successful company based on a real need. We had a relaxed working relationship, and I always appreciated his occasional contributions to this blog, which offered a “big picture” understanding of what he intended when he started the company. It was an honor to work for someone like him.

Fourth Marc Farley, storage rapper and 3Par’s social media whiz. Marc was one of the first people to respond to a Twitter tweet of mine and chat with me. We talked about whether it’s possible to remove the light bulb in your fridge, I seem to recall. Marc and I ended up creating a video together that became something of a viral hit within storage circles. He is one funny, cool guy and a true storage industry veteran who nevertheless has stayed ahead of the curve.

Fifth, Storagebod, also known as Martin Glassborow. Aside from being a great source of interesting blog posts that always kept me on my toes and wanting more, Martin is a fantastic Twitter conversationalist. He seems to have read every book on the planet. He also knows a great deal about a whole lot of other subjects, from music to health to wine. A true renaissance man and therefore someone I could always count on for a laugh or a chat–often when I most needed it.

Sixth, Greg Knieriemen. Greg has been a great guy to know, and has given me lots to think about through the exciting and active storage community he created, StorageMonkeys. Last fall, he had me on as a guest on his podcast, Infosmack, where we talked booth babes and other hot topics. He’s also a very funny guy–and a sense of humor is everything in this intense business.

Seventh, George Crump. George, of Storage Switzerland, was kind enough to give me all kinds of advice about how to run a successful blog when we first relaunched. I give myself credit for listening to him, and the results were notable. We have had a great run here, and one that I am sure will continue as I pass the baton to Mike Davis and any others who jump on the bandwagon known as Online Storage Optimization. I hope you’ll keep reading. I know I will.

Data is growing at a dizzying rate. We need only look at our home computers to get a sense of how easy it is to fill our hard drives to overflowing with all manner of flotsam and jetsam. From family photos to LOLcats to videos of our kids, we’re finding it difficult if not impossible to keep down the rising tide of files.

There is a cost to this, as many if not most enterprises are now recognizing. Recently, InfoWorld launched a special section, Data Explosion that guides companies through the myriad problems that arise from having too much data to handle. With headlines like: “The big data addiction,” the new section promises to address the issue with step-by-step guides, white papers, and other instructional pieces.

Infoworld blogger Matt Prigge delves into the topic in a post today, “The high cost of lazy storage.” He says that users need to take responsibility for keeping their data under control. Despite this admonishment, he admits that he himself is an “excellent example of the problem.” He saves all of his email, because he never knows what he might need later. Sound familiar? If someone whose blog is called “Information Overload” can’t get control of his personal data, it’s hard to imagine how anyone else can.

Prigge writes, “The bigger that data gets, the more effort required to put the genie back in the bottle.” He pushes the metaphor even further (and more gruesomely) by suggesting that at some point it’s easier to kill the genie and throw away the bottle. Now, that does strike us here at Online Sto Op as rather extreme. Why not simply put the genie back into that nice, compact bottle where she was living perfectly happily for so many years?

As we all know from 70s TV, those bottles were well-upholstered and downright comfortable living spaces for many a genie. And while it’s true that some genies (or Jeannies) would get so angry they’d stomp their feet when they were magically sent back there, they eventually settled back onto the purple pillows, kicked off their metallic platform heels, dug their toes into the shag carpeting and relaxed. Same goes for data reduction. A combination of approaches seems the most sensible answer. Data needs to be managed. There is something that is known as 100% compression–it’s called “deletion.” But short of that, there are ways to reduce data by as much as 90%. There are solutions for reducing the types of files that are driving the fastest storage growth, such as JPEGs, documents, videos, graphics, and other large files. An intelligent, content aware approaches that includes both deduplication and compression is what this blog’s parent Ocarina provides.

Four out of five college students agree, this is not the way to deal with data growth. How about this instead?

I’ve noticed a few blog posts recently about speed of deduplication in the modern data center. I agree that speed is an important factor, but keep in mind that not all dedupe is created equal. That is to say, fast is good, but only if you are also effective. One of the tricky things has been that the easiest data to compress is also usually the most carefully performance tuned. A great example of this is a database. This is because databases are comprised of simple alphanumeric fields and sparse tables. All of that is easy to reduce in size.

However, a company’s core transactional database is the most conservative asset in the data center. Introducing compression would save space, for sure, but you could only use very fast, simple compressors there. At the same time, customers will be hesitant to deploy a new layer of processing in their most sensitive application.

So, where is most data growth? In fact, it’s being driven by unstructured data – Office documents, rich media, email with attachments, PDFs, Flash videos, and so forth. This complex data does not lend itself to fast simple compressors. But perhaps we should back up for a moment and think about how customers have been behaving all along.

Throughout the history of storage, there have always been tradeoffs available between fast expensive storage, and slower but cheaper alternatives. This is not a bad thing. It gives users alternatives based on their priorities and budgets. Back in the old mainframe days, these choices were between very expensive mainframe memory and “offline” storage like drums, cards, and tapes. Today the technology is all much bigger, faster, cheaper and sexier. But really, the tradeoffs are the same.

Data reduction technology adds another layer of choice above and beyond the traditional hardware choices. Now in addition to choosing whether you want fast, expensive solid state disk (SSD) or slower but very cost-effective SATA, you can also choose whether you want to compress and/or deduplicate the data that is stored on those disks.

Just like physical disks, compression and dedupe come in a range of speeds and capabilities. There are simple and very fast compressors that are essentially invisible in terms of their impact on storage performance. There are more complex compressors that get better results, but which may take longer, either to compress or to decompress the data. Deduplication, done well, should always be pretty fast, and streaming dedupe rates of well of 300MB/sec are now available from many vendors (including Data Domain and Ocarina).

The emergence of tools to automatically tier data to its appropriate place help make the use of all of these technologies more feasible. That applies as much to solid state disks as it does to dedupe and compression. When data tiering can be made invisible to end-users and applications, then implementing multiple physical and logical tiers of storage becomes practical.  Good examples would include EMC’s new FAST tools, Compellent’s “Fluid Data Storage”, and HDS’s Data Migrator. When users or administrators have to move data by hand to get it to a compressed tier or a solid state disk, then the operational costs offset the capital savings.

You might want to be wary when someone’s biggest claim to fame is fast dedupe. Just as the old mainframe admin had to decide whether something was important enough to live in RAM, or could be stored on cheaper tapes instead, today’s IT shops have to decide where it is most important to try to get data reduction, and what tool will get the most bang for the buck for that kind of data. You need the whole story, and then you can decide based on your own priorities.

With all the talk about the data inconsistencies around climate change theory, one issue that I’d hate to see lost in the shuffle is the actual environment. That is, while I personally have been skeptical for some time about the alarmist tone many scientists took regarding global warming, it would be a shame if there was such a backlash that people forget about the much more crucial, larger issue at stake. That is, we need to look at all the ways –on macro- and micro-scales–that we can reduce the overall pollution we generate through our daily habits.

One of the persistent myths about the Internet is that it is clean and green. We overestimate the value of going “paperless” while lowballing the effect on the environment of data centers. One need only look at an online pub like Data Center Knowledge to see that one of the most talked about issues in data centers today is how to reduce rack space, cooling and other energy costs associated with storage. (Another great resource is Greg Schulz’s StorageI/O blog.) This is particularly true of the data being generated through our new Web 2.0 sharing habits. Jon Toigo can laugh about the exploding digital universe all he likes, but it’s still the case that data growth is going like gangbusters in this socially networked era. Recession or no recession, there is a growing demand for ways to make storage more efficient.

Large players in this space are all too aware of the environmental and financial costs of such rapid data growth. Every time you share a photo or video, you’re contributing to it. And who among us doesn’t do this nowadays? In response. companies are experimenting with all kinds of techniques, including new building designs making use of outside air, reducing overall rack space usage with data reduction such as is offered by this blog’s parent Ocarina, cloud adoption, and so on and so forth. Companies like Google, Yahoo and Facebook are also creating next generation storage architectures that are more efficient for handling the realities of today’s internet. In short, let’s be sure, as we discuss the fallout from the latest global warming debate that we don’t start acting too lax about the effect of our actions on the planet.

For IT decision makers it’s imperative that you keep up with the latest news and information. Yet, the overall shakeup of the media has left many confused about where to turn. Industry pubs are getting slimmer and slimmer. Some are cutting back, others are consolidating, and a few have disappeared entirely. At the same time, the blogosphere is exploding with content. How do you sort it all out?

Here are some of the stops we at Online Storage Op make on a regular basis in order to stay up-date on IT infrastructure news without driving ourselves nuts in the process. We’d love to hear your suggestions–how do you find out what you need to know? What used to work and isn’t so much anymore? What do you wish were out there that isn’t? For now, here’s our list:

TechTarget - Still a prime source of storage news and views, particularly SearchStorage. Reporters to watch: Beth Pariseau and storage gossip watcher Simon Sharwood.

The UK Register - Chris Mellor, Timothy P. Morgan and others continue to churn out solid daily coverage of industry trends, with headlines that might make you laugh out loud.

Gestalt IT - I admit it, there are days when I don’t bother reading anything else except Gestalt to get my daily dose of storage news and views. With a solid lineup of independent writers and objective analysis on industry trends–not to mention the new addition of a humor column–it’s a one stop shop.

Network Computing - For those who used to read Byte & Switch, this is the new site that integrates it with other networking news. A necessary update in these lean times one supposes. Solid regular contributions from such writers as Howard Marks and George Crump offer simple, straightforward information and advice about products and platforms.

Wikibon -Dave Vellante and others contribute to this blog, which picks out some of the hottest trends in storage. A good way to get a quick hit on what the Wikibon analyst community is talking about.

Emulex’s Shared Items - An easy cheat sheet on what the latest industry observers and vendor bloggers are talking about. Easy to track on Twitter or through Google Reader.

Storage Monkeys - This community site has a lot going on, so I tend to just quickly check the blogs and then take a listen to the latest episode of the Infosmack podcast, which is posted each Monday morning. The blogs tend to be a little on the insiderey side, so if you’re not actively working in the data center you might find them too granular. On the other hand, the podcast is very much the 30,000-foot view of overall storage, networking and virtualization trends, served up in a highly entertaining radio format with two great hosts, Greg Knieriemen and Marc Farley.

Twitter - It sometimes seems like more trouble than it’s worth, but truth be told, the best way to find out what people in storage are talking about, worried about, and trying to fix is to sit around and listen to what they’re saying on a daily basis. Go ahead and follow a couple of Storage lists and you’re pretty much all set–here are a few I’d recommend:

We Follow Top Storage Twitter

Bas Raayman’s Storage List

So, what did I miss? Inquiring storage minds want to know.

This blog’s parent Ocarina and I have something in common. Can you guess? That’s right. We both have names that are cause for frequent comment. The first question anyone ever asks me when they meet me is, “is that your real name?” And the first question that anyone asks a person from Ocarina is “where did you get the name?” In essence, this is the same question in two different forms.

I almost always answer in the same way when asked about my name. I confirm the fact that my parents were indeed hippies (as the questioner already suspected) and that yes, I do live in California and always felt I belonged here. My full name actually means “Sunshine from the West,” something I have noted in one of my many blog entries on the concept known as “aptonyms.”

For the folks at Ocarina, the response is to explain that the name came from the founders, who decided that it sounded good. They also liked that it is a real word, rather than some kind of computer-generated mashup of random “Xs” and “Zs.” Most people are satisfied with this answer, though it does tend to be a bit of a conversation stopper. You want them to draw some connection between the clay flute that is the actual ocarina and the company mission, which is to reduce data through content aware compression and deduplication. But truth be told, there isn’t one.

As someone who spends a good deal of time tracking what’s being said on blogs, Twitter and other social media platforms, the name Ocarina poses a bit of a problem. The Ocarina of Time is a game within the extremely popular Nintendo series known as the “Legend of Zelda.” Hundreds of tweets are posted almost every day about this game, and numerous blog posts–all of which clog up my RSS feeds and Twitter search data. There’s even a video celebrating the music of the game, which has had over 65,000 views on YouTube for some unfathomable reason.

Here it is for all you Zelda freaks:

There is also an iPhone app called the “Ocarina” that allows you to play your own iPhone and share your music with others. Again, very popular–which leads me to wonder if there’s something about the name Ocarina that naturally resonates with people. What do you think? Are there any other associations you have with the name you’d like to share?

Last week’s BD Event was more than just a deal making event. It was a chance to learn about new product releases and trend in the storage industry. The big picture: gone are the days when end users had to accept whatever the storage industry handed down to them. Today’s small-to-medium-sized storage operations are all about designing systems in response to customer needs. Whether that’s developing end-to-end dedupe, refining and improving processes for data recovery management, delivering automated marketing tools, improving data migration, or creating storage that is more energy efficient, the push is towards designing systems with real world customer needs in mind.

The BD Event organizers’ deep connections within the storage arena meant that the two-day conference in Palo Alto drew a who’s who of industry folks. I was particularly pleased to see the number of analysts and consultants on site, including Jerome Wendt, George Crump, Deni Connor, Dave Vellante, Stephen Foskett and Tony Asaro (who unveiled his new project, Voices of IT). I also spoke at length with storage writer Howard Marks, who has a new project called DeepStorage.net that looks very promising for companies seeking solid research that they can use as outbound marketing.

Pleasingly enough, this blog’s parent Ocarina was very much the talk of the conference after kicking off the first day’s emerging vendor showcase. Carter George, VP Products gave away the fact that end-to-end dedupe is becoming a part of the overall strategy for the company. This information set tongues wagging. As DCIG’s Jerome Wendt later blogged: “Ocarina Networks is another company that is adapting to new demands from its customers. Originally it started out doing post-process deduplication of large image files (JPGs, MPEGs, etc.) that had been dormant for 30 days or more - great stuff! But now its customers and even OEMs (Ocarina did not say who) are coming to it and asking for it to do end-to-end data deduplication from primary disk to backup disk without ever reconstructing it. After all, once the data it deduplicated on primary storage, why reconstruct it to then deduplicate it again when it is backed up?”

A good question, and one that was hotly debated and discussed among those in attendance. As Jerome notes, this is a perfect example of the customer responsiveness trend. It’s also an acknowledgment of something that’s been obvious to end users for some time–data reduction shouldn’t have to be isolated within each storage sector. In this day and age you really shouldn’t have to buy separate products to dedupe within primary, nearline, and backup. It’s like having to buy a separate dishwasher for your pre-rinse, wash, and dry cycles.

Other standouts at the event included Bocada, which has updated its DR management software by introducing a new product, Prism. I plan to have the CEO Nancy Hurley on my podcast, and so will learn more about how this update is serving its existing and new customers. I confess that I went to her presentation mainly because I wanted her on my show, but I quickly realized that there was something here of note. That is, the company is addressing a real gap in how well these processes are managed and improved, a key consideration with a crucial component like data protection. She gave a brief overview of the user interface, and on its face it seemed intuitive and flexible.

TechValidate also served as a great example of a company that has evolved based on customer needs. As CEO and founder Brad O’Neill explained during his emerging vendor presentation, originally the company was formed to serve companies that were having trouble getting customer references. These all-important testimonials are sometimes difficult to get–as many industries are gun shy about trumpeting their connections with too many IT and storage vendors. However, O’Neill soon recognized a larger need among its customers for usable marketing materials that could be generated from the information they were gathering. Now, the company has a wide range of customers across numerous industries that are using it as a way to serve up marketing publications.

One final highlight of the event–I got to speak with the NetApp blogger known as “Dr. Dedupe,” Larry Freeman. Larry is best known for running around in a lab coat and stethoscope asking people if they know anything about dedupe. The videos of these shenanigans are posted on his blog and on NetApp TV on YouTube. I suppose in a sense he and I are competitors. Turns out, he’s been writing a book, “Evolution of the Storage Brain” and posting it as he writes it, chapter by chapter, on his blog. This means that readers have a chance to comment on it and shape it as it goes along. Check it out!