Online Storage Optimization

Data deduplication and compression are quickly becoming standards, with customers in a wide variety of markets and business sizes recognizing the value of saving money and reducing management through storage optimization. Earlier this week, InformationWeek released a research report highlighting how IT professionals were working to meet today’s exploding storage capacity demands. The report, unsurprisingly to us, showed nearly 80 percent of survey respondents said they were using compression technologies or had them under evaluation – while more than half reported similar implementations and plans for dedupe.

What was even more impressive about this survey was the respondents themselves. While it’s easy to understand for the largest companies with their heavy data sets that scalability is a major strain, two thirds of these respondents reported having less than 50 terabytes of storage, meaning these challenges have now hit mid-size businesses as well. The <50TB crowd is going to be more risk averse, and reducing their storage from 3 arrays to 1 array isn’t necessarily the “killer” value proposition. The value proposition of compression and dedupe technologies includes things like reduced backup windows, which can only be obtained by a solution that supports end-to-end benefits, proves to customers that it’s trustworthy and doesn’t carry additional administrative burden.

Also intriguing was the comment by many respondents that the growth of storage was in transactional applications (database and e-mail) rather than unstructured data. Many of Ocarina’s clients in multiple market segments are seeing dramatic growth in unstructured data, but in this business tier, more traditional applications are adding to the data glut. Also, there is little doubt that in a short amount of time VM applications will appear as part of that category. This is also in contrast to the “Petabyte vertical markets” which are chock full of unstructured file data, and a good lesson for OEMs to take away; any embedded dedupe implementation needs to deliver results in structured-data applications, and that’s a non-trivial requirement. Common dedupe solutions in the market today would create noticable performance impact on these apps, which is why Microsoft removed single-instancing as a native feature from Exchange. A dedupe solution in these applications needs to include workflow and application awareness, for example compressing and deduping only those portions of the database (or VMDK) that are inactive. Ocarina calls this Heat Index Management, and it’s a feature included in our ECOsystem for OEMs.

As industry observers have seen in the last several years, backup has been the killer driver for deduplication – the first wave, if you will, because the ROI of D2D backup is tremendous with this feature in place. But in the survey there is strong adoption (>50% of deployments) of dedupe for archival and other non-backup apps. Of course this isn’t unexpected to Ocarina, because we’ve helped drive this shift towards compression on primary storage and the entire data lifecycle.

As data growth continues in all markets, we believe strongly in the need to compress and optimize across the datacenter, from primary storage to backup and archival. InformationWeek has done a fantastic job demonstrating real customer demand for these technologies, which we expect will increase. If you haven’t walked through the presentation, make sure you do at http://www.informationweek.com.

Today is a bit of a sad day for me, as this will be my last post on Online Storage Optimization. For those who are just joining us, I’ve been the regular “newsy” blogger on this site. It’s never been a traditional setup. I’m not an employee of this blog’s parent, Ocarina Networks. Rather, I’m an independent social media consultant who started out as a PR rep for the company. I moved into this role when we relaunched the blog as more of a publication in early 2009. It’s been a wonderfully fruitful arrangement that allowed me all the freedom and breadth to think, talk and learn to my heart’s content.

As much as I’ve loved working on this project, the reality of my life as a consultant has meant that I am being pulled in many directions. I’m now in the process of launching a new business, currently in stealth but soon to be revealed. (And I hope I can count on your support when it does.) This new set of responsibilities makes it impossible for me to continue to follow the daily ups and downs, trials and tribulations and fascinating personalities of the storage industry as an active blogger. I will of course be watching from afar. More than that, I’ll never forget the warm welcome I received in the storage blog-o-tweet-osphere.

With that said, I feel compelled to thank certain specific people who have made the experience of being part of the storage industry particularly enjoyable and enlightening. First, Carter George, VP Products at Ocarina and the lead blogger on this site. Carter took me under his wing from the get-go, sharing his vast wealth of knowledge as a leader in the industry. He took every single one of my questions seriously, no matter how stupid, and answered them in ways that expanded my understanding of this complex and technical subject area. For those who don’t know him personally, Carter is also one of the nicest and most approachable people you could meet. He encouraged me to stretch myself, and the result is that, lo and behold, I became a reasonably well-known and recognized “storage blogger.” Not something I would’ve dreamed of in million years.

Second, Stephen Foskett, publisher of Gestalt IT. A couple months after I started working on this blog, I got a DM on Twitter from him that read, “I hope Ocarina appreciates what you’re doing for them.” It couldn’t have been better timed. I was buried in working on a white paper about Ocarina’s newest release that I honestly didn’t believe I would ever have the technical know how to finish. At the same time, I was struggling to come up with topics for the blog. I also worried, continually, that I had pissed someone or other off by what I said on this blog or on Twitter. To get this message from someone as well-respected as Stephen gave me a much needed sense that somehow or other, I was doing okay.

Third, Murli Thirumale, CEO of Ocarina. Murli doesn’t have the personality you associate with your typical Silicon Valley CEO–he’s about as far from the image we all know of the crazed egomaniac as you can get. He is a thoughtful, respectful, and yet endlessly upbeat person who has built a successful company based on a real need. We had a relaxed working relationship, and I always appreciated his occasional contributions to this blog, which offered a “big picture” understanding of what he intended when he started the company. It was an honor to work for someone like him.

Fourth Marc Farley, storage rapper and 3Par’s social media whiz. Marc was one of the first people to respond to a Twitter tweet of mine and chat with me. We talked about whether it’s possible to remove the light bulb in your fridge, I seem to recall. Marc and I ended up creating a video together that became something of a viral hit within storage circles. He is one funny, cool guy and a true storage industry veteran who nevertheless has stayed ahead of the curve.

Fifth, Storagebod, also known as Martin Glassborow. Aside from being a great source of interesting blog posts that always kept me on my toes and wanting more, Martin is a fantastic Twitter conversationalist. He seems to have read every book on the planet. He also knows a great deal about a whole lot of other subjects, from music to health to wine. A true renaissance man and therefore someone I could always count on for a laugh or a chat–often when I most needed it.

Sixth, Greg Knieriemen. Greg has been a great guy to know, and has given me lots to think about through the exciting and active storage community he created, StorageMonkeys. Last fall, he had me on as a guest on his podcast, Infosmack, where we talked booth babes and other hot topics. He’s also a very funny guy–and a sense of humor is everything in this intense business.

Seventh, George Crump. George, of Storage Switzerland, was kind enough to give me all kinds of advice about how to run a successful blog when we first relaunched. I give myself credit for listening to him, and the results were notable. We have had a great run here, and one that I am sure will continue as I pass the baton to Mike Davis and any others who jump on the bandwagon known as Online Storage Optimization. I hope you’ll keep reading. I know I will.

Data is growing at a dizzying rate. We need only look at our home computers to get a sense of how easy it is to fill our hard drives to overflowing with all manner of flotsam and jetsam. From family photos to LOLcats to videos of our kids, we’re finding it difficult if not impossible to keep down the rising tide of files.

There is a cost to this, as many if not most enterprises are now recognizing. Recently, InfoWorld launched a special section, Data Explosion that guides companies through the myriad problems that arise from having too much data to handle. With headlines like: “The big data addiction,” the new section promises to address the issue with step-by-step guides, white papers, and other instructional pieces.

Infoworld blogger Matt Prigge delves into the topic in a post today, “The high cost of lazy storage.” He says that users need to take responsibility for keeping their data under control. Despite this admonishment, he admits that he himself is an “excellent example of the problem.” He saves all of his email, because he never knows what he might need later. Sound familiar? If someone whose blog is called “Information Overload” can’t get control of his personal data, it’s hard to imagine how anyone else can.

Prigge writes, “The bigger that data gets, the more effort required to put the genie back in the bottle.” He pushes the metaphor even further (and more gruesomely) by suggesting that at some point it’s easier to kill the genie and throw away the bottle. Now, that does strike us here at Online Sto Op as rather extreme. Why not simply put the genie back into that nice, compact bottle where she was living perfectly happily for so many years?

As we all know from 70s TV, those bottles were well-upholstered and downright comfortable living spaces for many a genie. And while it’s true that some genies (or Jeannies) would get so angry they’d stomp their feet when they were magically sent back there, they eventually settled back onto the purple pillows, kicked off their metallic platform heels, dug their toes into the shag carpeting and relaxed. Same goes for data reduction. A combination of approaches seems the most sensible answer. Data needs to be managed. There is something that is known as 100% compression–it’s called “deletion.” But short of that, there are ways to reduce data by as much as 90%. There are solutions for reducing the types of files that are driving the fastest storage growth, such as JPEGs, documents, videos, graphics, and other large files. An intelligent, content aware approaches that includes both deduplication and compression is what this blog’s parent Ocarina provides.

This Monday morning we all awoke to the news that U.S. House of Representatives passed H.R. 3590, a major health care reform package. Known as the “Patient Protection and Affordable Care Act,” it aims to ensure that more Americans will be covered by health insurance. It also makes it more difficult for insurance companies to deny coverage. Whatever your views on the debate around this legislation, one thing is clear: this reform shines a light on the many IT and storage challenges associated with the health care industries. On the most basic level, making changes and adding millions to the insured will mean an influx of new paperwork, and that translates to files that must be stored and managed.

As ZDNet’s Dana Blankenhorn reports, at least one investor, Bill Miller of Legg Mason, is bullish on healthcare IT as a result of this bill. As he sees it, this reform actually gives insurers a boost because it requires a raft of new signups. This will benefit IBM and GE, among many others. It’s easy to forget that a year ago, our industry was in a lather over the fact that President Obama was committing stimulus money to the health care industry for a push to digitize medical records. We are still sorting out the myriad implications of this new mandate, which sets aside nearly $20 billion for the effort. We do know, for example that tech giant Microsoft jumped all over it by introducing something called HealthVault Community Connect–specialized software to manage these records.

Taking a step backwards, the new landscape of medical research has put a strong demand on storage resources. As our blogger Mike Davis reported from a recent “next generation sequencing” conference, the pace of cutting edge genomic research, and the types of files it produces has led to an upsurge in the need for storage capacity. As it happens, Ocarina just put out a case study about the work it’s doing with Cornell University Center for Advanced Computing, which handles a massive influx of genomics files on a daily basis. For the full case study, go to the company Resources page and click on Cornell Case Study (may require signup).

What do you think? Are there storage challenges you or your organization predict as a result of this new legislation? We’d like to hear. The forum (below, in comments) is open!

It’s back… That’s right, on Gestalt IT you can now find details on the upcoming Tech Field Day, to be held in Boston April 8 and 9. This event brings together bloggers from around the world for two days of deep dives at tech companies. The result is expected to be a multitude of tweets, blog posts, videos and photos. The concept is a clever one, and as participants in the first Tech Field Day, we’re thrilled to see it continue. The upcoming Boston event has an impressive list of presenting sponsors: EMC, Cisco, Data Robotics, HP, and something called VKernel, a VM optimization company.

Today I spoke to organizer and Gestalt IT Publisher Stephen Foskett about his plans. I couldn’t help but wonder, when looking at the list of companies that will be presenting, whether the profusion of big names represents a new chapter for Tech Field Day.

“I was trying to find interesting companies in Boston and they just happened to be more of the big IT companies,” said Stephen. “As always, I’m inviting great companies. That’s the real story.”

There is plenty of new blood among the invited bloggers. About half of the delegates weren’t at the last Tech Field Day. The list for Boston includes some well-known virtualization bloggers: Jason Boche, David Davis, Edward Haletky, and Gabrie Van Zanten. Another new face is Matt Simmons, a system administrator with a popular blog called Standalone SysAdmin. Meanwhile, the previous delegates developed some documents designed to help both sponsors and bloggers. These are now posted on Gestalt IT as well, at this link.

I also got the scoop on the upcoming Seattle Tech Field Day, slated for mid-July. While he declined to name the companies, Stephen did say it’s already half booked with presenting sponsors.

“Redmond is home to a huge number of tech companies. I underestimated the number,” he said. “Some of the companies have lots of different product lines and want to do repeats.”

For the many of us who aren’t going to be able to attend, the Twitter hash tag #techfieldday is the way to get a sense of the proceedings in real time. There’s also a handy Twitter list of the Boston delegates. We’ll be watching!

Across the storage blog-o-tweet-osphere today folks are donning green scarves, putting four leaf clovers in their lapels, and generally proclaiming the luck of the Irish. Yes, it’s a good life in storageland. And there’s plenty of news to amuse and bemuse.

EMC made a big splash this week with a presentation to analysts by President and COO Pat Gelsinger that outlined a new vision for virtual storage. You can listen to the whole thing here. Chris Mellor at the Register called the plan a sign that EMC has “lost their marbles,” but others think it represents the future of storage.

Here is some of the commentary from both within and outside EMC:

EMC:

Chuck’s Blog - This changes everything

Blog Stu - Virtual Storage, not just another V-word

Commentary:

**New addition thanks to @sfoskett** Burton Group: EMC’s Global Storage Vision

Gregs’ StorageIO blog - Virtual Storage and Social Media: What did EMC not Announce?

Chris Mellor, The Register - Gelsinger stuns analysts and colleagues with storage pool plan

Stacey Higginbotham , GigaOM (yes, GigaOm! Welcome to sto-land, Stacey) - EMC’s Crazy Plan to Create a Worldwide Data Cloud

In other news… there’s a really sweet video on the Hitachi Data Systems site that talks about its partnership with this blog’s parent Ocarina Networks, and how this will benefit customers, reducing their data at rest by 10:1. Ocarina CEO Murli Thirumale makes a pixelated, jazz music backed appearance.

Here’s the video in its entirety, or go to the HDS blog site and watch a higher quality version:

Meanwhile, it’s not all sword crossing in the land of the storers.

As we already know, our kind can rally for a good cause. This past week, arch rivals NetApp and EMC raised money for kids with cancer by shaving their locks for St. Baldrick’s. NetApp led the charge, and EMC responded in kind.

Virtual Geek Chad Sakac sums it up here: A little EMC/NetApp Fun - to help cure cancer…

A heartwarming effort.

That’s all for now. Remember, it’s not what you store, it’s how you store it.

Four out of five college students agree, this is not the way to deal with data growth. How about this instead?

This headline doesn’t refer to data in any kind of virtual sense of the word. Rather, there is an interesting factoid buried in a piece on the site Data Center Knowledge. Companies are finding it difficult to find big chunks of contiguous floor space, despite a growing demand.

Citing a recent survey by Digital Realty Trust, the article reads: “… 70 percent of companies planning data center expansions say they envision large projects of at least 15,000 square feet in size or 2 megwatts or more of power.”

Of all the companies surveyed, a whopping 83 percent said they plan to expand their data centers in the coming 12-24 months. Yet, the availability of this space is dropping precipitously. This could lead to a serious supply and demand crunch, according to Data Center Knowledge. Not only that, but the cost of powering these data centers is already the number one concern for many companies.

What do you think? Is this a concern for your company or those with whom you partner or serve?

Isilon Systems had some fun with this video. It’s definitely been making the rounds on Twitter. And so we thought, why not post it on our site and grab some of the fun? So for those who have yet to watch it, here’s a little weekend magic for your enjoyment.

For IT decision makers it’s imperative that you keep up with the latest news and information. Yet, the overall shakeup of the media has left many confused about where to turn. Industry pubs are getting slimmer and slimmer. Some are cutting back, others are consolidating, and a few have disappeared entirely. At the same time, the blogosphere is exploding with content. How do you sort it all out?

Here are some of the stops we at Online Storage Op make on a regular basis in order to stay up-date on IT infrastructure news without driving ourselves nuts in the process. We’d love to hear your suggestions–how do you find out what you need to know? What used to work and isn’t so much anymore? What do you wish were out there that isn’t? For now, here’s our list:

TechTarget - Still a prime source of storage news and views, particularly SearchStorage. Reporters to watch: Beth Pariseau and storage gossip watcher Simon Sharwood.

The UK Register - Chris Mellor, Timothy P. Morgan and others continue to churn out solid daily coverage of industry trends, with headlines that might make you laugh out loud.

Gestalt IT - I admit it, there are days when I don’t bother reading anything else except Gestalt to get my daily dose of storage news and views. With a solid lineup of independent writers and objective analysis on industry trends–not to mention the new addition of a humor column–it’s a one stop shop.

Network Computing - For those who used to read Byte & Switch, this is the new site that integrates it with other networking news. A necessary update in these lean times one supposes. Solid regular contributions from such writers as Howard Marks and George Crump offer simple, straightforward information and advice about products and platforms.

Wikibon -Dave Vellante and others contribute to this blog, which picks out some of the hottest trends in storage. A good way to get a quick hit on what the Wikibon analyst community is talking about.

Emulex’s Shared Items - An easy cheat sheet on what the latest industry observers and vendor bloggers are talking about. Easy to track on Twitter or through Google Reader.

Storage Monkeys - This community site has a lot going on, so I tend to just quickly check the blogs and then take a listen to the latest episode of the Infosmack podcast, which is posted each Monday morning. The blogs tend to be a little on the insiderey side, so if you’re not actively working in the data center you might find them too granular. On the other hand, the podcast is very much the 30,000-foot view of overall storage, networking and virtualization trends, served up in a highly entertaining radio format with two great hosts, Greg Knieriemen and Marc Farley.

Twitter - It sometimes seems like more trouble than it’s worth, but truth be told, the best way to find out what people in storage are talking about, worried about, and trying to fix is to sit around and listen to what they’re saying on a daily basis. Go ahead and follow a couple of Storage lists and you’re pretty much all set–here are a few I’d recommend:

We Follow Top Storage Twitter

Bas Raayman’s Storage List

So, what did I miss? Inquiring storage minds want to know.