Online Storage Optimization

Four out of five college students agree, this is not the way to deal with data growth. How about this instead?

This headline doesn’t refer to data in any kind of virtual sense of the word. Rather, there is an interesting factoid buried in a piece on the site Data Center Knowledge. Companies are finding it difficult to find big chunks of contiguous floor space, despite a growing demand.

Citing a recent survey by Digital Realty Trust, the article reads: “… 70 percent of companies planning data center expansions say they envision large projects of at least 15,000 square feet in size or 2 megwatts or more of power.”

Of all the companies surveyed, a whopping 83 percent said they plan to expand their data centers in the coming 12-24 months. Yet, the availability of this space is dropping precipitously. This could lead to a serious supply and demand crunch, according to Data Center Knowledge. Not only that, but the cost of powering these data centers is already the number one concern for many companies.

What do you think? Is this a concern for your company or those with whom you partner or serve?

Isilon Systems had some fun with this video. It’s definitely been making the rounds on Twitter. And so we thought, why not post it on our site and grab some of the fun? So for those who have yet to watch it, here’s a little weekend magic for your enjoyment.

For IT decision makers it’s imperative that you keep up with the latest news and information. Yet, the overall shakeup of the media has left many confused about where to turn. Industry pubs are getting slimmer and slimmer. Some are cutting back, others are consolidating, and a few have disappeared entirely. At the same time, the blogosphere is exploding with content. How do you sort it all out?

Here are some of the stops we at Online Storage Op make on a regular basis in order to stay up-date on IT infrastructure news without driving ourselves nuts in the process. We’d love to hear your suggestions–how do you find out what you need to know? What used to work and isn’t so much anymore? What do you wish were out there that isn’t? For now, here’s our list:

TechTarget - Still a prime source of storage news and views, particularly SearchStorage. Reporters to watch: Beth Pariseau and storage gossip watcher Simon Sharwood.

The UK Register - Chris Mellor, Timothy P. Morgan and others continue to churn out solid daily coverage of industry trends, with headlines that might make you laugh out loud.

Gestalt IT - I admit it, there are days when I don’t bother reading anything else except Gestalt to get my daily dose of storage news and views. With a solid lineup of independent writers and objective analysis on industry trends–not to mention the new addition of a humor column–it’s a one stop shop.

Network Computing - For those who used to read Byte & Switch, this is the new site that integrates it with other networking news. A necessary update in these lean times one supposes. Solid regular contributions from such writers as Howard Marks and George Crump offer simple, straightforward information and advice about products and platforms.

Wikibon -Dave Vellante and others contribute to this blog, which picks out some of the hottest trends in storage. A good way to get a quick hit on what the Wikibon analyst community is talking about.

Emulex’s Shared Items - An easy cheat sheet on what the latest industry observers and vendor bloggers are talking about. Easy to track on Twitter or through Google Reader.

Storage Monkeys - This community site has a lot going on, so I tend to just quickly check the blogs and then take a listen to the latest episode of the Infosmack podcast, which is posted each Monday morning. The blogs tend to be a little on the insiderey side, so if you’re not actively working in the data center you might find them too granular. On the other hand, the podcast is very much the 30,000-foot view of overall storage, networking and virtualization trends, served up in a highly entertaining radio format with two great hosts, Greg Knieriemen and Marc Farley.

Twitter - It sometimes seems like more trouble than it’s worth, but truth be told, the best way to find out what people in storage are talking about, worried about, and trying to fix is to sit around and listen to what they’re saying on a daily basis. Go ahead and follow a couple of Storage lists and you’re pretty much all set–here are a few I’d recommend:

We Follow Top Storage Twitter

Bas Raayman’s Storage List

So, what did I miss? Inquiring storage minds want to know.

This blog’s parent Ocarina and I have something in common. Can you guess? That’s right. We both have names that are cause for frequent comment. The first question anyone ever asks me when they meet me is, “is that your real name?” And the first question that anyone asks a person from Ocarina is “where did you get the name?” In essence, this is the same question in two different forms.

I almost always answer in the same way when asked about my name. I confirm the fact that my parents were indeed hippies (as the questioner already suspected) and that yes, I do live in California and always felt I belonged here. My full name actually means “Sunshine from the West,” something I have noted in one of my many blog entries on the concept known as “aptonyms.”

For the folks at Ocarina, the response is to explain that the name came from the founders, who decided that it sounded good. They also liked that it is a real word, rather than some kind of computer-generated mashup of random “Xs” and “Zs.” Most people are satisfied with this answer, though it does tend to be a bit of a conversation stopper. You want them to draw some connection between the clay flute that is the actual ocarina and the company mission, which is to reduce data through content aware compression and deduplication. But truth be told, there isn’t one.

As someone who spends a good deal of time tracking what’s being said on blogs, Twitter and other social media platforms, the name Ocarina poses a bit of a problem. The Ocarina of Time is a game within the extremely popular Nintendo series known as the “Legend of Zelda.” Hundreds of tweets are posted almost every day about this game, and numerous blog posts–all of which clog up my RSS feeds and Twitter search data. There’s even a video celebrating the music of the game, which has had over 65,000 views on YouTube for some unfathomable reason.

Here it is for all you Zelda freaks:

There is also an iPhone app called the “Ocarina” that allows you to play your own iPhone and share your music with others. Again, very popular–which leads me to wonder if there’s something about the name Ocarina that naturally resonates with people. What do you think? Are there any other associations you have with the name you’d like to share?

Last week’s BD Event was more than just a deal making event. It was a chance to learn about new product releases and trend in the storage industry. The big picture: gone are the days when end users had to accept whatever the storage industry handed down to them. Today’s small-to-medium-sized storage operations are all about designing systems in response to customer needs. Whether that’s developing end-to-end dedupe, refining and improving processes for data recovery management, delivering automated marketing tools, improving data migration, or creating storage that is more energy efficient, the push is towards designing systems with real world customer needs in mind.

The BD Event organizers’ deep connections within the storage arena meant that the two-day conference in Palo Alto drew a who’s who of industry folks. I was particularly pleased to see the number of analysts and consultants on site, including Jerome Wendt, George Crump, Deni Connor, Dave Vellante, Stephen Foskett and Tony Asaro (who unveiled his new project, Voices of IT). I also spoke at length with storage writer Howard Marks, who has a new project called DeepStorage.net that looks very promising for companies seeking solid research that they can use as outbound marketing.

Pleasingly enough, this blog’s parent Ocarina was very much the talk of the conference after kicking off the first day’s emerging vendor showcase. Carter George, VP Products gave away the fact that end-to-end dedupe is becoming a part of the overall strategy for the company. This information set tongues wagging. As DCIG’s Jerome Wendt later blogged: “Ocarina Networks is another company that is adapting to new demands from its customers. Originally it started out doing post-process deduplication of large image files (JPGs, MPEGs, etc.) that had been dormant for 30 days or more - great stuff! But now its customers and even OEMs (Ocarina did not say who) are coming to it and asking for it to do end-to-end data deduplication from primary disk to backup disk without ever reconstructing it. After all, once the data it deduplicated on primary storage, why reconstruct it to then deduplicate it again when it is backed up?”

A good question, and one that was hotly debated and discussed among those in attendance. As Jerome notes, this is a perfect example of the customer responsiveness trend. It’s also an acknowledgment of something that’s been obvious to end users for some time–data reduction shouldn’t have to be isolated within each storage sector. In this day and age you really shouldn’t have to buy separate products to dedupe within primary, nearline, and backup. It’s like having to buy a separate dishwasher for your pre-rinse, wash, and dry cycles.

Other standouts at the event included Bocada, which has updated its DR management software by introducing a new product, Prism. I plan to have the CEO Nancy Hurley on my podcast, and so will learn more about how this update is serving its existing and new customers. I confess that I went to her presentation mainly because I wanted her on my show, but I quickly realized that there was something here of note. That is, the company is addressing a real gap in how well these processes are managed and improved, a key consideration with a crucial component like data protection. She gave a brief overview of the user interface, and on its face it seemed intuitive and flexible.

TechValidate also served as a great example of a company that has evolved based on customer needs. As CEO and founder Brad O’Neill explained during his emerging vendor presentation, originally the company was formed to serve companies that were having trouble getting customer references. These all-important testimonials are sometimes difficult to get–as many industries are gun shy about trumpeting their connections with too many IT and storage vendors. However, O’Neill soon recognized a larger need among its customers for usable marketing materials that could be generated from the information they were gathering. Now, the company has a wide range of customers across numerous industries that are using it as a way to serve up marketing publications.

One final highlight of the event–I got to speak with the NetApp blogger known as “Dr. Dedupe,” Larry Freeman. Larry is best known for running around in a lab coat and stethoscope asking people if they know anything about dedupe. The videos of these shenanigans are posted on his blog and on NetApp TV on YouTube. I suppose in a sense he and I are competitors. Turns out, he’s been writing a book, “Evolution of the Storage Brain” and posting it as he writes it, chapter by chapter, on his blog. This means that readers have a chance to comment on it and shape it as it goes along. Check it out!

Last week, a group of us participated in a groundbreaking new anti-trade show, The Business Development Event. Organized by industry veterans Greg and VaNessa Duplessie, the event was the second of its kind and the first in the Silicon Valley area. Held in Palo Alto, California, it drew dozens of storage industry members who spent three days talking, networking, making deals happen and sharing their skills and expertise.

Online Storage Optimization was on the scene–tweeting, talking and taking the occasional sip from a glass of wine that happened our way. Our parent Ocarina Networks was also featured in the “emerging vendors” showcase. And this blogger was on a panel on social media with VaNessa and Stephen Foskett, director of consulting at Nirvanix and publisher of Gestalt IT.

Here’s a small video “tribute” to the event that I hope gives a sense of it:

The BD Event, January 2010 from Sunshine Mugrabi on Vimeo.

In this video:

Nancy Hurley, CEO Bocada

Bill Basinas, Dir. of Business Development Tarmin

Alan Atkinson, Pres. & CEO Xiotech

Jerome M. Wendt, DCIG

Steve Sicola, CTO, Xiotech

Camberley Bates, Managing Director, Evaluator Group

Julie Ryan, Director of Alliances, Engenio Storage Group LSI

The next BD Event will be held in Boston this summer. Woe betide any east coast storage folks foolish enough to miss it!

Interesting story from the vault of the Ocarina case study library. Social network Tagged is the third largest social network in the U.S. It has seen traffic increase 10x over the past two years. With its focus on making new friends rather than simply getting to know existing ones, it has carved out a successful niche and is building an international subscriber base of over 80 million members.

The cost of this success? Data growth. Tagged’s storage infrastructure has been doubling every single year. With 1 million new photos uploaded every single day, Tagged needed a way to expand capacity and fast.

Compression with Ocarina meant about 10 TB of additional free space, which in turn meant they could put off buying new NAS equipment by several months. The lower average image size also meant reduced bandwidth and 15%-20% reduced monthly content delivery network (CDN) costs.

The company chose to go with Ocarina’s newest specialized image reduction technique, native format optimization (NFO). This is visually lossless compression of images that nevertheless delivers significant space savings–a technology that’s perfectly suited to the social networking environment.

The other crucial benefit to reducing image size was improvements in site responsiveness. “We’re sure that using Ocarina to reduce image sizes has helped improve our page rendering times,” said company CTO Johann Schleier -Smith. “That’s a big deal because it creates a better user experience, which means improved customer loyalty and higher market share.”

Read the entire case study by clicking here. Or visit the Ocarina resources page and click on the Case Studies tab, where you’ll find several others.

Bleary-eyed, the storage industry has begun to wake up from its holiday stupor. VMware has decided to go into the email business. EMC continues to vacuum up talent like a Roomba on a tear through the world’s biggest living room. Meanwhile, the jokers over at Gestalt IT are picking up the “Fake Steve Jobs” meme and running with it. Their version is actually funnier than the original — at least to this blogger, perhaps because I know the players and situations.

The increasingly crowded and competitive Storage Monkeys Top Vendor Blogs contest is about to screech to its exciting conclusion. Voting ends Friday. Front runners are EMC bloggers Chuck Hollis and Storagezilla. Third place at the moment is the HP Storageworks blog, helmed by fearless blogger Calvin Zito. This puts early front runner Marc Farley, founder of the vaunted Steering Wheel Camera Society of America in fourth place. Step on it, Marc! In fifth right now is the Storage Anarchist, Barry Burke, who is just barely edging out NetApp’s Val Bercovici. Well, it ain’t over till it’s over–these things can change fast.

So before the week is out–why not VOTE?

Speaking of which, blogger extraordinaire Stephen Foskett has started a series that delves into the whole vendor blogging question. He has two posts up on the topic, “Vendor Bloggers 1: Why Does It Matter?” and “The Spectrum of Vendor Blogs.” Mr. F cites none other than Online Storage Op as an example of a hybrid “independent-seeming official” blog, but credits us for being transparent about the fact that our parent is a company. No doubt Stephen and I will hash this out further when we give a talk on social media to a group of storage industry pros at The BD Event in Palo Alto next Wednesday.

But wait… there’s yet more news, and this is actual news:

Nexsan and FalconStor are teaming up to try to defeat rival Data Domain. It can get really interesting when two vendors come up with a combination product that serves a larger purpose than they would’ve had if they acted alone. Two pieces on the topic caught my eye this week:

Beth Pariseau, SearchDataBackup - Nexsan and FalconStor gun for EMC Data Domain with Dedupe SG 2 data deduplication backup device

Writes Beth: “Analysts say a series of updates to Dedupe SG — comprised of FalconStor’s dedupe software and Nexsan enterprise data storage systems — put it into closer competition with the 800-pound gorilla Data Domain.”

She quotes ESG’s Lauren Whitehouse, who says that the high-availability config on this combo is a poke in Data Domain’s eye. And Dave Vellante of Wikibon calls the bundle the “best of both worlds” due to the fact that it’s compatible with existing home office systems and reduces data over the WAN–though he questions how it will do in real world deployments.

Joseph Kovar, ChannelWeb - Nexsan, FalconStor Join Forces On Newest Backup Appliance

Joe, for his part talked to Greg Knieriemen at Chi Corp., which partners with both Nexsan and FalconStor who is impressed with among other things the potential inherent in its 10-GB-ethernet option. Hmm, where have I heard that name Greg Knieriemen before?

Well, that’s all for now folks. Maybe next time I talk to you I’ll be checking my zMail.

There’s a lot of talk about compression these days, but how much do we know about it? Well, for one thing, compression as a research area for mathematics has evolved much faster than most people realize. The thing is, most compressors used in computer products, including dedupe appliances, use generic algorithms rather than making use of these advances.

Most storage products use Lempel-Ziv (LZ) or derivatives, and try to use that single compressor to compress everything. These algorithms have been around forever, and for the most part, have not evolved much in the last ten years other than in the area of performance. This is too bad, because compression has advanced in exciting ways. LZ and its cousins work well on the kinds of data that were around 10 or 20 years ago - plain text, plain numbers, or combinations of those things. They do not work so well on a lot of modern data - images, video, Office documents, PDF’s, already-compressed files like Zip, encrypted data, etc. What’s important to understand is that all the most notable advances in compression that apply to storage have taken place not in generic compression algorithms, but in file type specific ones. File type specific compressors can, in fact, deal with all those modern data types.

Compression is all about pattern recognition and prediction. You look for patterns in a file and if you can find those patterns you try to predict their occurrence. If you can predict a pattern, you can compress it. So understanding the kinds of patterns that might show up in a file - video, a Zip file, music, and a PowerPoint are all very different - is the key to building a compressor for that file type.

What’s especially relevant is that the most important thing in compression of data today is recompression. Almost all of the file formats that are driving data growth, and taking up the most space on backups, are already compressed. Think of a file type that’s eating up space, and it’s likely to an already-compressed format: JPEG, video, Office, PDF, mp3, medical images … all compressed already.

A generic compressor won’t get any results at all on an already-compressed file. That’s because the first compression obscures the patterns that a compressor would look for. That’s why if you try to compress, say, a Zip file, if anything you’re likely to make it bigger. Recompression means first decompressing the file and then recompressing it with a better compressor. To do that, you have to recognize what kind of file it is, what kind of compression has been applied, and how to decompress it. By first decompressing it, you are able to see and process the patterns that make better prediction and compression possible.

Almost every market has a set of well-defined file types that make up the bulk of its unstructured data. In medical imaging, it’s Dicom (which in turns contains JPEG 2000, JPEG LS, and TIFF). In seismic, it’s seg-y. In satellite imaging, it’s NTF, MrSID, GeoTIFF and a few others. In the average business, it’s Office, PDF, photos and video.

In specific industries, you see very advanced compression implemented in the application layer, not in storage. Video is a great example - the whole concept of the video codec is all about compression. Whole companies exists specifically to do better video compression (On2 is a good example), but this compression is done primarily for transmission, and implemented as part of the video application workflow, not as a storage technology.

In a world that had all plain ASCII text data, generic compressors would be great. But that’s not the world we live in. For compression to have any meaningful impact on today’s data sets, you have to have file type aware recompression.

It’s a shame that most storage products today have not implemented the most exciting advances in modern compression mathematics. My company Ocarina is quite frankly one of the few exceptions. The compressors found in tape drives or in dedupe appliances represent the best of the evolution of the generic compressor. The thing to look for going forward is the emergence in storage products of the next generation set of file type aware compressors, which is where all the action has been over the last ten years.