Online Storage Optimization

Last week, a group of us participated in a groundbreaking new anti-trade show, The Business Development Event. Organized by industry veterans Greg and VaNessa Duplessie, the event was the second of its kind and the first in the Silicon Valley area. Held in Palo Alto, California, it drew dozens of storage industry members who spent three days talking, networking, making deals happen and sharing their skills and expertise.

Online Storage Optimization was on the scene–tweeting, talking and taking the occasional sip from a glass of wine that happened our way. Our parent Ocarina Networks was also featured in the “emerging vendors” showcase. And this blogger was on a panel on social media with VaNessa and Stephen Foskett, director of consulting at Nirvanix and publisher of Gestalt IT.

Here’s a small video “tribute” to the event that I hope gives a sense of it:

The BD Event, January 2010 from Sunshine Mugrabi on Vimeo.

In this video:

Nancy Hurley, CEO Bocada

Bill Basinas, Dir. of Business Development Tarmin

Alan Atkinson, Pres. & CEO Xiotech

Jerome M. Wendt, DCIG

Steve Sicola, CTO, Xiotech

Camberley Bates, Managing Director, Evaluator Group

Julie Ryan, Director of Alliances, Engenio Storage Group LSI

The next BD Event will be held in Boston this summer. Woe betide any east coast storage folks foolish enough to miss it!

An earthquake shook up San Jose (not mention Twitter) today. But that didn’t stop the storage industry’s movers and shakers from making all kinds of interesting news. Here’s a quick roundup from where we sit.

A game of musical chairs…

EMC lost storage tech consultant and blogger Steve Kenniston to inline dedupe player Storwize, where he will be Vice President of Technology Strategy. Steve continues to blog avidly and well at The Storage Alchemist. We may find ourselves crossing swords with him occasionally over here at Online Storage Op, but we always read his posts with interest.

And EMC has been no slouch in scooping up major talent:

Gestalt IT contributor Ed Saipetch (known to many of us edsai) started this week at EMC as Senior VCE Specialist. Prior to that, he was a systems engineer at Network Storage.

And Scott Lowe set tongues wagging when he announced last week on his blog that he’ll also be joining EMC, as a VMware-Cisco Solutions Principal. This seems a very shrewd move on their part, as Scott is well-known for his Cisco expertise and virtualization knowhow–both of which no doubt will be extremely handy as UCS takes off.

Nice going, EMC.

And in other news…acquisition fever!

Disk drive array subsystem provider Dot Hill has bagged Israeli storage virtualization company Cloverleaf for $12M in cash and stock, the Register reports. Clearly, they’re locking onto the virtualization and cloud storage trends with a vengeance. This may also give them a new edge in their battles with competitors like LSI and Xyratex.

Plenty of takeover rumors swirling around 3Par, although for now no one’s confirming anything. Whatever happens, it seems that everyone’s impressed by 3Par, the little thin provisioning engine that could. Over here at this blog, we’re consistently impressed and amused by their creative blogger Marc Farley. Financial pub Barron’s, in addition to initiating the speculation, called 3Par a “small but scrappy” possible takeover target. But who will be the suitor? Feel free to add in any and all rumors and speculation in our comments field below.

And don’t let us forget EMC, which in addition to snagging talent has picked up Archer Technologies, an IT governance software company that will be rolled into its overall security offerings for its RSA division. As Beth Pariseau reports on Storage Soup, the acqui could affect some others in the industry.

Writes Beth, “Archer brings with it a business continuity software module, which could affect those who manage disaster recovery in the storage environment. It also extends EMC’s move to inject automation into its software offerings, which we’ve seen in the storage market with last month’s first release of FAST, and is a part of EMC’s vision for archiving and e-Discovery.”

Well, that’s all for now. No doubt we’ll be seeing all manner of intrigue, rumor, speculation and other fascinating stuff now that 2010 is upon us.

It’s been a tough year — a worldwide recession, a sluggish housing market, rising unemployment … and on top of all that, the tarnished image of one of sports’ most squeaky clean players. Well, actually, there have been some bright spots. As DCIG blogger and storage analyst Jerome Wendt notes while looking back at the past year, “Deduplication is the Big Success Story of 2009.”

Wendt writes: “Deduplication is arguably one of the most notable trends of 2009 as it has been widely adopted by users after bursting onto the scene just a few years ago and has grown to be included in both software and hardware products.”

Wendt focuses on dedupe for backups, where there has been much publicized activity over the past year. The big storage story of 2009 was of course the battle between storage titans EMC and NetApp over backup dedupe specialist Data Domain. He cites an industry survey from SearchDataBackup that indicates that 41% of enterprises either are or are seriously considering dedupe to control data growth and costs. He also notes that the despite the predicted demise of Quantum, that dedupe company remains strong.

Dedupe for backups is one part of the cost reduction puzzle. Another part is to reduce data at the source, in primary storage. This is of course the specialty of this blog’s parent Ocarina, which implements a unique combination of content-aware dedupe and compression to achieve startling results. It focuses on the very types of unstructured data that are driving storage growth today–emails, images, documents, and so on. The company has been partnering with almost every leading storage provider, including HP, EMC, HDS, BlueArc, and Isilon. Another  leader in this space is NetApp, which has a strong dedupe for primary offering that has also garnered a great deal of attention.

Here’s the thing, the economy might be slowing down, but data growth continues apace. This is one reason that the storage industry has been thriving this year. But rather than standing still, what is spells is a concerted effort to keep that data under control. As Wendt notes, another of the year’s big trends is cloud storage, which offers companies more flexibility for storing some percentage of their data. I would also add that virtualization has taken a huge leap forward, not only in terms of the technology itself, but also in terms of adoption over the past year. Yet another way to attack the problem.

So if 2009 was all about dedupe for backups, I’m going to guess that 2010 will be very much about data reduction at all points on the data life cycle. What do you predict?

Image: Gizmodo

Two items jumped out at me this week. First, Francine Hardaway’s post on being an early adopter, “Do You Write in the Cloud?” and second, this New York Times story about Cormac McCarthy’s typewriter.

Hardaway, a serial entrepreneur and founder of Stealthmode Partners talks about how she and other bloggers compose their text. Mashable’s Pete Cashmore, along with Buzzmachine’s Jeff Jarvis and Gina Trapani all write in the cloud. Hardaway either writes directly in Gmail or sends one to Posterous from her Gmail account. If there is a bad wifi connection, she uses Evernote and synchs it to the cloud before moving it to Wordpress. (Am I the only one who composes directly into Wordpress?)

The main point here is that Hardaway illustrates how she was ahead of the curve each time, and was able to show successfully that you could still think and write using the latest “newfangled” tech–whether a manual typewriter or a word processor. Hardaway recollects:

“…50 years ago, I started composing on a manual typewriter when most other people composed in longhand and transcribed.  Many famous authors wrote in longhand their entire lives. People thought I was crazy for going straight to the typewriter. But I’ve gone seamlessly from the typewriter to the word processor (we could dictate into a  Digital Equipment Corporation word processor in the Maricopa Community Colleges in the 70s, when I was a professor.)  They thought I was crazy for dictating instead of typing when I did that.”

All of this was sloshing around in my brain when I came upon the aforementioned story of Cormac McCarthy’s typewriter. McCarthy is the author of a number of bestselling books, including the recent “The Road“–now a movie, and “All the Pretty Horses,” among many other award winners. How did he compose his text? On a portable Olivetti typewriter he bought at a pawn shop in 1963. He moved on from long hand, but after that he just stopped.

We’re talking about two ends of the spectrum. The early early adopter. And the … well, I don’t think we can call Cormac McCarthy a late adopter now can we? Of course, most people are somewhere in the middle. Enterprises are managing an influx of data that is based on what would be considered the median behavior–a need to store all manner of files, from Office documents to family snapshots to the latest Lady Gaga download. Oftentimes, the response is to try to ignore the problem. More and more files are shoved onto expensive primary storage. Suddenly, the problem reaches massive proportions and emergency measures are taken, rather than a well thought out tiering/deduplication plan or a move to a reliable cloud storage platform. But I digress. The main question here is, what kind of adopter are you? And what are the strengths and weaknesses of each approach?

Excitement is building here at Ocarina as tomorrow is the big kickoff for Tech Field Day, and we’re a presenting sponsor of the event. As we speak, cables are being laid, demos are being run through, and a certain amount of nervous excitement is in the air.

In just two short days, a horde of smart, savvy bloggers will descend on the Ocarina offices on Airport Parkway in San Jose, bearing thumb drives packed with their toughest data sets–part of a challenge we laid down in order to show them how well we can compress and dedupe that data.

Sponsored by online IT pub Gestalt IT, the next two days are sure to be worth watching even if you’re not directly involved. Tech Field Day seems to me to represent a new kind of experimentation with social media in both its on- and offline form. The event brings together influential bloggers and others from around the world for two days of hands-on demos at storage and virtualization companies around Silicon Valley. As I mentioned on the VMWare Communities Roundtable today, this group of presenting sponsors represent Silicon Valley at its innovative best. And the fact that they were quickly able to recognize the value of such an event is a further testament to their forward thinking.

The chatter on blogs and Twitter in the weeks leading up to this event has been intense, which makes it even more exciting to be a part of. So, what the heck is this Tech Field Day that everyone’s talking about? It’s a fairly simple concept. Gather a group of the leading independent bloggers in IT, storage, virtualization and related fields and take them to companies around Silicon Valley for a deep dive on some cutting edge technology they may not yet know a whole lot about. No slick marketing presentations, no PR. Just plain, straightforward geeking out. Gestalt IT’s founder/publisher Stephen Foskett has said he plans future events as well, and they will encompass other areas of high tech.

Social media made this thing possible, and it’s making it easy to follow as well. Here are some of the ways to get in on the fun:

Twitter: Hashtag #TechFieldDay

FriendFeed

Or, The Official Tech Field Day Scoreboard, which aggregates all of the posts *and* tweets about the event in real time.

See you there!

What a long, strange trip it’s been for storage recently. First, there was the disaster that turned out to be less of a disaster than we thought. All those Paris Hilton phone numbers and Perez Hilton black book emails lost to a Sidekick in the ribs … But then it turned out all was not lost — in fact, most data was found, wagging their tails behind them.

Many in the industry have been howling about how unfair it was that the entire cloud took such a massive hit in light of the Sidekick meltdown. But then the skies cleared, and in his Enterprise Storage Strategies blog, Stephen Foskett came out with a surprising argument. All the more so as he works in the very industry that has been so massively hammered by the whole Danger/Sidekick/MS/EMC/HDS/cloud/whoever-else-we-can-blame disaster.

He writes: “Although my professional focus is at the forefront of the cloud storage wave, I can not disagree with the content of articles with sensational headlines like “Cloud Storage: It’s Strictly For Airheads” and “Why Cloud Storage Use Could Be Limited in Enterprises“. The authors are doing exactly what everyone should be doing: Questioning the viability and suitability of cloud storage in the enterprise.”

I agree, and would add that this should be the case when considering other solutions, including, yes, data reduction. In fact, our recent post on dedupe misconceptions has gotten all manner of attention recently for its even-handed response to alarmism about the necessity for dedupe for primary. It garnered a quick mention in Simon Sharwood’s roundup of storage blog-o-tweet-osphere smackdowns on SearchStorage. And there is plenty more to say about this–so much in fact that it doesn’t fit into this small space.

Not to worry, because over at his new Isilon blog, Nick Kirsch is asking for input on deduplication for primary, nearline and backups. Is it here to stay, or a craze? He asks. A good question, and one that might be answered with another question, “how fast is unstructured data growing?”

Finally, Online Storage Optimization hit the big time this week, getting a mention in the Forbes Velocity blog. Staffer Brian Caulfield, in search of a way to promote his new blog, called upon our wisdom of booth babes.

Happy weekend to you all.

Bo Peep Image: http://www.teachersandfamilies.com/nursery/bopeep.html

Friday’s CNBC segment is worth a watch if you’re wondering where the greatest growth will be over the next decade in the technology sector.

“Everything that we are using now–videos on YouTube. Everything we’re doing on our phone. On and on. It’s all about storage and putting stuff on the net,” opines Roger Nusbaum of Your Source Financial, who adds, “Cloud computing comes into play here.”

This year’s VMWorld ended with a bang–as rock legend Foreigner played for a crowd of 10,000 virtualization fanatics at the Yerba Buena Center in S.F. This really was one heck of a jampacked and exciting event. It is clear that this is becoming one of the key shows for the virtualization and storage markets. There were two–count ‘em!–EMC acquisitions. The big buzzword was “cloud.” There were more than a few product launches, and even a few new companies on the scene.

On a more personal note, this was a great way to connect with a whole lot of storage bloggers that I’d met online but not in person. The storage “tweetup” on Tuesday evening drew a crowd that could be called a veritable who’s who of the storage blogging community, including Marc Farley, Stephen Foskett, “HPStorageGuy” Calvin Zito, NetApp’s Alex McDonald, Dave Vellante of Wikibon, Ed Saipetch, “Storagenerve” Devang Panchigar, and others. I also met EMC bloggers Dave Graham and Chad Sakac, and NetApp blogger Vaughn Stewart. What a talented community!

One thing that made VMWorld stand out was the diversity of companies represented there–from huge heavyweights such as EMC to newish tech cos. like Veeam to startups just coming out of stealth. Perhaps the hottest of these new entrants was EvoStor–which I mentioned in yesterday’s post. As Storage Switzerland’s George Crump says in a recent VMWorld dispatch on Byte and Switch:

“EvoStor, a VMware only, first that I have seen, storage solution. This is more than focus, this is exclusive support of a single environment … The system is basically a storage grid similar to other scale-out models like LeftHand or Isilon but again, focused on VMware.”

Also this week, EMC went on a shopping spree. Tuesday, we learned it had picked up e-discovery specialist Kazeon. As Stephen Foskett notes in Gestalt IT, “EMC will likely integrate the Kazeon technology with their SourceOne archiving and discovery platform.” He also says that while EMC probably paid a fair price for the technology–just $75 million–it’s not without its challenges.

A day earlier, on the first day of VMWorld, EMC announced it had acquired “application image management” company FastScale. On his blog, Chuck Hollis describes this as the final piece in a puzzle that it has been assembling for some time with Ionix. He calls their method of prepping images for virtualization “predupe,” reducing footprint in memory.

As Beth Pariseau writes in Storage Soup, the FastScale acqui is a sign that EMC and VMWare are acting more like two interrelated companies. The more logical acquirer would be VMWare, since this is a server virtualization company. This could signal a shift in focus away from their purely hands-off approach that meant that it sometimes even signed agreements with EMC arch-competitor NetApp. Time will tell where this is leading.

There’s much more to say about the show, but this will have to do for now.

There’s been a lot of discussion about tiered storage lately. Most notably, Stephen Foskett has written a series of posts on the topic on his Nirvanix blog, Enterprise Storage Strategies. In his latest post, he essentially argues that tiered storage hasn’t turned out to be cost effective and that cloud storage could be the best option for the lower tier.

We certainly agree with him that unstructured data has become unmanageable due to the proliferation of rich media and other large files. We also agree that tiered storage hasn’t lived up to its promise to a large extent. However, let’s not be too quick to throw out the baby with the bathwater. As Hu Yoshida has discussed in a recent post, tiering has come a long way in light of new technologies, particularly virtualization. In our view, by combining virtual tiering at the block level (as described in Hu’s post) with virtual tiering at the file level you can get the best of both worlds.

Tiered storage used to be about moving data from one physical storage place to another. The premise there was that some storage was fast and expensive, and other storage was slower but cheaper, and that you could save a lot of money by moving data to the appropriate place.

This was a good idea in theory, but as it turned out there were a number of unforeseen problems. First, the tools for moving files were themselves sometimes expensive. There goes your cost savings. On top of that, they were sometimes good at moving the files but not at getting them back. And further, in situations where the fast tier and the cheap tier were not from the same vendor, it often proved difficult to make finding files that had been moved transparent to users and applications. As you can guess, these types of problems often made the whole thing more trouble than it was worth.

The fact remains, though, that most files are stored on storage that has more performance, and costs more, than is necessary for that file. Most storage admins know that 80% of their files could be stored on a cheaper tier, if it wasn’t a hassle or too expensive to do so.

One solution with immense potential is to have virtual tiers within a single filer or namespace. Virtual tiers are levels of dedupe and compression applied to a file, making it cheaper to store because it’s taking up less space. In a virtual tier, the file does not have to move anywhere – it can stay right where it is, but you reduce the cost of storing it by shrinking it. With dedupe and compression, there are lots of choices for trading off performance versus space savings.

Sun’s file system ZFS allows this, and cloud storage like Nirvanix can do this too — having the advantage of using the  latest technology, and that the technology behind the cloud interface is invisible to the user of the cloud. Either way, let’s look at how you can implement virtual tiers while keeping files in the same place that they were created in.

Let’s say Tier 1 is for your fast hot files - they live on your Tier 1 filer, uncompressed.  In that case, you might have a Virtual Tier 2 be all the files that have not been modified in 7 days, and Tier 2 would be that same filer, same volume, but with a policy that those files that meet the Tier 2 definition are deduped. No compression, just dedupe. In that case, read back times will be quite fast. Maybe not exactly as fast as reading the original un-deduped file, but almost.

A Virtual Tier 3 might be “files that have not been modified in 30 days” and the tier might be defined as dedupe plus light compression. Read back will be a tad slower, but space savings greater than dedupe alone.  Finally, you might have a virtual Tier 4 – dedupe and maximum compression. This might fire more complex compressors that take longer to compress (and decompress) a file, but will get excellent space savings. Read back performance for tier 4 might be quite a bit slower, but the space savings might be 90% or more reduction in the file sizes in that tier.

Here’s the kicker: All of this can be done without moving a file off the filer it started on. Users and applications can still find the file right where it always was. If they access the file, the optimization solution will transparently “rehydrate” the file.

There are different solutions that can do some or all of these things today. NetApp’s dedupe can only dedupe all of the files in a volume or none, so it can’t be used today to create logical or virtual tiers within a volume. But other solutions, like the Ocarina ECOsystem, are policy-based and can be used to create multiple different logical (or virtual) tiers within a single filer or volume, with multiple dedupe settings (including Ocarina’s patented Object Dedupe) and multiple levels of compression, with choices of over 100 compressors for different file types.

Ocarina has been tightly integrated with certain types of storage – including  cloud solutions like Nirvanix – and the most transparent virtual tiers would be with the combination of Ocarina and one the filer choices that have tight integration with Ocarina: BlueArc, EMC, HDS HNAS, HP, Isilon and Nirvanix (in alphabetical order – no vendor prefences implied!).

Of course, virtual tiers can be combined with real physical tiers, so that you can combine the level of storage optimization (dedupe, compression) with storage of different physical characteristics (expensive filers, cheap filers, cloud storage) to provide an environment that is not just a simple two-tiered model but a policy-driven environment of possibly a dozen or more logical tiers, with files being tiered-in-place or migrated-and-optimized automatically based on policy with little or no storage admin involvement.

As you can see, there is vast potential in this new approach to tiering. Even better, it can be achieved in such a way that storage admins’ jobs become easier, rather than harder. Like a lot of things, storage tiering has always been a good idea, but sometimes the technology has to catch up with the idea for implementation to become a good idea.  Given the growth of storage, and the improvements in physical and virtual tiering, I think doing a better job of tiering must rank close to top of the list for many customers.

Not so funny story. My dad called me up today and told me that he received a letter from his credit card company, Chase. The letter stated that one of their tape backups had been lost. That tape contained information including his name, address, and social security number. The tape was held in a vendor’s warehouse, and had somehow been mislaid.

“You’re in this storage business. What do you think of this?” he asks.

“I think it shows that tape isn’t necessarily the safest way to back up data,” I answer. “Tapes can be mislaid, mishandled–even stolen. Let’s hope that latter possibility isn’t what happened here.”

According to the letter he received, the information on the tape is encrypted and would require “special software” to read. Also, the information that was lost didn’t include my father’s financial data, bank or credit card numbers. Still, he was worried by the letter. And I can understand that. For all the fears people have about the cloud, this is a fairly low tech example of how security breaches can occur. He said he didn’t know how many people were affected–or received the letter–but I’m waiting for a TechCrunch article on this to come out any day now.