VM Squad

I’ve uploaded pictures of our DR and HQ setups and included notes on everything installed in the racks. If you have any questions please feel free to post a comment.

Server Rack set on Flickr

So a month at the new day job and I’m tasked with getting a new Cisco ASA 5510 configured and ready to install at our DR site. Last thing I worked on with Cisco was an old 1700 router so not exactly the same but oh well. We already have one 5510 installed at our main office, copied the config, updated the boot and asdm images and ready to go. Changed the IP addresses for the new router before putting on the network and tested to make sure everything came up correctly on reload and it did. Hooray for me, always nice to have something actually work when you have no experience with it.

Our DR hardware consists of an IBM x3650 server with dual quads and 32GB of ram and one LeftHand storage module that has a total of 1.6TB of storage I think. We’ve had this equipment installed and running here at our main location for a few months and one of my tasks when I started this new job was to get this DR equipment actually installed at the DR site. So yesterday the boss and I took a road trip with the equipment up to the DR site. We have a 100mbit link TLAN link from Cox between our HQ and DR sites so replication is not an issue. The only major issue we had setting everything up was actually getting the link to come up between the two sites. It had been installed for three months, we were getting a link up there but Cox said they couldn’t see anything at our HQ location. Upon getting back to the office, noticed that the cable was unplugged. Yea it kind of helps when things are actually plugged in but lesson learned.

So now I get to continue planning for a failover test later this month to show the people who sign our checks that yes, this money actually did buy something useful.

Marc from Dell posted a reply to my More Meetings post where I talked about the concern I have with the upgrade process with their Equallogic SAN. According to the sales rep we had out last week it takes on avg 15 seconds to upgrade the controllers inside their storage array. It first upgrades the “passive” controller then takes over as primary so it can upgrade the “active” controller. This process takes around 15 seconds and during the switch over all network access is disabled which i understand completely. What raised my and another admins brow was when we asked about data corruption during that upgrade window we were just told it won’t happen. He didn’t explain why it won’t happen or what in iSCSI is able to handle the lag in communication. He only said it won’t happen.

I’m sorry but I have had to eat my own words too many times in the past by putting faith in a product only to have it let us down and I just will not do that with my companies data. Just think if we all put blind faith into something a sales rep said and never questioned anything. We’d all be wearing Bill Gates t-shirts and blindly upgrading our desktops to the newest and latest from redmond.

And I was going to mention how immature I think it is that some companies so directly attack their competition instead of relying on facts, but then I remembered who is involved. Guess those in Texas just handle things differently.

Well as meetings progress and executives sit impatiently, our D.R. plan is slowly coming together. Having to test and choose products is always a great learning experience, though doing this with deadlines takes some of the fun away.

We’ve already sat through the presentations from Lefthand and Equallogic since we’ve narrowed our choices for our production SAN down to these two venders. Just from meeting the reps from both of these companies, I’d have to say that the Lefthand guys seem more down to earth and know what the hell they are talking about. The Equallogic reps we’ve had out are just salesmen who have had minor hardware training and only answer questions with generic answers. The Lefthand guys do this for a living and for fun it seems, the Equallogic guys are just salesmen, period.

One of the biggest downsides for us with Equallogic was that when it comes down to upgrading the firmware on the modules, it will take down network connectivity for 15 seconds while it switchs over to the secondary controller inside the array. I don’t know about other companies and how they would handle their SAN being down for 15 seconds a couple times a year but I’ll pass. With Lefthand, because of their network RAID design, no downtime is required. I’d have to say we’re leaning very heavily towards Lefthand for the SAN purchase.

We’ve settled on VMware 3.5 for our virtualization needs. Virtual Iron will hopefully make it into our plans some day but they just aren’t mature enough for what we need. I’d like to actually use the licenses we’ve purchased for Virtual Iron to create a testing lab for our dept since we just don’t have space for phsyical machines.

And I finally found the time to install and configure Cacti to replace Servers Alive. The newest release from CactiEZ is really the best and easiest solution i’ve found for the price paid, FREE! The features that cacti has and the plugins available really do compare to most enterprise solutions that cost a bucket full of cash. I have almost all of our servers setup and am going through networking equipment to add as needed. In the past if a developer said they needed a new server because a current server just couldn’t handle the load, we would just buy it. Now with performance monitoring and tracking I can ask them to prove the need. In the past month i’ve had to pull statistics on a few servers for this exact reason.

Well it’s been awhile since I’ve posted on anything I’m doing at the day job.

Because of the usual budget crunch I’m building an Openfiler server to use as a temp solution. It’s amazing how much the lack of space across the board hinders your overall day to day job.

We’ve pretty much settled on VMware ESX 3.5 for our virtualization platform. Virtual Iron really just isn’t up to the table all the way when it comes to the extra features that VMware offers. They may seem like minor features but honestly it’s what makes VMware the best solution for us.

I’m also really looking hard for something to replace our current monitoring solution that is way long in the tooth. A previous administrator had Servers Alive setup for simple monitoring of availability and Solarwinds setup to monitor bandwidth across our VPN concentrators and frame routers. I would really like something open source of course, but something that combines availability and bandwidth monitoring. Alerting is a must, we use email but are considering setting up SMS for emergency alerts. Any suggestions? Please drop them in the comments.

Thanks, and I’ll try to keep this updated a bit more.

John Spiers from Lefthand has a blog called the Virtual View. I love blogs like this. You get the true unfiltered opinion from these guys. There are a few blogs like this where the owners truly let it all out for the world to see and it’s really needed. Sometimes sales depts can start throwing numbers and specs and it can get kind of blurry. To have someone like John just putting the truth out there it really helps to see who is behind the scenes and what they are about.

John’s blog of course focuses on their SAN/iQ offering but as more posts have appeared he’s started to post about other items that relate to it. Like how awesome (it really is!) iSCSI is compared to it’s older but aging competitor Fiber Channel. He brings up a lot of good points that companies need to look at as they move forward or begin looking at the differences between the two. Even with FCoE (Fiber Channel over Ethernet) FC still has a long way to go.

Note: We still haven’t bought our SAN solution yet and I wasn’t paid for this post.