Wow, it’s been over 2 months since my last post. I guess you can say that things got a little busy with packing up and moving a few hundred miles. I think I’m finally settled at least enough that things are mostly normal.

The last few months have been nothing short of awesome! Have there been bumps along the way? Sure. There always are, but the awesomeness has been overwhelming. I want to thank each and every person who has prayed for me, encouraged me, helped physically move stuff and listened to me gripe and complain when things don’t go well (yes, I do my share of that even though I don’t like to admit it).

Here’s a quick recap of the last few months:

• I made a visit to Seacoast back in late August, then again in October for CITRT. After lots and lots of prayer and changing my mind a few times, I felt the Lord leading me to take a leap of faith and move to SC to join the team at Seacoast. Yes, I’ll admit - it was scary.
• I returned to the Charleston area to look for houses. Amazingly, I found the perfect house the first day!

• I had to say goodbye to all my awesome friends at JFBC. That was hard. To make it harder, they took me to lunch, got me a cool card, and had a “Going Away Party” for me.

• I packed up to move. This was a LOT of work, but thanks to some great help, got it mostly done in one day.

• Thanks to my Seacoast friends, we got everything unloaded in under 2 hours! 7 weeks later, I’m still trying to figure out what to do with all of it.

• Now I’m here, mostly settled in, and get to serve at an absolutely incredible church! I’m excited about what the coming months and years will bring!

It was way back in 1998, almost ten years ago to this day, but I remember it almost as if it were yesterday. Sitting in a gym at First Baptist Church of Jonesboro without about 500 other teenagers I, for the first time in my life, heard the gospel of Jesus Christ shared with me a way that was crystal clear. I prayed right then and there to ask him to come into my heart and forgive me of my sins. Let’s just say the ten years that followed have been nothing short of AMAZING!

The following week I made my profession of faith public in baptism. I immediately dove in head first and became very active in the church’s media ministry where I served for the next six years, primarily in the areas of audio and lighting and, later, video. Over the next few years was also able to develop and awesome relationship with the media team at Johnson Ferry Baptist Church where I had the opportunity to come in for verious special events as Lighting Director.

In the summer of 2000, just after graduating high school, I began a part time job with an Atlanta area land developer as a web designer. Ths roll quickly transitioned into an IT Director roll where I had the opportunity to design, build and maintain the computer network linking all of the company’s offices and country clubs.

Fast-forward to February of 2006: due to economic factors as well as some changes in the structure of the company, my position was eliminated and outsourced. Around the same time, discussions began with Johnson Ferry about a roll they were looking to fill on their media team. After a few weeks of filling the roll on a temporary basis, lots and lots of prayer and one wrong decision that clearly wasn’t God’s will, God made it clear to me that he had called me to serve him full time.

In May of 2006, I joined the media team at Johnson Ferry. In December of that same year, I had the opportunity to transition to the IT department where I am serving today as Network Administrator. It certainly hasn’t all been smooth sailing - there have been many bumps along the way. But, God has confirmed over and over again that he, without a doubt, has called me to dedicate my life and career to serving him.

This brings us to today. God works in amazing ways, and the bits and pieces don’t always seem to fit together along the way. In the end, though, it is always made clear that his plans are bigger than ours. Now is no exception.

So what’s happening you say?

I am absolutely pumped to announce that in a few weeks I will be making a move to the Charleston, SC area to join the team at Seacoast Church. At Seacoast, I will be filling the roll of “IT and Production Specialist” where I will be involved in planning, design, implementation and maintenance in the both the IT and audio-visual areas.

I’m experiencing every emotion all at once as I type this. It’s incredibly exciting, still a little scary and at the same time very sad to leave all the awesome people I work with at Johnson Ferry. I want to thank everyone who has been praying for me over the last few weeks. It’s going to be awesome!

Early today, as part of our CITRT pre-activities, I met up with several church IT guys for dinner and a tour of our facilities at JFBC. Just before I left to to meet them at Moe’s, I saw a twitter message that they were “Geeking Out”. There was much discussion about the term “Geeking Out”, and this is what I found when I arrived at Moe’s:

I actually felt a bit left out since I left my computer back at the office. Fortunately, I did have my iPhone with me.

We had an awesome time together tonight checking out JFBC’s infrastructure and talking about IT and ministry. Looking forward to heading out in a few hours to Charleston to meet up with 60+ IT guys from churches around the country.

For a couple of months now, I’ve been hearing about the upcoming 4.0 firmware and Auto-Snapshot Manager, VMware edition for the Equallogic PS series SAN. This new snapshot provider would allow us to coordinate snapshots between VitualCenter and the SAN, and, according to Dell/Equallogic, allow easy restoration of a single Virtual Machine from the SAN-based snapshots.

I had the opportunity on Thursday night to watch a pre-recorded demo as well as to attend a live webinar on Friday morning. I must say that after these demos and seeing exactly what this product does, I am stuck somewhere between excitement and disapointment. It’s a really cool concept, but I believe it still needs a lot of polishing, especially on the recovery side of things.

There are lots of awesome features in the new snapshot provider. We will have the ability to automatically, in a single click or scheduled task, trigger an ESX snapshot, including memory dump, then snapshot the SAN volume, followed by removing the ESX snapshot. This eliminates the journaling effect and associated performance hit and disk requirements of the ESX snapshots. This is all handled through a nice web interface, and the VirtualCenter folder tree is carried over, allowing snapshot schedules to be applied to groups of Virtual Machines.

There are, however, some catches. Only the selected VM’s are triggered for ESX snapshots, but the entire SAN volume, which may contain many other VM’s is snapshotted. This makes the ability to group VM’s using VirtualCenter folders less than useful. Let’s say I have four VM’s split between four volumes and want one machine on each volume to be snapshotted everu 12 hours. Then, I want one machine per volume to be snapshotted every 24 hours. In this scenario, I will actually end up with, at the SAN level, two snapshots per day of both entire volumes and all the VM’s since the entire volume is snapshotted. So, in my opinion, snapshotting VM’s by any grouping other than an entire SAN volume isn’t going to be practical without a lot of wasted disk space.

On the recovery side, I think there is a lot of room for improvement. It is very easy to revert an entire volume and all the VM’s it contains. Beyond that, restoring a single VM, for example, becomes a somewhat lengthy process. Basically, it involved going back to the Equallogic Group Manager, setting the snapshot online, going to ESX and mounting the snapshot as a new volume, deleting the damaged VM, copying it manually from the snapshot to the production volume, adding it to inventory, booting it up, and then unmounting the snapshot. Alternately, the VM can be booted from the snapshot volume, then migrated back to the production volume using Storage VMotion. Storage VMotion, however, requires accessing the ESX command line.

It is my hope that, in a future release, Dell will automate some of the recovery process using the VMware API’s. Currently, there are lots of improvements in creating the snapshot, but no real change in the process of recovering a VM.

I am looking forward to getting the Auto-Snapshot Manager, VMware edition installed in our environment and actually seeing it in action in a production environment. Expect another post in the future with more details once I actually get this up and running.

I’m excited about this month’s House Pay-Off Spectacular.  The green and pink are actually touching now!  I’m praying and looking forward to being able to go all green very, very soon!  Hopefully in the next month.  At this point, I’m just waiting on some funds in CD’s to become available.

This is awesome!

Total Squares: 562
Paid-For Squares: 323 336
Savings to be committed to mortgage: 232 219
Squares Remaining: 7 0

If you are not aware yet, a major bug has been revealed in ESX 3.5 and ESXi 3.5 Update 2.  Apparently, the beta was coded to expire on August 12, 2008 and this code failed to be removed from the actual release.  Details are available in the VMware Knowledge Base and This Topic on their forums.  You might also checkout This Post on Matthew Marlow’s blog for more information.

On the morning of the 12th, I was greeted with several errors like this one in the logs for our ESX cluster:

VMware finally released the patch really late Tuesday night, which, unfortunately kept me up most of the night getting our cluster patched.  It involved setting back the clock on all of the hosts so Vmotion would work, manually migrating VM’s off of a host, going into maintenance mode, applying the patch via the command line, then migrating the VM’s back.

VMware is one of my favorite companies and it know for delivering rock-sold, enterprise-class products, so it really disappoints me that they would let something like this slip through the cracks.  Imagine how many hundreds of thousands (Maybe millions?) of VM’s this affected.  They do seem to be committed to fixing their mistake and making things right.  You can check out the Letter From Their CEO for more info.

I had a conversation yesterday with Bakbone about their NetVault product.  As we’ve moved heavily into virtualization (90% of our infrastructure is virtualized at this point), backup and DR has become a growing challenge.  Ideally, we need to be able to back up entire virtual machines directly from the SAN, with the ability to restore and entire VM, or individual files within a VM.  In addition, properly protecting Active Directory, SQL Server, and Exchange are high priorities.  The ability to do message level restore in Exchange is also somewhat important.

Our aging Backup Exec installation seems to become more and more cumbersome and problematic, and seems to have the common problem of one product trying to do way too much and not doing any one thing exceptionally well. I think it’s time to move into a more enterprise-class product - something more closely tuned to our needs.  NetVault initially seems like a potentially good fit.  If anyone has any experiences with NetVault or has any other recommendations, I’d love to hear from you.

Last December, we implemented 7TB of Equallogic storage as the backbone of our VMware Virtual Infrastructure implementation, as well as to serve as primary storage for our file, Exchange, and SQL servers.  Little did we know that 6 months later we’d be near capacity and shopping for more storage.

Thanks to James at EIS, we now have another 16TB of raw storage online.  Combined with our existing array and considering RAID overhead, we now have just under 15TB of usable iSCSI storage.  I’m excited to have this done!

I absolutely love our Equallogic SAN!  In less than 30 minutes, the new storage array was configured and added into the cluster.  The volumes were automatically distributed and network traffic load balanced among the arrays.  The only complaint I have is that their rack rail system could use some improvement.  Getting the array installed in the rack is the most time consuming part of the entire implementation.

Check it out:

Array was sitting in my office when I arrived on Friday:

Unpacked and ready to be installed.  It has 16 drives with a capacity of 1TB each:

Racked next to it’s little brother.  This is a total of 30 disks and 23TB of raw storage capacity:

Total storage capacity of 14.82TB:

The performance and raw throughput of the Equallogic gear amazes me.  Orion was reporting 534mbps of traffic between the arrays during setup:

Hopefully we will have plenty of space for a while now.  Although, as we move toward implementing Final Cut server and centralized storage for digital video, that may change.

Matt McCook from Xirrus has offered to do a presentation on high-density WiFi and how their products can meet these needs starting at noon on July 17 at JFBC.  We have been using Xirrus products at JFBC for just over a year now and have been extremely happy with the performance and reliability of the Xirrus solution.  You can read about our initial deployment on the Xirrus Site.  I’m excited to be able to have Matt come and share their solution with other Atlanta area Church IT guys.  If you have a need for a high-density WiFi solution and would like to join us, please drop me a note at derek.schwab@jfbc.org or post a comment.

Wow!  It’s time for the July update on my House Pay-Off Spectacular and am PUMPED about it.  It still feels kind of silly to get so excited about coloring in squares in Photoshop, but I’m thrilled with what God has givin me.  He has truly blessed me with way more than I deserve.

It looks like there will actually end up being a couple of unexpected months of overlap between the green and the pink due to the fact some of the pink $$$ is in CD’s and not immediately accessible.  It’s still awesome though!

Total Squares: 562
Paid-For Squares: 308 323
Savings to be committed to mortgage: 232
Squares Remaining: 22 7