Healthcare IT Blog

Have a Fall Back Plan

Published on 04/09/2013 by Jorge Navedo
Category: Healthcare IT

I have been working as an engineer in the MEDITECH space for over 11 years now, and boy, how time flies.  And if I’ve learned anything throughout the years, it’s to “have a fall back plan”.  We are constantly engaged at PPI to migrate MEDITECH data or MEDITECH servers, both physical and virtual.  Sometimes the migration is a simple as a file copy of a single server and sometimes it’s as complicated as migrating an entire ESX cluster to a new storage array on a new set of ESX hosts on a new set of SAN Switches.  No matter how complex or simple the project, every customer appreciates a good fall back plan.  Of course the plan is to never have to use the “fall back plan”, but it is better to have one and not need it, than to need one and not have one, right?

Not all fall back plans are created equal.  The plan can be as straightforward as doing a backup as close to the time you are doing your work as possible.  For instance, let’s say you are migrating a server’s data from one disk to another, and you plan on starting at 8pm and a backup of that servers data takes 2 hours.  I would run a backup at 5:30pm, assuming it would finish by 7:30pm, giving me a good point to fall back to in a worst case scenario.  The good thing with a data copy is that we are only reading data and not modifying the source, so we would go back to the source before we would go back to tape.  But what if you are making modifications to Windows (i.e. updating drivers, installing critical updates, updating firmware on HBAs)?  Everyone is familiar with the BSOD (blue screen of death), right?  Well, luckily most of us are working with virtual machines nowadays so we have the luxury of being able to take a VMware snapshot before making such upgrades.  That would be your “fall back plan”.  Many of the mid-range storage arrays today offer the same snapshot technology.  So before you do migrations or modify data, take a snapshot of your LUN.  It can save you from going back to a tape restore.

Also, don’t delete things before you are 100% sure that you will not need them again.  For instance, if you are zoning servers to a new storage array, don’t delete the old zones to the old storage array until you are positive you will not need to fall back to that array.  Keeping old zones around costs nothing and can save you a lot of work if you ever have to fall back.

I explain to customers on almost a daily basis how I plan on moving their data.  I am often asked what our fall back plan is.  I can tell that each customer is appreciative and feels more secure with my work when I explain it.  Honestly, I think the fall back plan is just as important as, if not more important than, the actual migration plan.  Fall back plans don’t come without a cost of course, there is added downtime when implementing steps like taking snapshots, and in a world where downtime is a four-letter word, adding time is frowned upon.  But in my professional opinion you can’t afford not to have a good, well thought-out “fall back plan”. 

Jorge Navedo Jorge Navedo is a Senior Systems Engineer at Park Place International.  With over eleven years’ experience working with MEDITECH hospitals, he has performed countless MAGIC and Client/Server migrations.  In addition to his hands-on experience working at customer sites, Jorge maintains certifications in every MEDITECH-certified technology.