Zotob, zotob, zotob... Once again, a malicious worm is running amok amongst enterprises everywhere - leaving endlessly rebooting systems in its wake. As with previous worms, a fully patched system is completely "immune." However, before we all start putting up pikes topped with the skulls of our IT administration personnel, consider the fact that this time, administrators had less than 3 business days to apply the patch before the storm started.
We've known for some time that the window between patch release and worm release is narrowing; that window has now become so small that developing a manual patching process that fits the time window is improbable. In this case, the -only- systems that are likely to be patched are the ones that applied the patch without any formal testing (such as those using the "auto-update" capability of MSFT operating systems.) In general, most enterprises try to ensure that any changes to production systems (servers, corporate desktops, etc.) are managed and tested before release; this attempts to keep downtime to a minimum by making sure that critical applications work just as well after the patch as before it. In this situation, enterprises that did that, that tried to ensure application uptime by moving patches through a formal process, were left with their pants down and are currently enjoying the "festival of pain" that is zotob.
I posit that there is a threshold where the amount of downtime associated with testing patches will exceed the amount of downtime associated with applying them without testing. Some might say this is heretical, but think about it: when was the last time that anything more than a minor inconvenience was caused by applying an OS patch to a production system? Not a huge number, right? Oh sure, for some legacy systems the incidence of patch-conflict is likely to be higher. However, maybe it's time to consider a phased approach - maybe if non-critical systems (e.g. corporate desktops) use an automated patching methodology like the MS auto-update functionality and only do the full testing for systems that are semi-stable or otherwise "touchy." Just something to chew on.
*** Updated: Sorry about the spelling in the previous iteration of this; should be fixed now. :-) ***
Posted by Ed at August 17, 2005 10:08 AM