I recently completed a small project for a Boston area firm which involved installing Microsoft Windows updates on all of my client’s domain controllers. They had not been installed in 3-5 years on any of these important servers! I don’t think it was truly neglect, I think it was just more of “if it isn’t broke don’t fix it” kind of approach. The reason they finally decided to go ahead with the updates was because of WMI performance issues and they thought that having me install the long list of updates (and hotfixes) would solve their problems.
Several days after the patching, the problems did subside, but unless they start getting more proactive about their updating (by developing a scheduled & predictable plan) they will likely find themselves in a similar position down the road. Additionally, if they want to introduce new domain controllers running a later server OS, they would be best served by running the latest and greatest updates and hotfixes on the existing systems. For Microsoft Exchange, this is a requirement. So why do some organizations not move forward with Windows updates?
One possible reason is that in the past they could be highly problematic. An update might have a code flaw, or even an intentional security hardening feature that causes issues with an existing configuration. (This happened in the past with a Windows NT 4.0 service pack which “broke” Lotus Notes access and more recently this June, when KB3163622 was released. KB3163622 can cause an issue with GPO permissions. See here for more details.) On the other hand, many IT pros have wasted a lot of time and money by opening a Professional Support ticket with Microsoft only to be directed to install a readily available update which would have fixed their problem.
So what is the best approach? As with many questions in the technology world, the answer is “it depends”. Ideally, every organization would have a lab that paralleled their production environment that contained each and every setting, application and configuration found in their real environment. For many, this is not a realistic expectation. A more reasonable approach would be to have a smaller subset of the production environment represented in a lab – 1 DC, 1 Exchange box 1 SQL system, 1 App server, 1 for file and print, etc. Yet even this may not be realistic for some. In that case a good approach might be to select certain development servers for “early bird” patching and then as the updates are deemed OK, they can be rolled out to the more important production servers.
A third approach is just to monitor blogs, TechNet forums, IT news websites, etc. and see if any issues surface after Microsoft patches have been released.
This is not the best way, as your environment may contain services, applications, and other code that is unique to your infrastructure. Several organizations like this choose to use a fixed time window (say a month) before installing Microsoft Windows updates and monitor the previously mentioned news resources to make sure there is no “bad press” about an update after it has been out for a month (or whatever time window they decide on), and then they go ahead and install. Updates can be uninstalled very easily of course, but they will likely require a reboot, and the uninstall may require an additional change control in some organizations.
Another important consideration is whether to lets machines get their updates directly from the Internet or use the Windows Server Update Services. WSUS can act as a central repository for all Windows updates, and greatly reduce bandwidth, but also consumes a large amount of disk space and may require a dedicated server (or servers) and more administration.
Many administrators (and consumers alike) do not like the fact that Windows update can reboot their systems automatically after the updates have been installed. Microsoft offers a registry fix to prevent this from occurring (the NoAutoRebootWithLoggedOnUsers key), but it can only halt the reboot if a user is logged on (and it is ignored by Windows 10), which is great for preventing unwanted reboots for client systems, but might not help with servers, as in many cases no one will be logged on even while the system is doing its job.
The pestering Microsoft Windows update reboot dialog that we all have seen.
To avoid the unwanted reboot or servers, the best approach may be to use the download updates, but let me choose whether to install them option, allowing the updates to be installed during a maintenance window, when rebooting can happen manually, at an acceptable & known time. Since the admin can control the reboot, then there are no surprises. Also, this method allows each individual update to be reviewed and installed or omitted.
As shown below, individual updates can be selected or deselected as desired.
A false belief is that if a network is secure at the perimeter, and a system has an antivirus product installed then Windows updates do not provide any security benefits. This is bad belief as an internal system such a contractor’s laptop or flash drive) could introduce a new security problem (zero-day exploit, malware, etc.) and a patched system could be far less vulnerable.
Increasingly, many organizations are seeing the value in having a trusted partner handle their software maintenance needs with regards to updating and patching. By contracting with a Managed Service Provider (MSP), the worries about handling the updates are taken away from IT staff, who are usually busy with existing projects, support and helping the business units with new initiatives. There are additional potential benefits as the MSP would likely have a closer relationship with the software vendor (Microsoft in this case) and would have access to more detail information about the updates and be aware of possible problems that the new code could cause.
In addition, since the MSP likely has several customers, they would have insight as to how the updates impact a variety of production infrastructures, with a diverse array of applications and use cases. Lastly, since the MSP likely schedules the updates in accordance with the client’s wishes and time table, any new problems could be easily traced back to the installation of the updates. Imagine if an admin installs the updates and then they go on vacation soon after they will not be available if something goes wrong.
In addition, the admin may miss a change control window, or even install updates without going through the change management process, and leave the business wondering what caused any sudden problems with their systems or applications, unaware that that admin “went rogue” and decided to install updates on the fly, without informing management or using a schedule.
While WSUS has a console that can show basic information such as which systems have received updates and which have not, SCCM can provide additional benefits. These include system targeting and support for 3rd party software updates. Also, by using a specific query, systems can be targeted by common searches or by meeting specified criteria beyond what WSUS can do, such as filtering by a specific string in the computer name.
This can be useful for staging update rollouts and can help prevent network bottlenecks. In addition, it can update a system image while it is offline, which prevents the need to rebuild the image. This is great for virtual machines. Also, scheduling can be much more advanced as it can coincide with the maintenance window feature that SCCM offers. Lastly, SCCM offers more advanced reporting then is provided by WSUS. Below is a screenshot of the software update console in SCCM software library.
In summary, if we lived in a perfect world, software companies would release code that was 100% bug free and secure right from RTM. However, in the real world this is not going to happen. Microsoft Windows updates ensure that you have the latest code, and with some planning and thought can be very manageable.
To speak with a Microsoft certified expert, complete the form below.