Snowballs and dominos
It often seems hard for me to believe that the Windows Registry exists. It’s such a monolithically bad design and implementation that it simply boggles my mind. With such a creaking, doddering foundation, it’s no wonder that Windows is so unstable.
Bad decisions tend to cause snowball effects; that is, they eventually cause more and more problems and get worse and worse over time. Many of Windows’ most high-profile problems originate in some way from the bad decisions of the Registry. What are these bad decisions?
Here’s the biggest one: the Registry is series of huge, centralized databases. That means each component of it is one huge file, tucked away somewhere you’ll never find it. Databases have a lot going for them, but security and transparency aren’t two of them.
Databases tend to be prone to data corruption. Since they’re one big file, if part of that file gets corrupted, the whole thing is shot; this is why storing preferences in multiple files is a better idea. The logical solution to this data corruption problem would be to regularly back up the registry–say, every time it was changed. Wouldn’t that be nice? As a matter of fact, it does this, but it only keeps one, so you can’t roll back changes to anything beyond the previous backup.
The registry is also not very human-readable. You need a special program to even access it, and even then, its terminology is cryptic. HKCU? HKLM? Dword? Why all the hexadecimal values?
Finally, the registry’s design encourages developers to rely on it for everything. Let’s say, for example, that when an application installs itself it creates a registry key listing its location: “C\Program Files\ScumCo\ScumEdit\”. Well, I and many others hate the tendency of software to install itself in a folder named for its parent company, rather then its own name. So I opt to change its location. I grab the ScumEdit folder and move it back down a level into C\Program Files\, then delete the redundant ScumCo folder. But now we have problems! When I try to run ScumEdit, it mysteriously fails or throws up an ugly error message. Why? Because it consults the registry for its location, and the actual location differs from the listed one. Oopsie! Now I have to edit the Registry to make my change reflected in the database. Fun fun fun.
Moving things in Windows that you didn’t specifically put there yourself is dangerous for precisely this reason. With the advent of the Registry in Windows 95, playing around with your computer became a minefield. But Microsoft came up with a solution: abstract everything! Suddenly, users didn’t need to navigate the filesystem; Windows 95 also debuted that big, pretty Start button that listed everything they had when clicked on. And that just solved everything!
Problem is, when users have no contact whatsoever with the filesystem because it’s an incomprehensible mess, it becomes easier for unwanted software to hide there. Ever looked inside an install of Microsoft Office? Subfolders within subfolders, lots of files with meaningless extensions, plenty of files with unreadable 8-character names, and a whole bunch that do both. Here’s what it looks like, by the way:
By contrast, here’s what a portion of my Mac OS X Applications folder looks like:
The abstraction required by the Registry’s deathgrip on information meant that users didn’t have to look at their operating system’s underbelly. In the absence of critical attention, mold started to grow there. Developers cared less and less how elegant their software’s installation routine was; after all, nobody would ever look at what happened. Scripting became more popular now that that pesky issue of users looking at what was happening was gone. Software began to use unofficial means to accomplish things.
Unfortunately, Malware took advantage of these developments. Poorly-conceived scripting environments made it easy for malware to automatically replicate, and huge folders full of files with bizarre incomprehensible names offered great places to hide. The Registry itself was a perfect place since it was hidden and inscrutable. Users knew less and less about the structure of their computers, while the malware developers knew more and more. Compounded with the problems of bad privilege control, open network ports, and hard-to-understand networking features, replicating to nearby machines and hiding there became a cinch. So naturally, it started to happen more and more. Suddenly, there was a full-blown security epidemic, and Microsoft was caught in the middle.
There were two options:
They could completely do away with all their poorly-designed insecure swiss-cheese-like OS holes by doing in the registry and starting from scratch, closing outbound network ports, properly separating user settings and programs from system settings and programs, and simplifying networking. The problem with this approach was that not only did it require fundamentally remaking most of Windows’ core, but it would also break compatibility with 100% of all existing Windows software, including their own! To bypass this, a sandboxed compatibility layer like Apple’s Classic during the Mac OS 9 to OS X transition would have been necessary, throwing up all sorts of other problems such as emulation, inter-process communication, and developer transition kits.
This effort would have taken years and years. If this was what they had decided was the best course of action, it probably would have been easier to pull an Apple and start over from a promising foundation owned by someone else. But what was available? Anything UNIX-based would certainly be mature enough, but its openness was utterly contrary of Microsoft’s corporate identity and business models of monopoly dominance and vendor lock-in. BeOS was dead, Solaris was becoming open-source, and Mac OS X was already used by a competitor. There really wasn’t anything for them to take and re-brand.
That left the sticky option of somehow retrofitting Windows to be more secure without breaking compatibility with existing software. Thus, the Registry and most other existing attack vectors for malware had to stay. Instead, Microsoft worked on locking down these vectors; in Vista, Internet Explorer 7 has a “sandboxed” mode where it is isolated from other operating system components, and other applications are more isolated from each other as well. The kernel itself consistently randomizes the location of its data, the built-in firewall was strengthened, and weak anti-malware was built in.
In a decidedly ironic twist, Microsoft also began to sell antivirus software, turning itself into a vendor for solutions to problems its own ineptitude had created, rather than finding a way to fix those original problems.
Finally, Microsoft decided it hadn’t annoyed its users enough by adding a hellish feature called User Account Control which demands confirmation when something happens. Want to install a program? You get a UAC confirmation dialog. Decide you don’t like it and want to uninstall it? You get another one. Access the Device Manager? Adjust the font size? Change the date or time? Share files? Let a program through the firewall? Add a new device? All these actions pop up UAC prompts to waste your time and blow your blood vessels.
The really great thing about a UAC prompt is that they dim the background and lock out everything but the dialog box, meaning you are forced to acknowledge its presence and deal with it, even if you were in the middle of something else. Also, the dialog box appears in a random location each time to prevent you from being able to quickly click on “OK”. No, I’m not making any of this up.
Note that none of these tasks actually require elevated permissions; Windows just feels the need to make you confirm your action to make sure it was originated by you and not a piece of malware. But imagine this situation: there’s a piece of software you want to install that has a virus inside of it. Upon trying to install it, it pops up a delightful UAC prompt. You click OK like you always do, since you want the software. But oops! You just installed the virus too! UAC does nothing whatsoever to counter this type of hijacking.
Basically, with Vista, Microsoft did what they always do: pile on more features and workarounds instead of solving the core problems. And it shows; Vista takes up 12 gigabytes of space compared to Windows XP’s 1.6, and feels noticeably slower. Copying files is glacial, and playing an MP3 will slow down your internet access! On top of that, Vista is still insecure; even after all these bolted-on features, it remains quite vulnerable to infection by malware. The herculean effort took 6 years and cost billions, driving up the price of windows to such levels that the major computer vendors are complaining and Dell is now selling machines pre-loaded with Linux.
It’s all a monstrous domino effect: Microsoft creates a poorly-conceived, insecure feature that negatively affects usability; to cover it up, they abstract everything, making it easier for malware to hide and replicate; to deal with the resulting security disaster, Microsoft piles on fixes and features without addressing the core problem, making their operating system late, slow, bloated, and expensive; Manufacturers complain and look for other options, while consumers slowly migrate to Macs.
Isn’t the Registry great?
September 5th, 2007 at 6:46 pm
You make some good points here, but also several rather bad ones.
Firstly: I completely agree with the main part of your argument regarding the registry: the *nix method (ini files) in superior to it in many, many ways.
A couple of points, however: you are incorrect in saying there is only one backup of the registry. It is backed up (in addition to key system config files) as part of setting a system restore point, which is done on installing new programs, making changes to the system, every so often on a sechedule, or can be done manually. You can roll back to any point (e.g. screenshot at http://snurl.com/1q8hi).
Your observation that the acronyms HKCU, HKLM etc. are cryptic is a slightly strange one: of course the acronyms are cryptic, all acronyms are cryptic; but they’re not referred to (in Windows) by their acronyms. The full names (HKEY Current User; HKEY Local Machine…) are what is used in regedit, are not at all cryptic. The data types used inside the registry (DWord, hexadecimal) are indeed cryptic if you’re not a programmer and aren’t used to them, but the registry is, after all, a database for programs and Windows to store their settings between sessions, and its datatypes are as such ones that will be useful to programmers; end users aren’t supposed to edit it manually.
You note that programs register their location in the registry for various purposes (for example, the path of the uninstall program so that add/remove programs knows where it is) and that if this is changed, you can have problems; but I’m afraid that I don’t see how this would be any different if ini files were used rather than the registry: you’d still need to edit the path in, say, the add/remove programs ini file in my example when you move the program.
(Incidentally, I completely agree with you about programs putting themselves in a subfolder under their company name).
Those are minor points, though; for the most part I agree with you. Now for a major one: your next point is baffling. You complain about the Windows 95 start menu, a way of abstracting users from the underlying program files. And you give a screenshot of underneath the abstraction; the MS Office program files. OK. Fair enough.
But then, “by contrast”, you show the Mac OS Applications folder. Which you seem to be unaware is a Macintosh program launching abstraction, equivalent to Windows 95’s Start menu. It is ridiculous to show screenshots of one OSes application launching abstraction and the other OSes internal program files folder as if the two were comparable. And it’s quite clear that you’re a very experienced Mac user, so surely you must realise that the Applications folder *is* an abstraction, that that’s not ‘all there is’. If you really didn’t, then, well, you’ll know for the future: to see the internal program files behind the abstraction, the analogue to the screenshot you post of the MS office files, control-click (or Right-click) on the application and select Show Package Contents from the context menu.
I completely agree with you on the irony of MS selling antivirus software; though I suspect that if they did try to bundle the capability in with the OS they’d face an antitrust lawsuit of epic proportions from Norton, Mcaffee et al.
What next? Ah yes, UAC. There are a couple of minor problems with your comment on this, and one very, very major one.
Minor problems first: you state that “you are forced to acknowledge its presence and deal with it, even if you were in the middle of something else”. This is rubbish. If a UAC prompt is triggered by a background window (say, a program installing in the background), it will stay in the background until you alt-tab or click on the background window. Next you state that “the dialog box appears in a random location each time to prevent you from being able to quickly click on “OK””. This is also rubbish. It appears in the exact centre of the window that triggered it. Next you say “No, I’m not making any of this up”, which is amusing, since there was no part of that paragraph which you *didn’t* appear to make up.
The major problem, though, dwarfs those into insignificance, and that is the statement that “Note that none of these tasks actually require elevated permissions; Windows just feels the need to make you confirm your action to make sure it was originated by you and not a piece of malware”.
Oh, dear.
UAC is a privilege elevation dialogue. That is what it is, and that alone; its purpose in life, if you will, is privilege elevation.
Just like gksudo or kdesu in Linux. Or Authenticate in… Ummm, Mac OS X.
That’s why your statement is so puzzling: ignorance in someone who’s never come across dialogue-based privilege elevation before is understandable, but the OS you normally use does exactly the same thing! In both, when running as a standard user, you are prompted to enter an administrator user and password to perform administrator tasks. Compare the Authenticate dialogue with the UAC dialogue. The only major non-aesthetic difference between them is that Vista gives you the names of all the administrators, so you just need to remember the password, wheras Mac OS doesn’t. Otherwise, the dialogues are substatially identical.
And the only major difference in functionality between Authenticate and UAC is that, with UAC, there’s an additional mode whereby, if you log on as an administrator, you can choose to run both standard and administrator tokens simultaneously, so programs normally run under the standard user token for security, but if you need to you can elevate programs to the admin token without needing to type your password (when running in this mode the UAC dialogue misses out the username/password entry). Mac OS has no such mode. Other than that, they’re substantially identical in functionality.
(Incidentally, the “sandbox” feature of IE7 is just a byproduct of UAC: IE runs with very low privileges, even lower than standard user privileges; you need to elevate to, for example, interact with other programs on the conputer).
Your situation of a virus embedded in software the user is installing is correct, but again, exactly the same “security hole” is present in all modern operating systems: if the user wants to elevate and knows the admin password, the OS can’t exactly stop them. How could it be any different?
Re the price: Windows 95 Full RRP’d for $209. Vista Home *Premium* RRPs $240. The difference can easily be accounted for by 12 years of inflation.
You’ve made some excellent posts on this blog, as I’ve said in the past; but this, I’m afraid, was not one of them. Fatally badly researched, and the “No, I’m not making any of this up” between two paragraphs both of which were substantially rubbish was especially galling. Dissapointing.
September 17th, 2007 at 6:37 pm
Simon,
Whoops, your comments wound up in my spam queue! Sorry for the obscene delay.
——The Registry——
I can’t believe I overlooked the fact that the Registry is backed up at system restore points! Duh! How would it be able to revert to old settings without having kept a copy of them? Oy.
Yeah, ideally users shouldn’t have to edit the Registry, but I’ve often seen them have to or been told to by a techie who was looking for a quick fix. Then again, this isn’t really so different from telling an OS X user to run a terminal command in its obfuscation level, so there you go.
I don’t believe that the install location of programs should ever be stored anywhere. With no method of keeping in sync the actual location and the Registry/.ini/preference’s value for the install location, problems will always crop up. What if I want to run it from the Desktop while I’m evaluating it? If it breaks because it was installed to the applications folder and can’t find its stuff as a result, then that system is broken.
Uninstalling things should be as simple as dragging to the trash, ideally. Even better would be if when you dragged an app to the trash it it asked you whether you wanted its preferences file and app support folder nuked too. But this wouldn’t have to be through paths, it could be done by name: whenever a .app “folder” is trashed, the system could look in the preferences folder to see if there’s a matching file. If there was, it would parse that file and find any other support folders and files. If the app was installed through an actual package installer, the system would additionally search through its receipt and offer to get rid of its support files. All this could be accomplished without paths to the application bundle at all.
——Abstraction (Start Menu vs. Applications folder)——
From a security standpoint, you’re close to the mark about Mac OS X’s abstraction, but not quite there, I think. Theoretically, a piece of malware could hide inside an application bundle, but this presents problems: with each software update, the bundled apps are updated by throwing away the old apps and replacing them with new ones, which would remove anything bad hiding inside of one. For any piece of third-party software, malware can’t be sure of it being installed at all, and apps that use Sparkle also trash their old selves and replace them with new ones. This leaves vulnerable third-party apps that don’t use sparkle or a comparable updating mechanism. How successful will a piece of malware be that can only target Mac OS X systems with Audio Hijack installed be?
Correct me if I’m wrong, but in Windows, updating an application involves throwing away old files and replacing them with new ones, rather than throwing away the whole C:\Program Files\myApp folder and replacing it with a new one, right? Say \myApp has the following files:
myapp.exe
myapp.lib
dependencies.dll
otherStuff.dll
bigHonkinVirus.exe
If this application updates itself by replacing the files it already knows about, it will ignore the virus hidden inside unless it is smart enough to check for the existence of unknown files, but this could break other things or erase user data that happened to be in there–which is possible since it’s just an ordinary folder (compared to bundles, which do not show up as folders without explicitly ordering them to show their contents). Unless the application completely throws away its parent folder when it updates, it can’t be sure of ridding itself of malware. Or at least this is the way it seems to me. Feel free to correct me if I’m wrong.
But primarily, the reason I prefer Mac OS X’s abstraction (applications folder) is that it’s closer to the source. If you delete something from your start menu, it’s still installed. If you torch an application’s folder in C:\Program Files, its start menu folder is still there, as are its Registry values, meaning it still shows up as installed from Add/Remove Programs and Programs & Features. Even if you uninstall, a lot of rogue programs won’t remove themselves from the start menu or clean up their shortcuts on the desktop and quick launch bar, sadly (although this isn’t really Windows’ fault).
Mac OS X’s abstraction is right on top of the implementation. If you delete the abstraction (the pretty icon) the implementation’s gone too. In order to break an application by tinkering with its frameworks and such, you have to pass through the abstraction—by right-clicking on it and choosing “show package contents”.
——UAC——
You’re right about UAC being privilege elevation when you’re not running an admin user. That’s absolutely correct, and something that I hadn’t even really considered when I wrote the post because I’ve never actually come across a non-admin Vista box. Most of the non-admin users I come in to contact with are in businesses and universities and such, and few have moved to Vista yet. My workplace is among those who have thus far stuck with XP (and Windows 2000, don’t ask).
What I was talking about, though, was when you ARE an admin user, because then you don’t need to put in a password at all. It just asks for confirmation, essentially badgering you for confirmation to do something that you *already* have sufficient privileges to do. I guess it’s cool that it runs as a standard user unless you hit something that needs admin rights, but really, is the dialog necessary? Am I going to decide not to change my desktop icons because I know it requires the admin rights I already have?
Nothing about a password-less UAC prompt will dissuade me from doing what I wanted to do. Keeping underprivileged users from changing settings they don’t have the right to change, fine. It’s teriffic that Vista has this ability. But badgering those who *already have* those rights every time they try to exercise them? It’s totally understandable for UAC password prompts to pop up for non-admin users, but anytime one pops up without a password, that’s basically asking you, “hey, I know technically you’re already cleared to do this, but are you sure it’s the right thing to do? Maybe you should take a closer look.” That’s infuriating to me.
Also, I had no idea that UAC prompts popped up in the center of their parent window! Since the screen dimmed, I never paid attention to anything but the prompt, and you have to admit, that seems to have been the design goal. I can see it now that you mentioned it, but to your average end user it’s sure going to look randomly placed, and regardless of reason, it still has the effect of inhibiting muscle memory, since its buttons are in different locations for each non-maximized window that spawns one.
I realized a little after I wrote the post that UAC prompts would indeed not interrupt something else. I admit I was in the wrong in not editing my post to reflect this, and I apologize for the error.
——UAC re: Virus in installer——
You’re right that a virus hiding inside a an installer or executable is something that all platforms are vulnerable to. But UAC gives you the illusion of security. If you think you’re being protected and you don’t really know the precise way in which the system can fail, you might let your guard down and rely on UAC as a security mechanism.
I’ve known people who did this and got burned as a result. Since previous versions of Windows did nothing overt about security, now that Vista does, people assume they’re more protected and try to rely on it to protect them. If it can’t, then it’s practically worse than nothing at all since it can drive them to unsafe activities without knowing that they’re not secure.
——Price——
Hmm, that’s interesting. I had no idea that previous versions of Windows were so expensive! I’ll admit that I actually did no research. Oops. Perhaps that was a poor idea. Perhaps I will abandon that habit in the future.