While in this case the jump into the middle of an instruction was not intentional, several decades ago many Microsoft programs for MS-DOS, like their BASIC interpreter, contained frequent intentional jumps into the middle of instructions.
This technique had already been used in many Microsoft programs for CP/M, i.e. for Intel 8080 or Zilog Z80 CPUs.
There were two reasons for the use of such jumps in the middle of instructions. One was to obfuscate the program for disassemblers, which were desynchronized with the instruction stream by such jumps. The other reason is that each such jump usually saved one byte of instruction code in comparison with conventional conditional program structures.
When the program size was still measured in kilobytes, saving even only one hundred bytes by this method was important.
The instruction into the middle of which a jump was performed was usually an instruction that loaded a 16-bit register with an immediate value that happened to be the encoding for some other instruction.
The value loaded into the register was not used. Therefore, when the instruction was entered through its start address, no operation was done, but when it was entered through a jump, some extra instruction was executed.
A typical use case would be at the beginning of a loop, which was entered after initializing a control variable, while whenever the loop was repeated later an extra instruction was executed (corresponding to the immediate value of the register load), to update the control variable. This saved one byte upon the standard loop implementation that uses an unconditional jump for entering the body of the loop in its middle, after the loop initialization.
Biology sometimes "jumps into the middle of instructions" when decoding DNA/RNA. The "instruction" length is always 3 nucleotides and the sequence usually has only one valid "entry point".
There are exceptions though. Some sequences do encode biologically useful information with sequences which not only overlap but do so with the starting points differing by a number not divisible by 3: https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading...
Rather than "jumping into the middle of an instruction" (which to me implies some crazy trickery where both the original instruction and the middle of it do something useful, like the sibling comment about biology), a more straightforward way of looking at it is that the "jumped-into" instruction is actually a single-byte jump which skips over the one or two following bytes - and those shouldn't be considered to be part of the instruction at all.
The opcode will of course perform whatever function it was normally intended for, but it will have been chosen by the programmer to be something that doesn't affect the state of the program in a meaningful way. So it really is just a shorter form of jump, not something beyond the understanding of mere humans - you could always replace it with a regular jump and get the same semantics.
CPUs at the time didn't have branch prediction. The 8086 did prefetch code, but always linearly, and even an unconditional jump would flush that prefetch queue. So having an instruction like "CMP AX,xxxx" (where xxxx=any 16 bit immediate value) would be both shorter and faster than the "proper" jump instruction, while affecting only the flag register.
And on the 8080, there was no short (8 bit relative) jump instruction, so this kind of trick even saved two bytes instead of just one.
> There’s at least one such utility out there that properly enough validates if their patches would work, or blocklist specific Windows builds known to break from it completely. I wish the other very-obvious-from-your-article software would do so too.
and I can't help but agree.
If you're going to reach into the guts of another process and start flipping bits, it's only Correct to take responsibility for doing so carefully enough you don't blow the entire thing up.
(I have written code that replaced/wrapped other people's function pointers for various reasons (occasionally even -good- ones) and considered any problem caused by an upgrade of the code hosting my cuckoo chick to be my bug - and while it was occasionally bloody annoying, I did this to myself ;)
Kind of sad. Even from this blog, he admits that the Microsoft of old would test software and see if it worked; the Microsoft of now obviously knows of the existence of these shell enhancements, but clearly doesn’t test patches against them at all.
A relevant quote about the lenghts they went to to assure stuff not getting broken:
"Jon Ross, who wrote the original version of SimCity for Windows 3.x, told me that he accidentally left a bug in SimCity where he read memory that he had just freed. Yep. It worked fine on Windows 3.x, because the memory never went anywhere. Here’s the amazing part: On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the memory allocator in a special mode that doesn’t free memory right away. That’s the kind of obsession with backward compatibility that made people willing to upgrade to Windows 95."
That's a lot of money and time to spend on supporting someone else's bug-riddled software. You can't test literally all software that exists before you release a security patch. Just imagine testing every single solitary Windows application that exists, or has every existed, just to see if one of them crashes due to intentionally doing the wrong thing. What are they supposed to do, fix the 3rd party software? Delay fixing the security hole?
Testing is the responsibility of the 3rd party (in addition to using supported methods). If the 3rd party was part of a preview release program, they could test changes before they are officially released.
> the Microsoft of now obviously knows of the existence of these shell enhancements, but clearly doesn’t test patches against them at all
Microsoft taking into account this method of modding explorer in its testing would be like asking Apple's design team to take into account the one in a million iPhone user who sticks their phone up their butt. [0] I don't know what it says about Windows or its users that there must be more than one in a million people running this stuff, but still.
I know it must be frustrating from Microsoft's standpoint, but those are needed because they keep changing things. Start All Back is one very popular explorer.exe patch that makes the Windows 11 taskbar function a lot like Windows 10. For one thing, it allows you to move the taskbar again.
I've also made an explorer.exe in-memory patch to disable flashing taskbar buttons in Windows 10, it injected itself to WndProc and set the specific message to zero. In Windows 11 it's now a setting, so I don't need that anymore.
Now I'm investigating an explorer.exe patch for Windows 11 to speed up wallpaper animations. Which cannot be configured at the moment, there is only one setting to turn off all animations but no setting to speed up the wallpaper fade in animation.
Oh geez these are annoying! I've spent hours trying to find a setting to disable the flashing taskbar on the Teams client on my corporate issued Win10 laptop. The best I've found is to keep the Teams client minimized in tray just to avoid this behavior.
It boggles my mind there's no way to disable this feature that is so distracting it almost makes working impossible whenever I forget to kill the Teams client window. And really goes to show why people need to resort to these hacks.
Wallpapers can be animated? This is like learning my car can count butterflies. I wonder why such a feature exists but also now feel obligated to hook VLC into the wallpaper function so i can play movies underneath all my other work. Can wallpapers have sound?
> I know it must be frustrating from Microsoft's standpoint
no sympathy. Chen seems a little irritated (as he should be) at the situation, but he's still missing (or purposely not stating) the real point. the correct response is alluded to by a couple folks in the comments section. i'll be more explicit:
- if microsoft was paying attention, it would simply add the functionality these 3rd party hacky extensions were adding, obviating the need for that extension at all. Like adding a setting for the flashing taskbar buttons you mentioned.
- failing that, simply allow some of those "internal" functions as a public extension points
the fact that a market for such egregious hacks implies significant demand for something.
gad, explorer and windows UI is such a dumpster fire. you don't even need to do any kind of fancy market research, just look at what the most popular extensions are doing to patch over deficiencies and them fix them!
So these patchers are monkey patching a system executable needed to have a bootable/usable system?
Why is this allowed? This sounds like a small hell.
I left Windows the moment we shifted from "developers, developers, developers, developers" to "advertisers, advertisers, advertisers, advertisers" and never looked back.
I wonder if there is a special name for this patching pattern, where not only does a piece of code depend on undocumented behavior, it's depending on the exact binary bytes.
Is it just me? His posts always begin with an interesting technical anecdote and then veer off into boring propaganda with sullies the reputability. It's an intellectual let-down.
Frankly, it is really annoying when developers working at one of the large closed-source platform oligopolists blame their users for trying to work around the deficiencies (or even outright hostile "business model" optimizations) in their software by whatever scant means are left to them: maybe if Microsoft didn't keep removing good functionality and stopped forcing bad "features" on everyone, fewer people would see a need to install and use these extensions that are being scapegoated here for the lack of satisfaction in Windows updates.
Just look through the comments on this article to see people pointing out how Microsoft keeps removing options for how people can do this in less risky ways, or even ideas for how Microsoft can improve the safety of these extensions (which aren't all exactly right but I will note that if Microsoft embraced these more and provided an official loader--not one that limited the access but merely coordinated the installation--this is easily solved with the same kind of "safe mode" I designed for shell extensions on jailbroken iOS devices).
To quote one of the comments:
> Not just this article, but far too often when articles such as this are written, about widely used 3rd party tweaks, and the commenting strongly supports the tweak in question, but MS charges forward that “all these enthusiasts are wrong, we will continue our way” is telling of how far out of touch the Win dev team is away from its user base.
A reply from the article author sort of communicates why they don’t work with the patcher:
> They don’t use the shell extension mechanism to get into the process. They sneak in by nefarious means. Patching is not supported. There is no “correct” way of doing it. Just different levels of bad.
I assume the “nefarious” ways are similar to rootkits in that they exploit vulnerabilities to gain system access they’re not intended to have.
It’s also only really the reason they don’t care that the patcher breaks. The reason I assume they don’t add or retain these features is maintenance and support cost.
In the open source world, I think the equivalent here would be to export an internal function in the desktop environment so you can call it with a third party extension. Open source might help you choose the function, but that's not a ton of help if the project owner rejects your pull request because it doesn't mesh with their worldview.
The “correct” way to do this would be to check that the surroundings of the code you’re injecting into are as expected, similar to how applying a diff patch file contains lines before/after to synchronize with the text being patched. When confronted with an incompatible change, the extension would just disable itself and not proceed with the patching. I’ve heard of MS Office extensions that function like that and work robustly.
Well yeah, I mean no one forces you to use Explorer for file management under Windows. I'm an old-time Norton Commander user, and when Windows came around I switched to Total Commander. There are open-source alternatives too, even cross-platform ones, like this one: https://doublecmd.sourceforge.io/.
That being said, no one forces you to use Windows either - except maybe your employer or the software you are using, but this is getting less and less of a problem fortunately (web apps, Proton etc.).
I disagree strongly. If you start mucking around in the memory of the system, and shit breaks, it's on you and not Microsoft. The amount of effort they put into compatibility is Herculean, and honestly is way above and beyond what is reasonable to expect. It's not their fault that they can't handle every single edge case of programmers going rogue and stomping all over the rest of the system.
You must do BJJ, to be able to twist yourself into such knots to blame Microsoft for a 3rd party vendor's unsupported buggy hacks. No mention of how the 3rd party vendor lies by omission to the users by not telling them that their software is very literally using unsupported, buggy means to accomplish its goals. Leaving the user to blame Microsoft for the crash, when if the user knew the truth, they may feel differently, and maybe not even use the 3rd party software. But it feels better to blame the big bad corporation, despite the fact that they have not lied, and are just trying to fix security issues.
There is an interview with Raymond Chen where he is trying to defend the
reason why they crippled the taskbar with Windows 11 [1].
Watching that I was just thinking NO NO NO - it worked fine for the last
25 years and there can be absolutely no reason why they had to destroy
the taskbar now.
He deserves every headache he gets with these support requests.
Or to quote this comment of the linked article:
> There wouldn’t be as much of a need if the windows shell team was
smarter [...] In fact I actively encourage, support, and celebrate
efforts like this. Make their life hell and delay them. If they spend
more time on investigating issues like this, they’ll have less time to
f*k something else up.
Unfortunately it's the designers that are screwing up the UI/UX of Windows, not the programmers. If you give more time to the designers then you're just going to get more of a clone of OSX rather than something useful.
Watching the clip, the improvements he's talking about are great and very welcomed. Is this a case of any change being considered bad because you're used to how things are?
I wouldn't say it worked "fine" for 25 years. It's still there all the time taking screen space. You can hide it but it will re-appear if you move your mouse to the now cleared screen real estate.
The reason I need taskbar hacks is that I want one feature they never added: hide the taskbar permanently until I explicitly ask to see it (for example by pressing a Windows Key). For me it doesn't matter much if the taskbar is horizontal, vertical or goes through the screen diagonally. Just let me hide it and keep it hidden. I am ok with full screen taskbar as long as it only appears when I ask it to. I see not adding this obvious feature as manifestation of "you will have it our way and you will like it" attitude of Windows UI team (or whoever is responsible for it).
It isn't 100% reliable but it's good enough for me. Now I hope to find something that prevents Windows 11 from dimming my screen when on battery after a few seconds of inactivity. Pretty please?
Maybe large swaths of users wouldn't download enhancement suites if you actually listened to them? It took like 3 years to support the basic feature to "never group tabs" in the Taskbar -- that's been supported by previous versions of windows for decades. That's why I downloaded patches.
adrian_b|2 years ago
This technique had already been used in many Microsoft programs for CP/M, i.e. for Intel 8080 or Zilog Z80 CPUs.
There were two reasons for the use of such jumps in the middle of instructions. One was to obfuscate the program for disassemblers, which were desynchronized with the instruction stream by such jumps. The other reason is that each such jump usually saved one byte of instruction code in comparison with conventional conditional program structures.
When the program size was still measured in kilobytes, saving even only one hundred bytes by this method was important.
The instruction into the middle of which a jump was performed was usually an instruction that loaded a 16-bit register with an immediate value that happened to be the encoding for some other instruction.
The value loaded into the register was not used. Therefore, when the instruction was entered through its start address, no operation was done, but when it was entered through a jump, some extra instruction was executed.
A typical use case would be at the beginning of a loop, which was entered after initializing a control variable, while whenever the loop was repeated later an extra instruction was executed (corresponding to the immediate value of the register load), to update the control variable. This saved one byte upon the standard loop implementation that uses an unconditional jump for entering the body of the loop in its middle, after the loop initialization.
praptak|2 years ago
There are exceptions though. Some sequences do encode biologically useful information with sequences which not only overlap but do so with the starting points differing by a number not divisible by 3: https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading...
rep_lodsb|2 years ago
The opcode will of course perform whatever function it was normally intended for, but it will have been chosen by the programmer to be something that doesn't affect the state of the program in a meaningful way. So it really is just a shorter form of jump, not something beyond the understanding of mere humans - you could always replace it with a regular jump and get the same semantics.
CPUs at the time didn't have branch prediction. The 8086 did prefetch code, but always linearly, and even an unconditional jump would flush that prefetch queue. So having an instruction like "CMP AX,xxxx" (where xxxx=any 16 bit immediate value) would be both shorter and faster than the "proper" jump instruction, while affecting only the flag register.
And on the 8080, there was no short (8 bit relative) jump instruction, so this kind of trick even saved two bytes instead of just one.
mst|2 years ago
> There’s at least one such utility out there that properly enough validates if their patches would work, or blocklist specific Windows builds known to break from it completely. I wish the other very-obvious-from-your-article software would do so too.
and I can't help but agree.
If you're going to reach into the guts of another process and start flipping bits, it's only Correct to take responsibility for doing so carefully enough you don't blow the entire thing up.
(I have written code that replaced/wrapped other people's function pointers for various reasons (occasionally even -good- ones) and considered any problem caused by an upgrade of the code hosting my cuckoo chick to be my bug - and while it was occasionally bloody annoying, I did this to myself ;)
bombcar|2 years ago
praptak|2 years ago
"Jon Ross, who wrote the original version of SimCity for Windows 3.x, told me that he accidentally left a bug in SimCity where he read memory that he had just freed. Yep. It worked fine on Windows 3.x, because the memory never went anywhere. Here’s the amazing part: On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the memory allocator in a special mode that doesn’t free memory right away. That’s the kind of obsession with backward compatibility that made people willing to upgrade to Windows 95."
https://www.joelonsoftware.com/2000/05/24/strategy-letter-ii...
throwawaaarrgh|2 years ago
Testing is the responsibility of the 3rd party (in addition to using supported methods). If the 3rd party was part of a preview release program, they could test changes before they are officially released.
justin66|2 years ago
Microsoft taking into account this method of modding explorer in its testing would be like asking Apple's design team to take into account the one in a million iPhone user who sticks their phone up their butt. [0] I don't know what it says about Windows or its users that there must be more than one in a million people running this stuff, but still.
[0] https://www.youtube.com/watch?v=bsbpFKDIaZ0
Ciantic|2 years ago
I've also made an explorer.exe in-memory patch to disable flashing taskbar buttons in Windows 10, it injected itself to WndProc and set the specific message to zero. In Windows 11 it's now a setting, so I don't need that anymore.
Now I'm investigating an explorer.exe patch for Windows 11 to speed up wallpaper animations. Which cannot be configured at the moment, there is only one setting to turn off all animations but no setting to speed up the wallpaper fade in animation.
vesinisa|2 years ago
Oh geez these are annoying! I've spent hours trying to find a setting to disable the flashing taskbar on the Teams client on my corporate issued Win10 laptop. The best I've found is to keep the Teams client minimized in tray just to avoid this behavior.
It boggles my mind there's no way to disable this feature that is so distracting it almost makes working impossible whenever I forget to kill the Teams client window. And really goes to show why people need to resort to these hacks.
nottorp|2 years ago
But but... think of their support costs if people accidentally move the taskbar!
They have to remove your options or they'll starve!
sandworm101|2 years ago
fargle|2 years ago
no sympathy. Chen seems a little irritated (as he should be) at the situation, but he's still missing (or purposely not stating) the real point. the correct response is alluded to by a couple folks in the comments section. i'll be more explicit:
- if microsoft was paying attention, it would simply add the functionality these 3rd party hacky extensions were adding, obviating the need for that extension at all. Like adding a setting for the flashing taskbar buttons you mentioned.
- failing that, simply allow some of those "internal" functions as a public extension points
the fact that a market for such egregious hacks implies significant demand for something.
gad, explorer and windows UI is such a dumpster fire. you don't even need to do any kind of fancy market research, just look at what the most popular extensions are doing to patch over deficiencies and them fix them!
unknown|2 years ago
[deleted]
kristianp|2 years ago
EdSharkey|2 years ago
Why is this allowed? This sounds like a small hell.
I left Windows the moment we shifted from "developers, developers, developers, developers" to "advertisers, advertisers, advertisers, advertisers" and never looked back.
Ubuntu is very pleasant to work with, nowadays.
unknown|2 years ago
[deleted]
omoikane|2 years ago
See also: https://www.hyrumslaw.com/
unknown|2 years ago
[deleted]
mbork_pl|2 years ago
layer8|2 years ago
deusum|2 years ago
saurik|2 years ago
Just look through the comments on this article to see people pointing out how Microsoft keeps removing options for how people can do this in less risky ways, or even ideas for how Microsoft can improve the safety of these extensions (which aren't all exactly right but I will note that if Microsoft embraced these more and provided an official loader--not one that limited the access but merely coordinated the installation--this is easily solved with the same kind of "safe mode" I designed for shell extensions on jailbroken iOS devices).
To quote one of the comments:
> Not just this article, but far too often when articles such as this are written, about widely used 3rd party tweaks, and the commenting strongly supports the tweak in question, but MS charges forward that “all these enthusiasts are wrong, we will continue our way” is telling of how far out of touch the Win dev team is away from its user base.
lcnPylGDnU4H9OF|2 years ago
> They don’t use the shell extension mechanism to get into the process. They sneak in by nefarious means. Patching is not supported. There is no “correct” way of doing it. Just different levels of bad.
I assume the “nefarious” ways are similar to rootkits in that they exploit vulnerabilities to gain system access they’re not intended to have.
It’s also only really the reason they don’t care that the patcher breaks. The reason I assume they don’t add or retain these features is maintenance and support cost.
13of40|2 years ago
layer8|2 years ago
rob74|2 years ago
That being said, no one forces you to use Windows either - except maybe your employer or the software you are using, but this is getting less and less of a problem fortunately (web apps, Proton etc.).
bigstrat2003|2 years ago
redleader55|2 years ago
The fact this is possible is the clowniest thing! Why would a major system component allow "extensions" to run without a sandbox and modify its code?
unknown|2 years ago
[deleted]
throwawaaarrgh|2 years ago
jackewiehose|2 years ago
Watching that I was just thinking NO NO NO - it worked fine for the last 25 years and there can be absolutely no reason why they had to destroy the taskbar now.
He deserves every headache he gets with these support requests.
Or to quote this comment of the linked article:
> There wouldn’t be as much of a need if the windows shell team was smarter [...] In fact I actively encourage, support, and celebrate efforts like this. Make their life hell and delay them. If they spend more time on investigating issues like this, they’ll have less time to f*k something else up.
[1]: https://www.youtube.com/watch?v=HDicLHBvQQM
daemin|2 years ago
surajrmal|2 years ago
bluecalm|2 years ago
To add a bit to this rant, here is a script that mostly solves the problem: https://github.com/CrypticButter/ButteryTaskbar
It isn't 100% reliable but it's good enough for me. Now I hope to find something that prevents Windows 11 from dimming my screen when on battery after a few seconds of inactivity. Pretty please?
userbinator|2 years ago
Do you have a timestamp? I'd rather not watch a 2h long video, even at 2x speed.
stronglikedan|2 years ago
dist-epoch|2 years ago
Also, "crippled", "destroyed", quite some heavy words. 99.9% of users don't care.
mauvia|2 years ago
unknown|2 years ago
[deleted]
TravHatesMe|2 years ago
TremendousJudge|2 years ago
unknown|2 years ago
[deleted]
Jones4000|2 years ago
[deleted]
revskill|2 years ago
[deleted]
protoman3000|2 years ago
[deleted]
snapcaster|2 years ago
[deleted]