In situations like this, you can report the bug to Microsoft Connect:
1. Submit bug report.
2. Wait six months.
3. MS tech will post a comment, "this will be fixed in the next release".
4. Wait two more years.
5. Bug report will be closed as "won't fix".
This. One bug in IE9 clickOnce launching thanks to them changing how download prompting works.
1. Reported to connect whilst in preview release status. Closed. Reported again. Closed. FULL test cases provided.
2. We're a gold partner with a £500k spend a year on licenses. Partner support. 19 hours on the phone over 6 months, blame shifting between the IE and .net teams and a daily call to get the case closed without resolution. Got a registry patch from ass-end support after 4 months that we have to ship to 2000 users at 200 different companies rather than an upstream fix. This checks a check box in the security settings.
They broke their own product and won't fix it. Basically you can't use JS to redirect to a clickonce URL.
Now today, IIS just stopped serving shit with no errors, nothing. Can't get anything out of minidumps+windbg. Just stops. None of our code is running.
I've had more luck then. Strange they first say it will be fixed and then later say it won't be fixed anyway, I always had the impression they had their stuff together at Connect. Slowly of course but I guess that's what's to be expected because of the sheer size (this is not in their defense, at all, just my idea of why that is). Probably also spend tons of time looking at issues posted by people thinking Connect is a Q&A like StackOverFlow. Anyway for the 6 or so bugs I filed, all for the C++ compiler, the flow was
1. Submit bug report.
2. Get response like "we're looking into it"
3. Wait 2 to 5 months
4a. MS tech (or sometimes event Stephan T. Lavavej himself) posts a comment, "this will be fixed in the next release"
5. Bug report closed as fixed
6. Wait x time where x mainly depends on the point in time of the release cycle where 1 occurred. The later 1, the smaller x. You've gotta time those bugs to get them fixed quickly :P
4b: "won'tfix" / "by design" i.e. I got an explanation of what I did wrong, or otherwise why it happens
4c: "deferred" i.e. posted something that can be worked around and considered too minor of an issue to fix soon
Funny, I had something to do with this code back in the day! I'm guessing it was a copy+paste bug and they copied from the LambdaCompiler, which uses StrongBox<T> for its closed-over parameters[1], since StrongBox<T>.Value is a field. The idea was to have the closures be really fast.
The history of ET compiler: it started with LINQ in .NET 3.5. Originally it was pretty simple and just handled expressions. In .NET 4.0 we merged the entire codebase with the IronPython/IronRuby compiler trees, expanding the "expression trees" to handle statements. IIRC, it can generate almost any IL construct that you might need, and is usually a lot easier to work with. But we found .NET's runtime compiler (DynamicMethod) was a bit too slow for a lot of use cases. It also wasn't supported on some CLR configurations. To address this we wrote an interpreter and some heuristics to switch from interpreted to compiled. But the actual System.Linq.Expressions.Interpreter must have happened after 4.0, because I don't remember that at all. Instead we just shipped it as a shared library used by IronPython and IronRuby.
IIRC, ExpressionQuoter was mainly to support the Quote expression, and was always a bit buggy. The 3.5 version was seriously messed up, and our prerelease versions of .NET 4.0 also had various bugs, and very few tests. I tried to fix it by having it use the same reuse closure mechanism as the normal compiler. Funny that same feature caused issues later on.
[1] one might wonder: why use StrongBox<T>, essentially boxing every parameter, rather than just generating a type with only the right fields? The reason was that generating a type in .NET 4.0 timeframe was absurdly slow. Like, a few hundred per second slow. I think this has been largely fixed now, but it was a huge performance problem for Iron* language runtimes back in the day
Should have just used dotPeek instead of ilspy and writing IL code by hand. Recompiling would have certainly been easier. http://www.jetbrains.com/decompiler/
Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; and GCC maintainers insist on precise test cases to vet a bug, even if the issue is immediately obvious from reading the source code and the bug only occurs in certain very complex situations.
(/me looks forward to the day Clang/LLVM becomes the default on Linux…)
So, no, it's not next-to-impossible. It's a little complicated, and that's why these projects exist, but my memory of crosstool was that it made it trivially easy: run the script, let it compile for a bit, and voila, you now have a brand-new cross-compiler toolchain.
Cross-compiling and Canadian-cross isn't fun but isn't that bad. There are good projects out there to help in the process. http://crosstool-ng.org/ is a good project that can make it much easier. They have Canadian-cross support but I haven't used it.
Just recently I wrote https://github.com/ikonst/LongPathFix to work around a clang/gcc issue, cause it seemed easier than going through the hoops of building clang for Win32, understanding its code and writing a patch good enough to be accepted.
I feel like a "freetard" "greybeard" "basement-dweller" (insert-your-own-pejorative-here) saying this, but: yet another reason why I will never ever base my livelihood on a closed ecosystem. Open source is certainly not a panacea, but needing to do something like this is just ridiculous.
I know a lot of people here balk at the idea of paying for tooling, but Red Gate's Reflector[1] is absolutely amazing for situations like this. Not only is it a decompiler, but it allows you to decompile at debug-time and step into third party libraries.
Assuming they haven't been obfuscated, this is an extremely useful tool. I've used it to track down a number of issues within Visual Studio itself and within some of the non-open sourced components of Roslyn.
I take issue with the authors use of the word "patch". He's not actually patching the DLL, just decompiling and recompiling which are quite familiar for most programmers. There might be some black magic associated with this particular DLL that we non-.net programmers doesn't understand.
I would personally be very careful with the recompiling dance, at least in other languages, as alignments and such might come out of place. A patch feels much less dangerous if this is something that is to be deployed.
(I always also try to rig such builds so that the build bombs if the dependencies change. That way it doesn't survive version changes without forcing someone to take a long hard look at it.)
DLLs are signed. But they aren't checked every time a DLL is loaded. It absolutely obliterates load time if you have to go through every byte of all your DLLs and hash them. (You have the option to turn this on within the registry, though)
Signing is almost useless in .Net. And it's certainly not in place for security purposes.
Don't worry, each new version of C# gets more features from F# and Scala, this year we will get more syntax sugar and an extensible compiler (with proper IDE support - unlike F#): https://roslyn.codeplex.com/wikipage?title=Language%20Featur... . They will add pattern matching and maybe something more interesting in the version after that. There aren't too many practical reasons to pick F# over C# today and soon there will be even less. Lesson to learn: don't pick *ML language as a foundation. I am glad that Scala is evading F#'s academic fate.
I do both C# and F#. But, at this point I think in ML no matter what which language I'm using. That said, though I'm delighted by F#, there are things that I really miss about C#:
* Haskell's type classes are awesome, and neither F# nor C# have them. But, you can realize them with a really simple pattern in C# whereas in F# you have to resort to reflection or generic hacks[1] (but inline functions usually make such hacks nicer in F#).
* Sum types are awesome. F# realizes them in two ways: with discriminated unions and pattern matching (mainly), or with abstract classes and inheritance. Both are powerful approaches, but the latter is particularly useful when you need unbounded sum types. Unfortunately, inheritance is somewhat crippled in F# because you can't define protected members (though you can override them). I find myself needing this when I'm designing a solution rather than merely implementing one. For instance, I find that designing a language/compiler is much easier with C# than with idiomatic F#.
* This is kind of a summary of the first two points: idiomatic C# is a better dynamic language, and I'm not referring to the `dynamic` keyword, though it certainly helps. I've created some really nice and secure APIs by making use of user-defined conversions. System.Linq.Xml is a good example of what I'm talking about.
* Finally, idiomatic C# is a nicer classical OO language than idiomatic F#. This is kind of obvious since F# is a "functional language" with OO features whereas C# is an OO language with functional features. But, when I say classical OO, I mean the Smalltalk style OO that came before C++. Alan Kay defined that kind of OOP as (1) Encapsulation, (2) Message Passing, and (3) extreme late binding. Neither C# nor F# quite has (2) and (3), but C# gets me the closest without coloring too far outside of the lines. Of course, this begs the question that OO is better than functional. Having done both for a long time, I'm convinced that it is the case (though I do think learning functional programming is the fastest path to getting good at OO). Lamentably, I can't yet defend this point too vigorously because I've only just discovered the joy or Smalltalk. But, here is a preview of sorts: Imagine being able to update a database schema and having your statically-typed code continue to run without needing to recompile.
Wow, that is some fancy work. Though I shudder to think about deploying and maintaining that fix, and bringing new developers up to speed on why and how it was done.
I was also about to tell him to skip all the ildasm stuff and just use the online reference source (referencesource.microsoft.com) ... but that assembly isn't in there. So I guess ildasm was the best option.
The fact that people think you shouldn't be able to do this is what really scares me. :)
I should be able to change the software on my own machine, stored on and running on hardware I own, in whatever way I desire and have it do what I want. (And in practice I have - opening a binary in a hex editor and changing a few bytes is not at all beyond me.)
You can produce and consume signed libraries in .NET very easily. In practice it's becoming less common as .NET becomes more open.
As an aside, OP says he works with Bart De Smet and they're messing with expression trees. I bet that this is work related to Cortana and the expression trees are used for compiling standing Rx (Reactive Extensions) queries which Cortana uses extensively on the phone and in the cloud.
The real news here is that this is considered special.
In most other languages, they'd have forked the source, fixed it, recompiled it for their own uses and submitted a pull request or patch.
I'm a big .NET fan, but the fact that we have to jump through such hoops to find and fix a bug, and then still have near certainty that we're going to have to reapply the patch for many updates to come, well, that's just a bit sad. It feels rather last-century, to be honest. Microsoft could save a lot of double work if they'd just open source .NET and attach a decent process to it. They can still be the Linus. Just consider my patches.
1) You can file bug reports against .NET and they get fixed. (Typically very slowly, but that's enterprise vendors for you...)
2) The tools they used to find & fix the bug are all based on fully-specified, documented, and supported mechanisms: ildasm and ilasm are stock .NET tools, and ilspy is using Mono's IL decompiler/compiler library.
3) You can compile an app against your own builds of any of the standard libraries if you want (including mscorlib, with caveats) and run it, so a fix like this can be deployed and maintained as much as you like, even if it's not Great Engineering.
4) This hand-patched DLL wouldn't corrupt any other app on your system, and you can ship and version it yourself, so it wouldn't break as a result of a system-wide .NET hotfix or anything like that. .NET's fully-specified ABI also means that it will work across all supported platforms.
5) Mono is open source, so you could always ship against mono and mono's mscorlib (both open source, naturally) if you absolutely can't wait for a .NET bugfix to ship AND you're unwilling to patch the dll yourself.
People love to bitch about .NET, but it actually provides a really good ecosystem here. Very little of it is truly a black box, the spec is extremely readable and precise about the things it covers, IL disassembly is extremely readable, IL disassembly fed through a decompiler like ILspy/dotPeek is doubly readable, debugging is straightforward even without symbols, and recompilation/patching is trivial because everything is well-specified and there's a robust open source library for it. (Even if there wasn't a library, you could just use the standard ildasm/ilasm pair.)
Oh, and you can compile native no-kidding C/C++ to .NET targeting the same instruction set and get all the benefits of the above. If you want to.
I work at a startup that uses almost all open source tools for our stack. To not be able to fork, fix, pull request a bug in software seems...barbaric to me.
What they're doing here is monkeypatching the language runtime. That's not trivial in most languages, least of all ones which typically rely on a centralized install of that language runtime to function.
The way they did this, only the program that needs this fix is actually using their monkeypatched Expressions library; other code on the same system - other libraries in the same application, even - will continue to use the standard .NET runtime classes. That's a pretty powerful capability, and not one that is necessarily made easier by an open language runtime.
I'm sad to report that I tried to replicate what the author did here and it didn't work...
I went to the weekly tech lead meeting, and when it was my turn to talk, I said: "I think we should increase out internal NuGet package release interval to every 18 hours. F--k the Police!"
[+] [-] stusmith1977|11 years ago|reply
1. Submit bug report. 2. Wait six months. 3. MS tech will post a comment, "this will be fixed in the next release". 4. Wait two more years. 5. Bug report will be closed as "won't fix".
[+] [-] allegory|11 years ago|reply
1. Reported to connect whilst in preview release status. Closed. Reported again. Closed. FULL test cases provided.
2. We're a gold partner with a £500k spend a year on licenses. Partner support. 19 hours on the phone over 6 months, blame shifting between the IE and .net teams and a daily call to get the case closed without resolution. Got a registry patch from ass-end support after 4 months that we have to ship to 2000 users at 200 different companies rather than an upstream fix. This checks a check box in the security settings.
They broke their own product and won't fix it. Basically you can't use JS to redirect to a clickonce URL.
Now today, IIS just stopped serving shit with no errors, nothing. Can't get anything out of minidumps+windbg. Just stops. None of our code is running.
Who am I going to call?
Redhat that's who.
[+] [-] stinos|11 years ago|reply
1. Submit bug report. 2. Get response like "we're looking into it" 3. Wait 2 to 5 months
4a. MS tech (or sometimes event Stephan T. Lavavej himself) posts a comment, "this will be fixed in the next release" 5. Bug report closed as fixed 6. Wait x time where x mainly depends on the point in time of the release cycle where 1 occurred. The later 1, the smaller x. You've gotta time those bugs to get them fixed quickly :P
4b: "won'tfix" / "by design" i.e. I got an explanation of what I did wrong, or otherwise why it happens
4c: "deferred" i.e. posted something that can be worked around and considered too minor of an issue to fix soon
[+] [-] InclinedPlane|11 years ago|reply
[+] [-] jmesserly|11 years ago|reply
The history of ET compiler: it started with LINQ in .NET 3.5. Originally it was pretty simple and just handled expressions. In .NET 4.0 we merged the entire codebase with the IronPython/IronRuby compiler trees, expanding the "expression trees" to handle statements. IIRC, it can generate almost any IL construct that you might need, and is usually a lot easier to work with. But we found .NET's runtime compiler (DynamicMethod) was a bit too slow for a lot of use cases. It also wasn't supported on some CLR configurations. To address this we wrote an interpreter and some heuristics to switch from interpreted to compiled. But the actual System.Linq.Expressions.Interpreter must have happened after 4.0, because I don't remember that at all. Instead we just shipped it as a shared library used by IronPython and IronRuby.
Here's the normal ExpressionQuoter: https://github.com/IronLanguages/main/blob/7be8b73e246bfb029...
And here was the interpreter. I don't see the ExpressionQuoter, so either that's a newer fork of the code that was rolled into System.Core, or maybe a completely new implementation. https://github.com/IronLanguages/main/tree/master/Runtime/Mi...
IIRC, ExpressionQuoter was mainly to support the Quote expression, and was always a bit buggy. The 3.5 version was seriously messed up, and our prerelease versions of .NET 4.0 also had various bugs, and very few tests. I tried to fix it by having it use the same reuse closure mechanism as the normal compiler. Funny that same feature caused issues later on.
[1] one might wonder: why use StrongBox<T>, essentially boxing every parameter, rather than just generating a type with only the right fields? The reason was that generating a type in .NET 4.0 timeframe was absurdly slow. Like, a few hundred per second slow. I think this has been largely fixed now, but it was a huge performance problem for Iron* language runtimes back in the day
[+] [-] CurtHagenlocher|11 years ago|reply
[+] [-] nope_42|11 years ago|reply
[+] [-] antics|11 years ago|reply
AWESOME TIP, stoked to try it out, thanks!
[+] [-] colanderman|11 years ago|reply
"But GCC's open source!" you say.
Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; and GCC maintainers insist on precise test cases to vet a bug, even if the issue is immediately obvious from reading the source code and the bug only occurs in certain very complex situations.
(/me looks forward to the day Clang/LLVM becomes the default on Linux…)
[+] [-] zvrba|11 years ago|reply
You're exaggerating. I had no problems compiling gcc 3.something targeting MIPS-I on an x86 Linux host.
[+] [-] davvid|11 years ago|reply
http://kegel.com/crosstool/
I haven't really kept up with the latest stuff, but I've seen mentions of crosstool-ng, which may be a newer take on it:
https://github.com/diorcety/crosstool-ng
So, no, it's not next-to-impossible. It's a little complicated, and that's why these projects exist, but my memory of crosstool was that it made it trivially easy: run the script, let it compile for a bit, and voila, you now have a brand-new cross-compiler toolchain.
[+] [-] stusmall|11 years ago|reply
[+] [-] ikonst|11 years ago|reply
[+] [-] kelnos|11 years ago|reply
[+] [-] Permit|11 years ago|reply
Assuming they haven't been obfuscated, this is an extremely useful tool. I've used it to track down a number of issues within Visual Studio itself and within some of the non-open sourced components of Roslyn.
[1] http://www.red-gate.com/products/dotnet-development/reflecto...
[+] [-] xorcist|11 years ago|reply
I would personally be very careful with the recompiling dance, at least in other languages, as alignments and such might come out of place. A patch feels much less dangerous if this is something that is to be deployed.
(I always also try to rig such builds so that the build bombs if the dependencies change. That way it doesn't survive version changes without forcing someone to take a long hard look at it.)
[+] [-] b0b0b0b|11 years ago|reply
[+] [-] Permit|11 years ago|reply
Signing is almost useless in .Net. And it's certainly not in place for security purposes.
[+] [-] kevingadd|11 years ago|reply
[+] [-] DanielBMarkham|11 years ago|reply
This code is getting to look butt-ugly. It is not a good thing if it continues like this. We already have C++. Don't need another one.
[+] [-] _random_|11 years ago|reply
[+] [-] noblethrasher|11 years ago|reply
* Haskell's type classes are awesome, and neither F# nor C# have them. But, you can realize them with a really simple pattern in C# whereas in F# you have to resort to reflection or generic hacks[1] (but inline functions usually make such hacks nicer in F#).
* Sum types are awesome. F# realizes them in two ways: with discriminated unions and pattern matching (mainly), or with abstract classes and inheritance. Both are powerful approaches, but the latter is particularly useful when you need unbounded sum types. Unfortunately, inheritance is somewhat crippled in F# because you can't define protected members (though you can override them). I find myself needing this when I'm designing a solution rather than merely implementing one. For instance, I find that designing a language/compiler is much easier with C# than with idiomatic F#.
* This is kind of a summary of the first two points: idiomatic C# is a better dynamic language, and I'm not referring to the `dynamic` keyword, though it certainly helps. I've created some really nice and secure APIs by making use of user-defined conversions. System.Linq.Xml is a good example of what I'm talking about.
* Finally, idiomatic C# is a nicer classical OO language than idiomatic F#. This is kind of obvious since F# is a "functional language" with OO features whereas C# is an OO language with functional features. But, when I say classical OO, I mean the Smalltalk style OO that came before C++. Alan Kay defined that kind of OOP as (1) Encapsulation, (2) Message Passing, and (3) extreme late binding. Neither C# nor F# quite has (2) and (3), but C# gets me the closest without coloring too far outside of the lines. Of course, this begs the question that OO is better than functional. Having done both for a long time, I'm convinced that it is the case (though I do think learning functional programming is the fastest path to getting good at OO). Lamentably, I can't yet defend this point too vigorously because I've only just discovered the joy or Smalltalk. But, here is a preview of sorts: Imagine being able to update a database schema and having your statically-typed code continue to run without needing to recompile.
[1] http://stackoverflow.com/questions/9868327/f-type-constraint...
[+] [-] breischl|11 years ago|reply
I was also about to tell him to skip all the ildasm stuff and just use the online reference source (referencesource.microsoft.com) ... but that assembly isn't in there. So I guess ildasm was the best option.
[+] [-] jacquesm|11 years ago|reply
[+] [-] userbinator|11 years ago|reply
I should be able to change the software on my own machine, stored on and running on hardware I own, in whatever way I desire and have it do what I want. (And in practice I have - opening a binary in a hex editor and changing a few bytes is not at all beyond me.)
[+] [-] Permit|11 years ago|reply
Second, if you're able to modify one DLL, you could overwrite whatever DLL was tasked with computing the checksum.
[+] [-] reubenbond|11 years ago|reply
As an aside, OP says he works with Bart De Smet and they're messing with expression trees. I bet that this is work related to Cortana and the expression trees are used for compiling standing Rx (Reactive Extensions) queries which Cortana uses extensively on the phone and in the cloud.
[+] [-] Locke1689|11 years ago|reply
[+] [-] x0n|11 years ago|reply
[+] [-] swalsh|11 years ago|reply
I wish i knew about it about a year ago for a project I did.
[+] [-] rubyrescue|11 years ago|reply
[+] [-] skrebbel|11 years ago|reply
In most other languages, they'd have forked the source, fixed it, recompiled it for their own uses and submitted a pull request or patch.
I'm a big .NET fan, but the fact that we have to jump through such hoops to find and fix a bug, and then still have near certainty that we're going to have to reapply the patch for many updates to come, well, that's just a bit sad. It feels rather last-century, to be honest. Microsoft could save a lot of double work if they'd just open source .NET and attach a decent process to it. They can still be the Linus. Just consider my patches.
[+] [-] antics|11 years ago|reply
I should disclose that I actually work for Microsoft, and still chose the hard way. Fuck the police!
[+] [-] kevingadd|11 years ago|reply
2) The tools they used to find & fix the bug are all based on fully-specified, documented, and supported mechanisms: ildasm and ilasm are stock .NET tools, and ilspy is using Mono's IL decompiler/compiler library.
3) You can compile an app against your own builds of any of the standard libraries if you want (including mscorlib, with caveats) and run it, so a fix like this can be deployed and maintained as much as you like, even if it's not Great Engineering.
4) This hand-patched DLL wouldn't corrupt any other app on your system, and you can ship and version it yourself, so it wouldn't break as a result of a system-wide .NET hotfix or anything like that. .NET's fully-specified ABI also means that it will work across all supported platforms.
5) Mono is open source, so you could always ship against mono and mono's mscorlib (both open source, naturally) if you absolutely can't wait for a .NET bugfix to ship AND you're unwilling to patch the dll yourself.
People love to bitch about .NET, but it actually provides a really good ecosystem here. Very little of it is truly a black box, the spec is extremely readable and precise about the things it covers, IL disassembly is extremely readable, IL disassembly fed through a decompiler like ILspy/dotPeek is doubly readable, debugging is straightforward even without symbols, and recompilation/patching is trivial because everything is well-specified and there's a robust open source library for it. (Even if there wasn't a library, you could just use the standard ildasm/ilasm pair.)
Oh, and you can compile native no-kidding C/C++ to .NET targeting the same instruction set and get all the benefits of the above. If you want to.
[+] [-] guelo|11 years ago|reply
Normally if you find a bug in the core you might try to monkeypatch but many times that won't work so you have to suck it up and just work around it.
[+] [-] toomuchtodo|11 years ago|reply
[+] [-] jameshart|11 years ago|reply
The way they did this, only the program that needs this fix is actually using their monkeypatched Expressions library; other code on the same system - other libraries in the same application, even - will continue to use the standard .NET runtime classes. That's a pretty powerful capability, and not one that is necessarily made easier by an open language runtime.
[+] [-] mrb|11 years ago|reply
[+] [-] Bognar|11 years ago|reply
http://referencesource.microsoft.com/#System.Core/Microsoft/...
[+] [-] serve_yay|11 years ago|reply
[+] [-] batmansbelt|11 years ago|reply
[deleted]
[+] [-] euroclydon|11 years ago|reply
I went to the weekly tech lead meeting, and when it was my turn to talk, I said: "I think we should increase out internal NuGet package release interval to every 18 hours. F--k the Police!"
People just looked at me weird.
[+] [-] cornholio|11 years ago|reply