Hotpatching a C Function on x86(nullprogram.com)
nullprogram.com
Hotpatching a C Function on x86
http://nullprogram.com/blog/2016/03/31/
21 comments
Back in the olden DOS days, instead of a configuration file, I had a struct in the code with the configuration data declaration. Then there was a static instance of the struct to inform the program of the current state.
To change the configuration, the program would write new values to the struct, and then patch the executable. Since I knew how to find the offset of the struct instance in the executable based on the runtime address of the instance, this was easy.
The big advantage to this was speed - being on floppy disk systems, having to do a file lookup/read was very slow.
Sadly, what killed this technique was when antivirus software appeared and decided that an executable patching itself was malware.
To change the configuration, the program would write new values to the struct, and then patch the executable. Since I knew how to find the offset of the struct instance in the executable based on the runtime address of the instance, this was easy.
The big advantage to this was speed - being on floppy disk systems, having to do a file lookup/read was very slow.
Sadly, what killed this technique was when antivirus software appeared and decided that an executable patching itself was malware.
Back in the olden Mac Plus days, I used a disassembler/monitor/mini-assembler called TMON[1] that allocated a chunk of otherwise unused memory - 256 bytes, I think, called "PlayMem".
Since linking on a Mac Plus took upwards of 11 minutes for our project, I'd would typically replace any particular line with a JMP $PlayMem to patch up whatever registers or memory wasn't right, and then jump back into the middle of the routine.
Oh, and then there was Steve Jasik's "The Debugger"[2] which would let you crash and then switch to a (capabilities limited) MPW environment to let you edit and re-link individual routines which it would then hot-patch back in place. I think for that one, he actually shipped own linker, which was a hand patched copy of Apple's.
Life in 24 bits of flat address space was more fun. Shorter, but more fun.
[1] http://www.mactech.com/articles/mactech/Vol.01/01.10/TMONDeb...
[2] http://www.jasik.com
Since linking on a Mac Plus took upwards of 11 minutes for our project, I'd would typically replace any particular line with a JMP $PlayMem to patch up whatever registers or memory wasn't right, and then jump back into the middle of the routine.
Oh, and then there was Steve Jasik's "The Debugger"[2] which would let you crash and then switch to a (capabilities limited) MPW environment to let you edit and re-link individual routines which it would then hot-patch back in place. I think for that one, he actually shipped own linker, which was a hand patched copy of Apple's.
Life in 24 bits of flat address space was more fun. Shorter, but more fun.
[1] http://www.mactech.com/articles/mactech/Vol.01/01.10/TMONDeb...
[2] http://www.jasik.com
DTrace has been doing this kind of things, in a safe and usable-in-production way, for years (and the code, while not trivial, is usually pretty cool to read): https://github.com/joyent/illumos-joyent/blob/master/usr/src...
For userspace probes DTrace uses INT3 AFAIK. Taking a signal each time a probe gets hit can be high overhead for certain usages though (like low overhead profiling as opposed to intrusive debugging).
Question: what if you just had some global function pointer that you would call instead, so hotpatching would just involve changing the pointer? That seems a lot simpler, there's less that can go wrong, and is a lot more portable. I get that this would probably be slower for the caller, but it's not that much slower. If the function is in the hot path that function pointer will be in cache anyway, so how slow is that read compared to the 8-byte NOP? It sounds to me like the OPs code will only be a cycle or two faster for each call during normal execution. Maybe I'm forgetting something.
it winds up being about the same. you need a pointer-wide atomic compare-and-swap (CAS) to swap out the function pointer safely.
on x86_64, a pointer-wide value is 8 bytes, which is enough space to put some instructions that detour to another function. so it's a question of whether you CAS the global function pointer, or the first 8 bytes of the function.
the "atomic CAS" part means you don't benefit from caching either, since you need to make sure the write is globally visible or threads on different cores will do different things. this kind of trickery combines two terrible things: reasoning about virtual method invocation (essentially) and lock-free programming techniques. have fun!
on x86_64, a pointer-wide value is 8 bytes, which is enough space to put some instructions that detour to another function. so it's a question of whether you CAS the global function pointer, or the first 8 bytes of the function.
the "atomic CAS" part means you don't benefit from caching either, since you need to make sure the write is globally visible or threads on different cores will do different things. this kind of trickery combines two terrible things: reasoning about virtual method invocation (essentially) and lock-free programming techniques. have fun!
Why do you need a CAS? Isn't an atomic write sufficient? Because we're just changing the function to use our code we don't really care what it was pointing to before, and we don't care if someone else changed the pointer before we did: it's just last write wins. In fact, the function pointer method will be able to have multiple threads try to hotpatch at once; idk about the OP code.
My guess is the OP's method can probably be extended to allow for more advanced hooking, like returning to the original function definition after running the new code or something. And, it means the caller doesn't have to know that the function is hot patchable, which I'm guessing is the most likely reason.
My guess is the OP's method can probably be extended to allow for more advanced hooking, like returning to the original function definition after running the new code or something. And, it means the caller doesn't have to know that the function is hot patchable, which I'm guessing is the most likely reason.
This works since x86 guarantees atomicity for aligned reads and writes w.rt. instruction fetch. LOCK prefix can be used for unaligned reads/writes to ensure atomicity of data reads and writes. But Intel SDM says (in 8.1.2 Volume 3A) "Locked instructions should not be used to ensure that data written can be fetched as instructions", which suggests instruction fetch is not atomic even with LOCK'd instructions, specially when the accesses are unaligned. Intel SDM (8.1.3 Volume 3A) suggests a cross modification protocol which requires global synchronization to ensure correct operation in such scenarios. Anyhow recently we unwittingly hit this limitation in our work and this led to some exploratory work for finding a way to relax the global synchronization requirement. Our work can be found at http://conf.researchr.org/event/pldi-2016/pldi-2016-papers-l... if anyone is interested.
Anyone interested in this might also be interested in Cycript [0] by saurik, which "allows developers to explore and modify running applications on either iOS or Mac OS X using a hybrid of Objective-C++ and JavaScript syntax [...]"
[0] http://www.cycript.org/
[0] http://www.cycript.org/
Huh.
Ages ago, I tried to figure out how to dynamically probe OpenGL extensions. My intent was that the first time a method call was attempted, the method proxy would
I didn't get very far. Looking at your implementation reminded me of that effort.
I haven't done OpenGL in probably 15 years. I don't even know if extension probing is still a thing (useful).
Thanks for sharing.
Ages ago, I tried to figure out how to dynamically probe OpenGL extensions. My intent was that the first time a method call was attempted, the method proxy would
look for a concrete implementation
on success, swap its pointer to the concrete implementation
on fail (not found), swap its pointer for a generic error method
then call itself again
I didn't get very far. Looking at your implementation reminded me of that effort.
I haven't done OpenGL in probably 15 years. I don't even know if extension probing is still a thing (useful).
Thanks for sharing.
Now, if we change the program while it is running, is it going to change the program on disk? I.E., once I've hotpatched this thing, can I rely on the patch "sticking" when I shutdown and reload my program? My understanding of this is rudimentary, so I want to make sure that I understand (even though I don't see a use case for me).
What, exactly, do you think is going on here? How on earth would the executable on the disk get modified by pointer manipulation? (without the use of mmap, of course)
Well, that's why I was asking, I wanted to understand. And now I do.
Are there any open-source tools (or even commerial options!) for hotpatching .NET assemblies at runtime?
As a C hacker, I really enjoy this guys blog. He posts about a lot of topics of interest to me.
[deleted]
If you enjoyed this, here's a great follow up:
http://jbremer.org/x86-api-hooking-demystified
Another interesting article by raymond chen on this topic: https://blogs.msdn.microsoft.com/oldnewthing/20110921-00/?p=...
I believe it's where the "ms" in "ms_hook_prologue" came from.
I believe it's where the "ms" in "ms_hook_prologue" came from.
There was some discussion of Raymond Chen's article a couple of months back [1].
According to the present article GCC emits an eight byte prologue (LEA RSP,[RSP+0x0]) at the start of the function, but Raymond Chen says that Microsoft's compiler emits five NOPS before the function start address and an overwritable two byte prologue (MOV EDI, EDI) at the start of the function itself. To me, Microsoft's approach seems more efficient - but I've never written any serious x86 assembly. Anybody knowledgeable want to comment on this?
[1] https://news.ycombinator.com/item?id=11063700
According to the present article GCC emits an eight byte prologue (LEA RSP,[RSP+0x0]) at the start of the function, but Raymond Chen says that Microsoft's compiler emits five NOPS before the function start address and an overwritable two byte prologue (MOV EDI, EDI) at the start of the function itself. To me, Microsoft's approach seems more efficient - but I've never written any serious x86 assembly. Anybody knowledgeable want to comment on this?
[1] https://news.ycombinator.com/item?id=11063700
One thing that comes to mind is that GCC's version could have a bit more overhead due to ESP-folding [0]. Basically, reading ESP/RSP directly can incur some overhead, because the register renamer is playing tricks to avoid actually adjusting the stack pointer on every push/pop until you actually read the stack pointer's value. It's unclear here why GCC chose RSP over some other register.
[0] the only reference I could find at the moment, but I've seen it documented elsewhere too: http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/fun...
[0] the only reference I could find at the moment, but I've seen it documented elsewhere too: http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/fun...
intel provides a tool called 'pin' that makes it possible to alter a program dynamically in runtime: https://software.intel.com/sites/landingpage/pintool/docs/65...
Intesting. ms_hook_prologue inserts 1 8-byte instruction ( lea rsp,[rsp+0x0]) instead of 8 1-byte nops. It executes much faster than 8 nopts if the function is not patched (no need to decode 8 instructions).
In the article it is also mentioned that it is important so that you can't have a thread that has "partially" executed your no-ops wen you replace it/them..
Shameless plug but my company is almost wholly oriented to binary intrumentation!
In our Deviare Hooking Engine/Deviare In Process [gihub-1] and RemoteBridge [github-2], we have a disassembler in place to hook Win32/COM/C++ vtables, so the hooking process is smarter if there are changes in the prologue. You can obviously take a look at the source code and learn a lot from it since it is state of art and perfectly competing with Microsoft Detours [3]. For an old comparison with Microsoft Detours you can check [4].
For anyone else looking for an extremely easy to use and higher level API, Deviare Hooking Engine makes extremely easy to hook and handle functions parameters and return value. Simple like this:
[snippet]
Docs are available here: http://www.nektra.com/products/deviare-api-hook-windows/doc-...
[github-1] https://github.com/nektra/Deviare2
[github-1 bis] https://github.com/nektra/Deviare-InProc
[github-2] https://github.com/nektra/RemoteBridge
[3] http://research.microsoft.com/en-us/projects/detours/
[4] https://www.reddit.com/r/programming/comments/22crn0/gpl_alt...
In our Deviare Hooking Engine/Deviare In Process [gihub-1] and RemoteBridge [github-2], we have a disassembler in place to hook Win32/COM/C++ vtables, so the hooking process is smarter if there are changes in the prologue. You can obviously take a look at the source code and learn a lot from it since it is state of art and perfectly competing with Microsoft Detours [3]. For an old comparison with Microsoft Detours you can check [4].
For anyone else looking for an extremely easy to use and higher level API, Deviare Hooking Engine makes extremely easy to hook and handle functions parameters and return value. Simple like this:
[snippet]
notepadPID = LaunchNotepadAndGetPid();
//in first place, hook DllGetClassObject of the target dll/ocx
hookDllGetClassObj = spyMgr.CreateHook("shell32.dll!DllGetClassObject", (int)eNktHookFlags.flgOnlyPostCall);
hookDllGetClassObj.Attach(notepadPID, true);
hookDllGetClassObj.Hook(true);
hookDllGetClassObj.OnFunctionCalled += OnDllGetClassObjectCalled;
Docs are available here: http://www.nektra.com/products/deviare-api-hook-windows/doc-...
[github-1] https://github.com/nektra/Deviare2
[github-1 bis] https://github.com/nektra/Deviare-InProc
[github-2] https://github.com/nektra/RemoteBridge
[3] http://research.microsoft.com/en-us/projects/detours/
[4] https://www.reddit.com/r/programming/comments/22crn0/gpl_alt...
Yes, while it's nice to learn about this, it's a lot more practical to use a library like that. Thanks for releasing your library as open source with a GPL license.
If I want to use it with the GPL license, that only means my own code using your library needs to have a GPL compatible license, right? I can use your library with my GPL compatible code to modify other closed-source or GPL-incompatible code, right? Just checking because some may view it as using a GPL plugin with GPL-incompatible program, which violates the GPL.
If I want to use it with the GPL license, that only means my own code using your library needs to have a GPL compatible license, right? I can use your library with my GPL compatible code to modify other closed-source or GPL-incompatible code, right? Just checking because some may view it as using a GPL plugin with GPL-incompatible program, which violates the GPL.
If I understand well you want to use Deviare in your GPL compatible project and hooking into a closed source application such as Microsoft Outlook, Internet Explorer, Skype, etc? The answer is yes, you can use Deviare in this context using the GPL license, we don't think that instrumenting a closed-source application via Deviare is part of your software, even when you can complement other closed-source applications.
Hm, didn't know putc was thread safe, but apparently it generally is (implementation dependent)[1]. Does this mean that using unlocked_stdio[1] is a good idea if you're writing single-threaded programs?
Another question, as I understand it, one must go to c11 before C gains any "standard" thread awareness -- does that mean that the use of a naked int for x in this article, probably should've been a _Atomic(int) x;? [ed: if the article conformed to c11 as opposed to c99, that is]
I suppose it depends where x ends up being stored, if multiple threads across multiple cores will always see/modify the same version of x?
http://linux.die.net/man/3/unlocked_stdio
Another question, as I understand it, one must go to c11 before C gains any "standard" thread awareness -- does that mean that the use of a naked int for x in this article, probably should've been a _Atomic(int) x;? [ed: if the article conformed to c11 as opposed to c99, that is]
I suppose it depends where x ends up being stored, if multiple threads across multiple cores will always see/modify the same version of x?
http://linux.die.net/man/3/unlocked_stdio
There are differences between atomicity and thread synchronization. An atomic write means that the variable is always in a correct state. For example, a non-atomic write could write half of the integer in one instruction and the other half in other instruction, so a thread could see that intermediate, incoherent state (actually, a write of a integer is always atomic in 32/64 bits, but this is just an example).
But that the write is atomic does not mean that it is synchronized with other cores. For example, thread A in core 1 could write the value of x, but later thread B in core 2 could read an outdated from its L1 cache. You should use memory barriers that force cache refreshing, so all the threads always see the same version of the variable and do not rely on possibly outdated cachés.
About the unlocked stdio, I don't think there would be a clear advantage. For best performance, I/O is done in blocks as big as possible, so locking mechanisms should not matter that much (you spend much more time in the actual I/O than in the locking). It should only affect significantly when doing lots of I/O calls, but even in that case, removing the locks would not improve performance as much as grouping and batching those calls.
But that the write is atomic does not mean that it is synchronized with other cores. For example, thread A in core 1 could write the value of x, but later thread B in core 2 could read an outdated from its L1 cache. You should use memory barriers that force cache refreshing, so all the threads always see the same version of the variable and do not rely on possibly outdated cachés.
About the unlocked stdio, I don't think there would be a clear advantage. For best performance, I/O is done in blocks as big as possible, so locking mechanisms should not matter that much (you spend much more time in the actual I/O than in the locking). It should only affect significantly when doing lots of I/O calls, but even in that case, removing the locks would not improve performance as much as grouping and batching those calls.
FTA: "In this post I’m going to do a silly, but interesting, exercise that should never be done in any program that actually matters."
What's the use case for this feature? Altering a thread's (or set of threads') work dynamically because synchronizing with those threads is too expensive? Or in an actual case would "goodbye()" contain code which was only generated as late as runtime?
It's neat, but I'm wondering why simpler things like updating a function pointer wouldn't be sufficient.
It's neat, but I'm wondering why simpler things like updating a function pointer wouldn't be sufficient.
Usually it's done when redesigning/recompiling the target function is not possible. Game modding, restartless updates, instrumentation, etc.
But that would only be possible from the same process space, right? So only applicable for plugins etc.?
What I'd like to see is hotpatching the function for another process, but I guess that's very hard to do with ASLR. Probably doable with some tricks, though.
Edit: Come to think of it, gdb is able to attach to a running process w/o debug symbols and find function addresses. So in other words, I just need to dig into and grok the gdb source.
What I'd like to see is hotpatching the function for another process, but I guess that's very hard to do with ASLR. Probably doable with some tricks, though.
Edit: Come to think of it, gdb is able to attach to a running process w/o debug symbols and find function addresses. So in other words, I just need to dig into and grok the gdb source.
The program being modified doesn't need some feature for loading your code as a plugin. You can instead use https://en.wikipedia.org/wiki/DLL_injection
If you are able to recompile the program, you can disable ASLR. For example on iOS it just requires a change to the MACH-O header of the binary. [0]
But I doubt you could disable ASLR of a running process, for somewhat obvious reasons...
[0] https://github.com/peterfillmore/removePIE
But I doubt you could disable ASLR of a running process, for somewhat obvious reasons...
[0] https://github.com/peterfillmore/removePIE
You can similarly disable ASLR via the header in Windows, though there are ways to override that.
ASLR shouldn't be a big problem though. Only the base address of code in each file (executable or library) is changed, and you can easily find it. Functions within one file are not shuffled around. ASLR only exists to stop you from hard-coding function addresses, to make exploits harder.
ASLR shouldn't be a big problem though. Only the base address of code in each file (executable or library) is changed, and you can easily find it. Functions within one file are not shuffled around. ASLR only exists to stop you from hard-coding function addresses, to make exploits harder.
If you want restartless updates, then you plan for it and modularize the software with dynamically loadable modules (think shared library "plugins" or kernel modules).
You're talking about when you own the source and therefore can redesign/recompile it. I began my comment explicitly ruling those cases out.
Let's say you're Red Hat, and want to release a super critical security fix for some package, but you absolutely cannot restart the affected process. This technique allows you to write some code that redirects functions to fixed versions, patching the process while it runs.
Let's say you're Red Hat, and want to release a super critical security fix for some package, but you absolutely cannot restart the affected process. This technique allows you to write some code that redirects functions to fixed versions, patching the process while it runs.
If you don't control the code and can't recompile it, you won't be able to add the ms_hook_prologue function property
If you don't control the code and are modifying it to do an in-service upgrade of a running binary, then this is a special situation: you're shooting a 2016 re-make of MacGyver targeted at a geek/programmer audience.
If you're Red Hat and you do have the source, but the service cannot be stopped, and yet it must be upgraded, yet you didn't plan this into the software (and yet you didn't write in it in Lisp which could save your ass here) then you're plain stupid. If you're stupid, you're not going to pull off this highly technical, intricate hack, that is looming with pitfalls.
If you're Red Hat and you do have the source, but the service cannot be stopped, and yet it must be upgraded, yet you didn't plan this into the software (and yet you didn't write in it in Lisp which could save your ass here) then you're plain stupid. If you're stupid, you're not going to pull off this highly technical, intricate hack, that is looming with pitfalls.
I mentioned this in another comment, but you don't need the ms_hook_prologue attribute, it's just helpful.
True, you don't usually need the prologue to hook a function, but we're talking about hooking functions that might be called by another thread at the same time.
If you just modify the instructions while another thread is running the code, you might crash and burn.
The prologue helps by ensuring the first 8 bytes of a function are just a large NOP.
Except the technique presented requires recompiling the target function, doesn't it? (To add all those GCC attributes, and to add the `lea` instruction.)
The specific technique shown here requires it, but you can also do it generically by overwriting the first 8 bytes of the function and then restoring it later if you want to undo the hook.
If you change the function pointer, then you have to be aware of the calling convention used by the compiler and make sure you replicate it. Another can of worms. https://en.wikipedia.org/wiki/X86_calling_conventions
as for the synchronization use case, even though it's probably way more expensive to invalidate another thread's instruction cache than any other usual sync. method, it can be useful if there is a huge number of synchronization point but it happens very rarely and you don't want to bloat the code with explicit checks everywhere.
An example of that is garbage collection safepoints, although in that case you also need to be able to patch a function in the middle (any potentially unbounded control flow loop)
An example of that is garbage collection safepoints, although in that case you also need to be able to patch a function in the middle (any potentially unbounded control flow loop)
If you want something simpler:
http://www.frida.re/
It's by far the simplest hooking framework on the planet.
http://www.frida.re/
It's by far the simplest hooking framework on the planet.
if you want to do something like change a function definition or data type at runtime "the right way" you could use the kitsune dynamic software updating framework to do it: https://github.com/kitsune-dsu
The linux kernel also hotpatches itself (not just kpatch either) for performance and errata fixing. For example:
https://lwn.net/Articles/620640/
Other unix kernels do the same for much the same reasons.
https://lwn.net/Articles/620640/
Other unix kernels do the same for much the same reasons.
[1] https://gcc.gnu.org/onlinedocs/gcc-4.7.2/gcc/Function-Attrib... [2] https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#GotoLab...