You read your chips' datasheets that go in detail on pinouts and protocols. You use logic analyzers that capture signals and programmatically decode a multitude of protocols from those signals. When you don't know what are the pinouts or protocols, you compare against similar enough known ones, or bruteforce them.
Some examples:
* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.
* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.
* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.
* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.
* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.
So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.
When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.
It looks like the compilation database is only used for discovering source files, without any additional cross translation unit analysis [1]. Even if you don't plan on implementing data flow tracking or other passes, it can still be interesting to know on which translation unit a function declaration is defined on.
Also, it seems we always have to recompile each unit when switching between source files. Consider pre-compiling and caching so it's only done once. That should also enable users to search across files/ASTs.
> Fraud detection in transaction data is mostly SQL. Not machine learning, not graph databases, not whatever Gartner is hyping this year. SQL, run against the right tables, with the right joins, looking for the right shapes.
It's also not all program-integrity, which is the only work that could justify such blanket statements. Worse is better as long as it addresses the problem domain.
Fintech clients are generally interested in knowing whether a transaction happening _right now_ is fraud. They want to know that in a few milliseconds, for high-dimensional data. It's work done at a scale where relational databases cannot meet these real-time constraints, and instead find other uses like historical data loading. That's how you end up with in-memory databases, stream-processing engines, and yes, even machine learning.
Having said that, some of the author's points are valid, and I'm looking forward for their next writings, in particular dealing with noisy alerts is a general problem beyond performance engineering.
The author is referring to limitations in analysing banking:
> Ghidra supports the 8051 architecture but not code banking.
Usually in these ISAs an I/O port or a register sets the bank number, so any processor module should be able to resolve concrete banked references. But you still need to know what that register holds in various code paths, which are likely dynamically computing those values.
No tooling can give this out-of-the-box, as it relies on knowing the concrete initial state of the system (i.e. memory and register contents), and knowing what to return when hooking into I/O accesses.
Once these are known, we can leverage the built-in pcode emulator and run it with this state. It seems nowadays Ghidra has some built-in support for Z3, but I personally never used it, so I'm not sure how viable it is for symbolic execution. Regardless, with either approach, we would now have concrete banked code references being resolved, and could script some auto annotation of the disassembly with these references. These would be equivalent to what the author gathered from the logic analyzer trace.
A pure static analysis approach seems to suggest one would manually brute-force through all possible bank numbers at any given code path, which I guess is only viable if you have the time for that.
Which drives and parameters for the READ BUF SCSI command yielded the expected 2366 bytes per sector? I imagine that it was combined with seeks to each sector before reading from the buffer (as it would be harder to isolate multiple sectors data in cache?).
Nice work Dmitry, looking forward to read your next article.
The later model Pixter Multimedia had the full memory space accessible via JTAG, which is how some carts and even boot ROM got dumped a while ago [1], is it the same deal with Pixter Color?
That OpenOCD script was a bit flaky, and sometimes the boot ROM would be already unloaded before reading, maybe you have some insights in how to make it more robust.
btw, have you looked into the original Pixter? The cart connector seems to have a very narrow bus, so it doesn't look like those carts have code, and probably can only be dumped with a decap.
For other architectures, it feels like a missed opportunity to not have an independent WASM build of MAME's debugger, as the whole project could already be built in WASM (although I think the latest versions were broken, as that target isn't actively maintained): https://docs.mamedev.org/initialsetup/compilingmame.html#ems...
I found their ELF format specification to have a decent coverage, even if not completely exhaustive (e.g. some debug info isn't breakdown after a certain point, but it just might be incomplete rather than limitations).
Besides the implementation effort in building from scratch, it also opens you up to unknown unknowns, while the limitations of mature software are better defined. Just because Jepsen didn't test other alternatives doesn't mean they are free from issues.
Regarding the Kafka issues pointed out:
* 2 only affect you if you are using transactions (and I would be interested in knowing of alternatives that better handle exactly-once semantics);
* Consumer.close() is likely off the critical path of your applications (personally I never encountered these hangs);
* Aborted reads and torn transactions were mentioned as being fixed by KIP-890, of which the mentioned ongoing work was completed since that article was published: https://issues.apache.org/jira/browse/KAFKA-14402
Without discrediting the author, as it's always cool to share these findings, was anyone actively looking for this over 27 years?
A lot of the times, it just happens that someone was the first person that even bothered trying digging into the code. Specially after decompilation became much more accessible for less popular architectures with Ghidra. Give it a try, there's plenty of low hanging fruit! I've submitted another case some time ago.
Also luckily, considering other OS easter eggs, it doesn't seem like there was any obfuscation involved, like "chained xor stored in bitmap resource of badly supported executable format": https://x.com/mswin_bat/status/1504788425525719043
Can you elaborate on that "little more work", given that resizing on demand isn't sufficient for this use-case, and predictive scaling is also out of the question?
AWS autoscaling does not take your application logic into account, which means that aggresive downscaling will, at worst, lead your applications to fail.
I'll give a specific example with Apache Spark: AWS provides a managed cluster via EMR. You can configure your task nodes (i.e. instances that run the bulk of your submitted jobs to Spark) to be autoscaled. If these jobs fetch data from managed databases, you might have RDS configured with autoscaling read replicas to support higher volume queries.
What I've frequently see happening: tasks fail because the task node instances were downscaled at the end of the job, because they are no longer consuming enough resources to stay up, but the tasks themselves haven't finished. Or tasks failed because database connections were suddenly cut off, since RDS read replicas were no longer transmitting enough data to stay up.
The workaround is to have a fixed number of instances up, and pay the costs you were trying to avoid in the first place.
Or you could have an autoscaling mechanism that is aware of your application state, which is what k8s enables.
Sleep obfuscation seems to be viable because scanners only execute periodically. I'm not very familiar with Windows internals, but why don't these scanners hook the VirtualProtect calls and only then scan the associated memory region? My understanding on using ROP is to make the calls seem to originate from trusted modules, but couldn't a kernel driver / hypervisor be able to detect all these calls regardless? Is it just too taxing on overall system performance or is there some other limitation?
x/w is a gdb command to dereference and print (x) 1 word-sized data (w) at the address given by the expression. So 0x08001ea1 was the unaligned address stored at $rsp + 8, passed as argument to __lll_lock_wait.
I wonder if MAME would work as an open-source alternative for these simulations. The built-in debugger and memory views cover most of that, and it doesn't take much to write a minimal driver that messes with some input file [1].
But if you do want some visual feedback like LEDs and such, you would have to write a layout and input logic for that (AFAIK has been done for MIDI synths).
It would also be more appropriate for shellcode / nostdlib experiments, although it's not clear how well CPUlator handles dynamic linking / relocations in their ELF executable support (seems like they build on their server [2], but run in a custom interpreter?)
Unfortunately, there's little curation of interesting debugging case studies or "debugmes". You can try to scavenge bug trackers, but they aren't optimized for this, just a bag of itches users want to scratch. Neither are war stories very fruitful, since rarely do you get to hunt down the corresponding bugfix commits, which are essential to not miss on the details that made those bugs tricky. But then, you already have the solution in front of you.
Instead, I've found much more deliberate practice and therefore value from reverse engineering: just like in debugging, you want to understand the underlying logic through program analysis.
You can pick security CTFs, crackmes, malware samples, proprietary software with bugs you want to fix, or functionalities you want to add, or even games where you want to find some hidden content, extract some resources, or modify some behaviors. In all that you will find something of your interest, and apply approaches that I think translate well to software development:
* Differential tracing: Want to know how some action reflects in the codebase? Take k instruction traces where you do everything except that action, then take a trace where you do that action, find out what are the unique differences in that last trace. Want to know what data gets written? Same approach with memory dumps and breakpoints. How do different inputs affect these changes? You will learn to be methodic and throughout in what you test and log.
* Recognizing patterns: Sometimes you don't have symbols for your functions, are you able to identify printf at a glance, or will you waste time following the logic of the function? Do you see relative offsets being used and recognize accesses to an array? With source code all this happens on a more macro view, such as algorithms or design patterns hidden in all those coupled functions and classes. But the micro view also applies: figure out what constants relate to specific functionalities, and you can grep your way to relevant functions or documentation.
* Avoiding boilerplate: This follows from the previous point, since you want to recognize the flow of data through the codebase, in order to have some call hierarchy to follow, otherwise it's easy to waste time on functionally that is irrelevant to you. Start with how data enters the application: stdin, files, database connections, http endpoints... Tests, examples, or client apps will also help here.
Oh and don't worry about learning some assembly language, just make that investment, since that's the straightforward, well defined, predictable part.
The implementation was not saving any of the ymm registers, you can see this in the bug report's attached patch. It also took me a while to get, because the last snippet in the writeup only highlights the ymm1 register, instead of showing the actual diff along with the comment.
As an aside, I didn't find the writeup to be overly descriptive. The author does a great job linking every step and how they came to each conclusion along the way, even including hypotheses where they came out empty-handed. When a writeup omits these, it makes the author look like they were somehow enlightened to find the solution, instead of just following a methodical approach.
Supposedly there's an easter egg in Windows CE's Solitaire [1] that has never managed to surface, beyond someone posting part of the credits text in a comment. No screenshots, videos, or any other evidence of its existence.
It's an exercise in frustration. Following the posted steps doesn't activate it in qemu or virtual pc (other vms don't seem to support Windows CE). Could be buggy virtualization, could be that it only exists in a particular version of Windows CE (judging from the date it was posted, maybe 2.11/3.0/HPC2000), or only in some OEM's custom ROM. Even if you wanted to dig into the ROM, strings are encoded in some LZ77 variant, so nothing greppable upfront. ROM dumping tools were made for extracting specific ROMs (you never know which ones and which offsets exactly), and are pretty much undocumented. Still plenty reverse engineering effort to be done.
Some examples:
* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.
* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.
* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.
* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.
* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.
So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.
When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.
[1]: https://qufb.gitlab.io/writeups/mysteries
[2]: https://github.com/qufb/PinoutDB
[3]: https://bsdl.info/index.htm