diff --git a/README.md b/README.md index 8edf32c..704b81a 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,12 @@ directly. than trusting what the manifest ships or what the device claims. - Working through **packed/hardened targets**: identifying the packer, unpacking, and turning a memory dump back into a patched, installable APK. +- Handling a **hardened library that terminates the process on purpose** — including the + deliberate-crash shape (`fault addr 0x4`) that looks exactly like an ordinary null-dereference + bug, and the "neutralise it, but never by making it *not return*" rule that decides whether the + fix works or freezes the whole app in a way that looks nothing like the cause. +- Knowing **which tools to reach for and where each one lies** — including the ones that only + exist as a GUI, so you ask for a human instead of silently substituting a weaker method. - Keeping a **long task honest**: a live record, graded conclusions, calibrated timeouts, and bounded waits, so progress is not lost and the same mistake is not made twice. - Avoiding the specific mistakes that produce an APK that builds perfectly and dies at @@ -50,6 +56,12 @@ references/ loaded on demand, one topic each native-and-so.md .so hosts, DT_NEEDED vs JNI_OnLoad, relocation limits, relocation-free bootstrapping, replacing Java methods natively, and which ABI/library is *actually loaded and executing* + native-tamper-and-suicide.md how a hardened library kills its own process: the visible + mechanisms, how to tell which one actually fires, how to find the + site, forged section headers, function boundaries from + PT_GNU_EH_FRAME, scanner traps, and neutralising safely + toolchain.md what to install, how to invoke it non-interactively, which tools + are GUI-only, version-alignment traps, working offline long-task-discipline.md live record, conclusion grading, drift control, timeout and wait calibration, deliverable-form drift, handover ad-removal.md ad taxonomy, wrapper mapping, callback trap, global gates, verification @@ -100,6 +112,14 @@ scripts/ parameterized, path-agnostic lib_map.py what is *actually mapped* into a live process: per-library path, base, architecture, and whether it came from the APK or was materialized at runtime + elf_plt.py resolve a PLT stub to its imported symbol (x86_64 + aarch64) from + the relocation table; list a symbol's callers; byte-diff two + libraries and name the symbol each changed stub belongs to + apk_diff.py entry-level diff of two builds: changed / added / removed, by + content hash so same-size replacements are caught + native_crash.py locate a native death from a log or tombstone: signal, fault + address, registers, frames split app vs system, the faulting + instruction, and a flag when the fault looks *arranged* blob_decode.py search, don't guess, the framing of a stored value (base64/hex x rotation x deflate); re-encode the edited payload snap.py bounded burst screenshots + control-tree capture with a stall @@ -139,7 +159,7 @@ Nothing is mandatory; each script checks what it needs. `references/pitfalls.md`. It is the most valuable file here — every entry is a failure that produced a broken artifact while looking completely healthy. -The three that hurt most: +The four that hurt most: 1. Stripping the whole `META-INF/` during a repack deletes ServiceLoader registrations and the app dies at startup with an error that names an unrelated library. @@ -147,6 +167,9 @@ The three that hurt most: dex rejected, while checksums and signatures verify perfectly. 3. Rebuilding a dex with a whole-tree smali round-trip damages R8 output invisibly — class tables compare clean, and it only blows up at runtime. +4. Neutralising a native terminate path by making it **not return**. A spinning stub does not + suppress the check; it freezes the caller and every thread behind it. The app hangs with *no + crash record at all*, and the eventual death gets blamed on whatever killed the frozen process. ## Scope diff --git a/SKILL.md b/SKILL.md index 3e8c7ba..f8cd8a0 100644 --- a/SKILL.md +++ b/SKILL.md @@ -57,10 +57,20 @@ Answer these before touching a tool. Every one of them changes the whole plan. routine, the rebuilt APK must hardcode the *original* certificate value at every read site, or every signed request fails while the app still launches and looks healthy. This is the single most expensive silent failure in a repack, and 15 minutes of grep prevents it. +11. **Does the app die on its own after a while — with no Java stack trace, or with a native + crash that looks like a bug?** + → `references/native-tamper-and-suicide.md`. A hardened library that decides the build is + tampered rarely calls `kill`. It more often **arranges a fault** (load a small constant, use it + as a pointer) so the death looks like an ordinary defect, and the system then reports it as an + app "crash" or "abnormal" dialog. Two rules before you touch anything: **enumerate which + mechanism actually fires** (the signal and the tombstone split them apart), and **neutralise by + returning, never by making it not return** — a spinning stub freezes the process and produces a + symptom that looks nothing like the cause. ## The workflow, end to end 1. **Preflight, then Recon** — `scripts/preflight.py` before anything else if a device is involved (it takes seconds and prevents a whole class of false conclusions), then `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours. **If it is packed, unpack before anything else** (`references/recon.md` §unpacking): you cannot patch code you cannot read, the encrypted payload lengths tell you which dumped dex is the original, and a memory dump must be de-duplicated by hash and structurally validated before any of it is trusted. + **If the app already dies on its own** — especially at a roughly constant time after launch, or with a native crash — locate the mechanism *before* planning any patch (`references/native-tamper-and-suicide.md`, `scripts/native_crash.py`). Record the observed time-to-death: it is the baseline every later attempt is measured against, and without it a surviving run cannot be told from a changed schedule. 2. **Extract strings and endpoints** — build a picture of the app's API surface and SDK inventory from the dex string tables. No decompiler needed for this, and it is fast. Scripts: `scripts/dex_strings.py`. 3. **Trace to the owning class** — find the class that wraps the behavior (the app almost always wraps third-party SDKs in one helper). Reverse-lookup instructions: `references/dex-patching.md` §finding-the-call-site. 4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`. @@ -76,6 +86,7 @@ Answer these before touching a tool. Every one of them changes the whole plan. - **Verify structure after every dex edit.** `scripts/dex_classdiff.py` must report zero differences in class set and access flags for classes you did not intend to change. - **Do not patch a method that is widely shared.** Before patching any helper, count its callers (`scripts/find_refs.py`). A `Long.valueOf` wrapper with 30 callers is not an ad-specific hook. - **Do not make an API fail to suppress a UI element.** A 404/400 on an endpoint that other features depend on takes the whole screen down with it. Suppress at the data-consumption or render layer instead. +- **Neutralise a native terminate path by returning, never by making it not return.** A stub, stub patch, or function entry replaced with a spin or a self-branch does not suppress the check — it freezes the caller, holding whatever lock it had, and unrelated threads wedge behind it. The symptom (a hang, an external kill, a restart loop) looks nothing like the cause, and there is no crash record to explain it. Return a benign value, and prefer success (0) over failure (-1). Never touch the ordinary-path symbols (`pthread_exit`, `exit`, `abort`, `snprintf`, `closedir`). `references/native-tamper-and-suicide.md` - **Look before you conclude — and look while you wait.** Execute, capture, and **inspect**; do not drive and sleep blind. A screen that is actually looked at answers in one step what coordinate-guessing cannot answer in five: the layout moved, a different dialog is up, a countdown is frozen, the text on screen says exactly why. Where the thing you are waiting on is visible, a sample you can inspect beats a duration you hoped was right, and byte-identical samples mean nothing is going to change. `scripts/snap.py`; `references/environment.md` §look at the screen. - **Put a timeout on every command, and calibrate it from measurement.** An unbounded call turns a stall into "the task stopped making progress", which is indistinguishable from slow work and costs hours silently. Time the operation once, record it, then derive the bound from it — that is what makes slow and hung distinguishable. A deadline that passes is a measurement, not a verdict. `references/long-task-discipline.md` §bound every wait. - **Every claim needs evidence.** "Probably", "should be", "in theory" are not findings. Either you observed it, or you label it unverified. @@ -95,6 +106,8 @@ Load only what the current step needs. | `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works | | `references/dart-aot.md` | The logic lives in a Dart AOT snapshot (`libapp.so`): pinning the Dart version, building a matching decompiler, the object pool and reference indexing, register/boolean conventions, locating and patching Dart code | | `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** | +| `references/native-tamper-and-suicide.md` | The process dies on its own (no Java stack, or a native crash that looks like a bug); you are about to neutralise a `kill`/`exit`/`abort` path; or a hardened library's sections/function boundaries look wrong | +| `references/toolchain.md` | Choosing or invoking tools, something is not installed, a tool's output smells wrong, or you need to know which tools exist only as a GUI | | `references/ad-removal.md` | Task involves ads, trackers, sponsored cards, splash/interstitial/reward | | `references/membership-and-limits.md` | Task involves VIP, subscription, paid content, unlock, "fully cracked" | | `references/server-api.md` | The behavior is decided by a response, or you need to know if a patch can even matter | @@ -138,6 +151,9 @@ All scripts are parameterized and path-agnostic; pass paths explicitly. Run `--h | `scripts/tls_check.py` | Strict certificate check for one or more hosts (expired / wrong host / untrusted CA) | | `scripts/preflight.py` | Read-only environment check before every experiment block: device, root, ABI/translation, clock skew, leftover proxy/forwards, dead device server. Run this before blaming a patch. | | `scripts/lib_map.py` | What is **actually mapped** into a live process: per-library path, base, architecture (`ELF e_machine`), and classification (system / from-APK / runtime-materialized). Answers "which library and which ABI is really executing". | +| `scripts/elf_plt.py` | Resolve a PLT stub to its imported symbol on x86_64 and aarch64 (from the **relocation table**, not from position or a comment), list a symbol's callers, and **byte-diff two libraries naming the symbol each changed stub belongs to**. Run this before patching any stub, and to audit a patch set you inherited. | +| `scripts/apk_diff.py` | Entry-level diff of two APKs: what changed, what was **added** (injection candidates), what was removed — by content hash, so same-size replacements are caught. Use it to audit a third-party build and to prove your own build was surgical. | +| `scripts/native_crash.py` | Locate a native death from a logcat capture or tombstone: signal, fault address, register state, backtrace split into your libraries vs system, the faulting instruction — plus an explicit flag when the fault looks **arranged** rather than accidental. | | `scripts/blob_decode.py` | Decode an opaque stored value by searching the parameter space (base64/base64url/hex × rotation × deflate/zlib/gzip) instead of guessing, then re-encode an edited payload with the same parameters. | | `scripts/snap.py` | Bounded burst screenshot + control-tree capture, with a stall detector and an explicit verdict on whether the accessibility tree is usable at all. Use it so you *look* at the screen instead of driving blind. | | `scripts/sig_probe.py` | Find the exact `signatures[0].toCharsString()` value: offline candidate enumeration from an APK (`--apk`), or the authoritative value read from a live package (`--live`). Feed the result into the hardcoded constant described in `references/signature-derived-keys.md`. | diff --git a/references/native-and-so.md b/references/native-and-so.md index 6413ab2..c93e7ca 100644 --- a/references/native-and-so.md +++ b/references/native-and-so.md @@ -3,6 +3,10 @@ Load this when the Java/dex layer is blocked or unsuitable, when you need code to run **before** the app's own code, or when a native library is the only editable place left. +If instead the problem is that the app **dies on its own** — hangs, restarts, or crashes natively at a +roughly constant time after launch — that is a terminate mechanism, not a hooking problem, and the +methods differ: read `native-tamper-and-suicide.md` before this file. + The Java layer is usually the right choice. Go native when you need one of these: - execution **earlier than any app code** (before `Application.onCreate`, before static initialisers), diff --git a/references/native-tamper-and-suicide.md b/references/native-tamper-and-suicide.md new file mode 100644 index 0000000..d079ae1 --- /dev/null +++ b/references/native-tamper-and-suicide.md @@ -0,0 +1,264 @@ +# Native Tamper Response — When the Library Kills Its Own Process + +Load this when the process dies **without a Java stack trace**, when a build that passed every +static check crashes seconds after launch, or when you are about to neutralise a native check. + +This file exists because of one specific, expensive mistake. Read §The rule before you edit a +single byte of a shell's native library. + +## The rule + +> **Never neutralise a terminate path by making it not return. Make it return.** + +A terminate routine is reached from ordinary code paths. If you replace it with something that +**never returns** — an infinite loop, a self-branch, a stub that spins — the caller never resumes, +locks are never released, and every unrelated thread that touches those locks wedges. You have not +suppressed the check; you have frozen the process. + +The distinction decides the outcome: + +| Neutralisation | Effect on the caller | Result | +|---|---|---| +| `ret` / return a benign value | continues normally | check suppressed, process healthy | +| spin / self-branch / `while(1)` | never resumes | process freezes, watchdog or user kills it, symptom looks unrelated | + +**Two failure shapes this produces, both of which have been misread as "the patch did not work":** + +- The app hangs with no crash record at all, then disappears. Logs show nothing, because nothing + crashed — it stopped. +- The watchdog or a system supervisor kills the frozen process, producing + `Force stopping … from uid 0` or an app-restart loop. The uid-0 killer is a strong tell: an app + cannot spawn a root-owned killer, so the executioner is **outside** the app, which means the app + froze rather than failed. + +Both are self-inflicted and both are avoided by returning instead of spinning. + +### The corollary: do not touch normal-path symbols + +A hardening library imports a lot of libc. Some of those symbols are terminate paths; most are not. + +**Symbols that must stay untouched** — they are on ordinary code paths and freezing them breaks +everything: + +`pthread_exit` · `exit` · `abort` · `android_set_abort_message` · `snprintf` · `closedir` · +`__cxa_atexit` · `__stack_chk_fail` + +A concrete case: a set of hand-made patches rewired five PLT stubs to self-branches on one +architecture and a different five on another. On the first architecture two of the five were +`snprintf` and `closedir` — a string formatter and a directory-close — so the app froze on the +first log line. On the second the five included `pthread_exit`, so every thread that finished +wedged. The patches were meant to suppress a suicide check and instead destroyed the process. + +**Before touching a stub, resolve what symbol it belongs to.** Do not infer it from the stub's +position or from a comment. See §Resolving a stub to its symbol. + +## How a hardened library terminates the process + +Enumerate these before you patch anything. Each has a different signature and a different remedy. + +| Mechanism | Log signature | Has tombstone? | How to neutralise | +|---|---|---|---| +| `kill(getpid(), SIGKILL)` | `Process N exited due to signal 9 (Killed)`, **no** exit code, **no** tombstone | no | patch the call site or make the `kill` stub return 0 | +| `exit()` / `_exit()` | process ends with an exit code, no signal | no | find the caller; usually one branch of a check | +| `abort()` | `signal 6 (SIGABRT)` + tombstone | yes | usually a genuine assertion — find what tripped it | +| **Deliberate crash** (see §Deliberate-crash stubs) | `signal 11 (SIGSEGV)`, tiny `fault addr` such as `0x4`, `Cause: null pointer dereference` | yes | nop the faulting store | +| `tgkill` / `raise` | same as kill/abort | varies | often not imported at all — check the import table before assuming | + +**Distinguish these before patching.** "The app crashed" is not a diagnosis. The signal number and +the presence or absence of a tombstone split the space in half; getting this wrong sends you to the +wrong layer for hours. + +### Deliberate-crash stubs + +The most deceptive mechanism: the library does not call anything. It **arranges a fault** so the +death looks like an ordinary bug. + +```asm +; arm64 shape observed in the wild +mov x0, #4 ; load a small constant… +mov w1, #1 +str w1, [x0] ; …and use it as a pointer. fault addr = 0x4 +``` + +Runtime shape: `SIGSEGV`, `SEGV_MAPERR`, `fault addr 0x4` (or `0x0`/`0x8`), `Cause: null pointer +dereference`, and — critically — **the register holding that small constant value**. + +How to recognise it rather than treating it as a real bug: + +1. The faulting address is a **small integer**, not a plausible pointer. +2. Disassembling backwards from the faulting `pc` shows the constant loaded **a few instructions + earlier, in the same basic block**, with nothing in between that could produce a real pointer. +3. The block sits immediately before a function epilogue (canary check + `ret`), i.e. the normal + exit is right there and this store is bolted on in front of it. +4. There is often a **delay loop just above it** — `sleep`/`usleep` repeated N times — because this + is a *watchdog*: wait a while, then die if the condition still holds. That delay is why the crash + is always "a little while after launch" rather than immediate. + +**Neutralise it by nop-ing the faulting store**, leaving the surrounding arithmetic alone. The +thread then falls through into the epilogue and returns normally. Do not remove the whole block: +the two `mov`s are harmless and keeping them makes the diff minimal and reviewable. + +**This is why patching `kill` alone is not enough.** A deliberate crash uses no imported symbol at +all, so every PLT-level fix — including a correctly implemented one — leaves it untouched. + +### Find every instance, not the first one + +Search the binary for the **byte pattern**, not with a linear disassembler. The exact encoding is +architecture-specific and must be derived from the crash you already have, then searched as bytes: + +- Take the faulting `pc`, read the instruction bytes backwards until you have the full constant-then- + store sequence. +- Search the whole file for that byte sequence. +- If the search returns more than one hit, patch each; if it returns exactly one, you have the + complete set — record that fact, it is a real result. + +A linear disassembler is the wrong tool here for a reason covered in §Scanner traps. + +## Resolving a stub to its symbol + +When a stub's target is a PLT entry, the symbol comes from the relocation table, **not** from the +stub's bytes or its neighbours. Two architectures encode this differently: + +- **x86_64** — the stub is `jmp qword ptr [rip+disp32]` (6 bytes). The target address is + `stub_addr + 6 + disp32`; look that address up in the PLT relocation map. +- **aarch64** — the stub is **four instructions / 16 bytes** (`adrp` → `ldr` → `add` → `br`). The + GOT address is `page(adrp) + add_imm`. Do not assume a 4-byte stub: a 16-byte stub leaves room + for a **two-instruction replacement** (`mov x0, #0` + `ret`), which is exactly what you need, and + assuming 4 bytes pushes you into borrowing the neighbouring slot — which belongs to a different + symbol and breaks it. + +`scripts/elf_plt.py` does both architectures, including the relocation lookup. + +**Never splice in a neighbouring stub's bytes.** Stubs are laid out back to back; the next one +along is a different function, and corrupting it produces a second, unrelated failure that you will +then spend hours attributing. + +## Patching a terminate path correctly + +Work in this order. + +1. **Confirm the mechanism first.** Get the tombstone or the signal number. Do not patch on a + guess about *how* it dies. +2. **Decide between call-site and stub.** Call sites are precise but you must find all of them. + A stub covers every caller at once, including ones your scan missed — but it changes behaviour + for every symbol user, so it is only safe when that symbol means "terminate" and nothing else. +3. **For a stub, return success, not failure.** Callers commonly inspect the return value. Returning + **0** (success) lets the caller take its "already handled" branch; returning **-1** pushes it + into an error path that may try a different termination method. Make the stub *succeed*. +4. **Preserve every byte you did not intend to change.** Record the original bytes so the patch is + reversible and auditable. +5. **Re-verify the file after patching.** Length unchanged, only the intended offsets differ, and + the normal-path symbols are byte-identical to the original. `scripts/elf_plt.py --diff` prints + exactly that. + +### What "faithful suppression" looks like + +A library that decides "tampered" and then cannot terminate has an inconsistent state — it believes +it has killed the process and continues. That is **acceptable and usually harmless**, because the +terminate path is the end of the check; nothing downstream re-reads "did I die". But verify it +rather than assuming: after a successful suppression the app must reach its **normal UI and stay +there**, not loop through half-initialised states. + +## ELF hardening you will meet in these targets + +Hardening libraries are not normal ELF files. Assume the following until measured otherwise. + +### Forged section headers + +A shell may ship a library whose section header table is deliberately wrong — `.text` sized to a +handful of bytes, `.dynsym` truncated, sections overlapping. Tools that walk sections therefore +produce **confidently wrong** answers (a symbol list with one entry, an import table that is empty). + +**Work from the program headers instead.** `PT_LOAD` gives you the real mapped segments and their +permissions; `PT_DYNAMIC` gives you the pointers you need, walked by hand: + +``` +DT_STRTAB(5) DT_SYMTAB(6) DT_STRSZ(10) DT_SYMENT(11) +DT_JMPREL(23) DT_PLTRELSZ(2) DT_PLTREL(20) DT_PLTGOT(3) +``` + +Then translate any virtual address to a file offset through the `PT_LOAD` that contains it. A +library with forged sections that yields a full import list this way is a good sign; one that +still yields almost nothing means the hardening is deeper and you should say so rather than +reporting the truncated result as the truth. + +### Function boundaries come from `PT_GNU_EH_FRAME`, not from prologue guessing + +You will need "where does this function start". Two bad answers and one good one: + +- **Reverse-decoding backwards from a call site does not work on aarch64.** Almost any 4-byte + window decodes as *some* instruction, so a naive scanner reports hundreds of "function starts" + that are all fiction. +- **Guessing at prologues** (`sub sp, sp, #imm` + `stp`) produces a plausible set with no guarantee + of completeness. +- **`PT_GNU_EH_FRAME` is authoritative.** If the segment survives (it usually does, because + unwinding is needed at runtime), parsing it yields the real function entry table — every entry, + exactly. It is several lines of code and it removes an entire class of wrong conclusions. + +**Check for it first.** If it is present, use it; if it is absent, say your boundary list is +heuristic. + +### A function's entry is not where its logic is + +Callers reach a routine through several routes: a direct branch, a cached table slot, or a function +pointer in writable data. Rewriting the entry point only covers the first. If a call resolves +through a cached pointer, your entry-point change is invisible to it — so before patching an entry, +scan for the other routes (`scripts/elf_plt.py`, or a byte-pattern scan for the address). + +## Scanner traps + +**A linear disassembler can stop silently.** Given a buffer whose start is not code, a decoder may +return a handful of instructions and then nothing, with no error. Concluding "there are no such +call sites in this library" from that output is a false negative with real consequences. + +Guards, in order of preference: + +1. **Byte-pattern search** for a sequence you already know (from a crash or a known call site). + Fast, exhaustive, no decoder involved. +2. **Fixed-bit-pattern scans** for instruction classes whose encoding is regular — the branch + instructions are the useful ones here (`BL`/`B` on aarch64, `call rel32` on x86_64). This finds + every call site without depending on linear decoding. +3. **Resynchronising scan** — decode, and on failure advance by one instruction unit (4 bytes on + fixed-width aarch64, 1 byte on x86_64) and retry. Slower, but it does not stop early. + +Whichever you use, **state which one you used and what it can miss**. "No hits" from a scan whose +coverage you cannot describe is not evidence. + +## Deciding whether a dynamic-resolution path matters + +Hardening libraries frequently build a table of function pointers with `dlopen` + `dlsym` — often +40+ symbols including terminate calls. This raises an obvious worry: *a call through that table +bypasses the PLT stub, so my stub patch does nothing.* + +**Measure it instead of assuming either way.** For each symbol of interest, find whether its table +slot is ever **read**: + +- Locate the slot's address (the store from the `dlsym` return, or the table base + index). +- Search the code for instructions that load from that address (`adrp`+`add`+`ldr` on aarch64). +- **Zero readers means the entry is written and never used.** The table is a redundant fast path, + not a bypass, and PLT-level patching is complete. + +Real outcome from a hardened target: the `kill` slot had **zero** readers while the `exit` slot had +exactly one, inside a routine that was already suppressed — so no dynamic-resolution bypass existed +at all, and the expensive "patch `dlsym` itself" plan would have broken 40 unrelated symbols for +nothing. + +**Do not blanket-patch a symbol resolver.** It feeds `open`, `read`, `mmap`, `dlopen` and more. +Changing its behaviour to reach one terminate symbol is a large blast radius for a problem you may +not have. + +## Verification + +A native suppression earns no credit until: + +1. The process **survives the window in which it used to die** — and that window must be measured, + not guessed. If the observed death was ~40 s after launch, a 30-second test proves nothing. +2. It then **keeps surviving** through real interaction, not just idling. +3. `crash`/`tombstone` buffers are clean, **and** the process id never changed (a restart with a new + pid is a death even when nothing logged a crash). +4. The normal-path symbols are byte-identical to the original build. +5. A **control build** — same pipeline, no native patch — still dies the same way. Without the + control you cannot tell suppression from an unrelated change in timing. + +**Record the observed time-to-death before patching.** It is the only thing that later tells you +whether your patch worked or merely moved the window. diff --git a/references/packers.md b/references/packers.md index 2dc34f1..1b4ffb9 100644 --- a/references/packers.md +++ b/references/packers.md @@ -134,6 +134,13 @@ The cheapest high-value experiment, and one that is routinely skipped: run the * build** and the **repackaged build** through the *same* probe and diff what differs. The difference is your detection signal. Doing this early is far cheaper than enumerating candidate checks one by one. +**When the symptom is a death rather than a dialog** — the process disappears, hangs, or dies at a +roughly constant time after launch — the check is not drawing anything, it is terminating. The +question then becomes *which mechanism*, and the answer decides the whole approach: +`native-tamper-and-suicide.md` covers how to tell an imported terminate call from an **arranged +fault** (which calls nothing and therefore defeats every PLT-level fix), how to find the site, and +why making a routine "not return" freezes the process instead of suppressing the check. + ## Reporting State the boundary you measured, the evidence for each entry, and what remains untested. "The shell diff --git a/references/pitfalls.md b/references/pitfalls.md index bfe9255..cb2df46 100644 --- a/references/pitfalls.md +++ b/references/pitfalls.md @@ -794,3 +794,236 @@ Two close variants of the same error: *change*, never *what* changed — and it never proves *absence*. Look at the images: a status-bar clock tick or a line of text reflowing changes the hash while a full-screen overlay would not have been missed if the frames had actually been inspected. + +--- + +## P31. Neutralising a terminate path by making it "not return" freezes the whole process + +**Symptom** + +The app hangs with **no crash record at all**, then disappears. Or an external process kills it — +`Force stopping … from uid 0`, an app-restart loop — and nothing in the log says "crash". + +**Root cause** + +A terminate routine (a shell's `kill`/`exit`/`abort` stub, or a self-terminating function entry) +was replaced with something that **never returns**: a self-branch, an infinite loop, a spinning +stub. The caller was written expecting the process to be gone. Instead control never comes back, +whatever lock it held is never released, and unrelated threads wedge behind it. + +**Why it is hard to see** + +There is no exception, no signal, and no tombstone, because nothing failed — it stopped. Absence of +a crash record reads as "it is still fine", and the eventual death is attributed to whatever the +killer happened to be. The tell is a **uid-0 killer**: an app cannot spawn a root-owned executioner, +so the executioner is outside the app, which means the app froze rather than died. + +**Do instead** + +**Return. Always return.** A `ret`, or a stub that loads 0 and returns. + +- Callers commonly inspect the return value, so make the suppressed call **succeed (0)** rather + than fail (-1) — failure can push the caller into an error branch that tries a *different* way + to terminate. +- Do not touch the ordinary-path symbols: `pthread_exit`, `exit`, `abort`, `snprintf`, `closedir`, + `android_set_abort_message`. Freezing `pthread_exit` wedges every thread that finishes; freezing + `snprintf` wedges the first log line. A five-stub patch intended for terminate symbols has hit + exactly those symbols before. +- If the fix is a delay-loop watchdog, the *faulting store*, not the loop, is the thing to remove. + +Full treatment, including how to tell which mechanism is actually firing: +`native-tamper-and-suicide.md`. + +--- + +## P32. Rewiring a stub without resolving which symbol it belongs to + +**Symptom** + +A patch meant to suppress one check breaks something entirely unrelated — often before that check +would even have run. + +**Root cause** + +PLT stubs are laid out back to back. A patch recipe that names offsets, or that assumes a fixed +stub width, is one arithmetic slip away from rewriting its neighbour. Real outcome of one such set: +five stubs intended to be terminate symbols resolved to **five different symbols on each +architecture**, including a string formatter and a directory call — so the app froze on its first +log line instead of suppressing anything. + +**Why it is hard to see** + +The offsets and the symbol names usually come from different sources (a note, a previous round, a +generated table) and nothing cross-checks them. The damage then presents as an unrelated stability +problem, and the real cause is two rounds back. + +**Do instead** + +Resolve every stub to its symbol **before** writing the patch, from the **relocation table** — +never from a comment, a position, or an assumed width. + +`scripts/elf_plt.py` prints `stub address -> symbol` for both x86_64 and aarch64, and +`--diff --name-regions` names the symbol each changed stub belongs to, which is the right way to +audit a patch set you inherited. + +**The aarch64 stub is 16 bytes (four instructions), not four.** Assuming the short form pushes you +into borrowing the next slot, which belongs to a different symbol — and a 16-byte stub is exactly +what makes a clean two-instruction replacement (`mov x0, #0; ret`) possible without touching +anything else. + +--- + +## P33. "The scan found no call sites" — from a decoder that stopped early + +**Symptom** + +A scan for call sites, stubs, or a byte pattern reports zero matches, and that zero becomes a +finding: "this library never calls `kill`", "there is no second site", "the payload is absent". + +**Root cause** + +A linear disassembler handed a buffer that does not start on an instruction boundary can return a +few instructions and then **stop, silently**. No error, no warning — and its partial output is +indistinguishable from a complete negative. On a fixed-width architecture, decoding from the wrong +offset also produces plausible garbage rather than failing. + +**Why it is hard to see** + +Zero is a comfortable answer. It usually agrees with what you were hoping ("the check is not +there"), so nothing prompts a second look. (See also P25 — same shape, different mechanism.) + +**Do instead** + +Never conclude absence from a scan whose coverage you cannot describe. + +- Prefer **byte-pattern search** for a sequence you already know, taken from a crash or a known + call site. +- For instruction classes with regular encoding, use a **bit-pattern scan** — branch instructions + are the useful ones, and this finds every occurrence without depending on linear decoding. +- Otherwise use a **resynchronising** scan (advance one instruction unit — 4 bytes on aarch64, + 1 on x86_64 — and retry). +- State which method you used and what it can miss. + +--- + +## P34. A deliberate crash read as an ordinary bug + +**Symptom** + +`SIGSEGV`, a real tombstone, `fault addr 0x4` (or `0x0`/`0x8`), `Cause: null pointer dereference`. +It looks like a plain null-dereference defect, so you go looking for the defect. + +**Root cause** + +There is no defect. A hardening layer that wants the process dead **without calling anything it +imports** arranges a fault — load a small constant, use it as a pointer: + +``` +mov x0, #4 +mov w1, #1 +str w1, [x0] ; fault addr = 0x4 +``` + +**Why it is hard to see** + +The tombstone is genuine, the signal is genuine, and `null pointer dereference` is the runtime's +honest description. Nothing distinguishes it from a real bug except the *shape* of the fault — and +"it crashes on a small address" reads like sloppy target code, which is exactly what it is +imitating. + +**Do instead** + +Read the fault address and the registers **together**: + +- Is the fault address a small integer rather than a plausible pointer? +- Does some register hold exactly that value? +- Does the instruction at the faulting `pc` load that constant a few instructions earlier, in the + same basic block? +- Is there a delay loop immediately above it (`sleep` repeated N times)? That is a watchdog, and it + explains why the death always comes "a little while after launch" rather than at once. +- Does the block sit right before a normal epilogue (canary check + `ret`)? The intended exit is + right there. + +If all of that holds, **nop the faulting store** and leave the surrounding arithmetic alone — the +thread then falls through into the epilogue and returns normally. + +`scripts/native_crash.py` extracts the frames, registers and faulting instruction, and flags this +shape explicitly. + +--- + +## P35. Blocking one termination mechanism and calling the check suppressed + +**Symptom** + +The `kill` stub now returns success, the import table is clean, the patch is verifiably present — +and the app still dies, with the same signal at the same time. + +**Root cause** + +A hardening library terminates through **several independent mechanisms that share no choke +point**: an imported `kill`, an imported `exit`, `abort()`, and a **deliberate crash that calls +nothing at all** (P34). A PLT-level fix covers the first two and has literally no effect on the +last one, because no imported symbol is involved. + +**Why it is hard to see** + +"It imports `kill`, so `kill` is how it dies" is a satisfying and often correct-sounding story. A +*correct* patch to a *real* mechanism produces no visible change when a second mechanism fires +first — so the conclusion drawn is "the patch did not work / the approach is wrong", and a working +route gets discarded. + +**Do instead** + +Enumerate the mechanisms before patching, then **measure which one actually fires**. The signal +splits the space, and the tombstone confirms it: + +| Observation | Mechanism | +|---|---| +| `SIGKILL`, no exit code, **no tombstone** | an imported terminate call | +| `SIGSEGV`, small fault address | arranged crash — no imported symbol involved | +| `SIGABRT` + tombstone | usually a genuine assertion | +| process survives a repack but the library is absent from `maps` | the patch never ran | + +**Then verify against the observed time-to-death.** If it died ~40 s after launch before, a +30-second test proves nothing, and a run that survives 45 s has not yet passed. + +--- + +## P36. Parsing a hardened ELF through its section headers + +**Symptom** + +A tool reports that a library imports one symbol, or has a 200-byte `.text`, or contains no +functions. You build a plan on that. + +**Root cause** + +Hardened libraries ship **forged section headers**: `.text` sized to a token value, `.dynsym` +truncated, sections overlapping. Anything that walks the section table returns confidently wrong +output — and because the output is well-formed, the wrongness is invisible. + +**Why it is hard to see** + +A truncated result and a genuinely minimal library look identical. The tool has no way to report +that it was lied to, so the failure is attributed to the target rather than the method. + +**Do instead** + +Work from the **program headers**, which the loader itself uses and which therefore cannot lie +about what gets mapped: + +- `PT_LOAD` → the real segments, their file offsets and their permissions. +- `PT_DYNAMIC` → `DT_STRTAB` / `DT_SYMTAB` / `DT_STRSZ` / `DT_SYMENT` / `DT_JMPREL` / + `DT_PLTRELSZ`, walked by hand. +- Translate any virtual address to a file offset through the containing `PT_LOAD`. + +A library that yields a full import list this way is fine. One that still yields almost nothing +means the hardening is deeper than the section table — **say that**, rather than reporting the +truncated answer as a finding. + +For **function boundaries**, use `PT_GNU_EH_FRAME` when it survived: it is authoritative and +complete, and it costs a few lines to parse. Do not reverse-decode backwards looking for a prologue +— on aarch64 almost any 4-byte window decodes as *something*, so a naive scanner reports hundreds +of fictional entry points — and prologue pattern-matching yields a plausible set with no +completeness guarantee. diff --git a/references/toolchain.md b/references/toolchain.md new file mode 100644 index 0000000..447863f --- /dev/null +++ b/references/toolchain.md @@ -0,0 +1,136 @@ +# Toolchain — What to Use, How to Invoke It, and Where It Lies + +Load this when you are choosing tools, when a tool produces an answer that smells wrong, or when +something is not installed. It is a map, not a tutorial: each entry says what the tool is *for*, how +to drive it non-interactively, and the failure mode that wastes time. + +**Prefer tools you can drive from a command line.** An agent cannot click. Tools that exist only as +a GUI are listed where they matter, marked as requiring a human, so you can ask for that instead of +stalling or silently substituting a weaker method. + +## Tier 0 — present on almost any machine, no install + +Use these before reaching for anything heavier. They answer structure questions in seconds and +cannot be broken by a missing dependency. + +| Tool | Use it for | +|---|---| +| `file`, `readelf -h/-l/-d/-r`, `objdump -d`, `nm -D`, `strings` | ELF structure, program headers, dynamic section, relocations, dynamic symbols | +| `unzip -l`, `zipinfo` | what is actually inside an APK, entry sizes and compression | +| `sha256sum` / `Get-FileHash` | identity. **Always hash before and after; never trust "it should be the same file"** | +| `xxd` / `hexdump` | byte-level ground truth when a tool disagrees with another tool | +| `python3` | all the scripts here; the standard library alone covers most parsing | + +`readelf`/`objdump` walk **section** headers. If a target's section table is forged (see +`native-tamper-and-suicide.md` §Forged section headers) they will print confidently wrong output — +cross-check with the program headers. + +## Tier 1 — dex and Java + +| Tool | Invoke | Why this one | +|---|---|---| +| `baksmali` / `smali` (jars) | `java -cp org.jf.baksmali.Main d -o ` | round-tripping, reading a method precisely | +| `dexlib2` | small Java program | **the preferred patcher** — method-level rewrite, leaves everything else untouched (`dex-patching.md`) | +| `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources | +| `jadx` | `jadx --no-res -d ` | readable Java for orientation; **not** a source of truth (see below) | +| `aapt2` | `aapt2 dump badging ` | manifest facts, package name, versions | +| `zipalign`, `apksigner` | from build-tools | alignment and signing | + +**Classpath gotcha.** `baksmali`/`smali` need their dependency jars on the classpath together +(`smali`, `antlr-runtime`, `stringtemplate`, `baksmali`, `dexlib2`, `util`, `jcommander`, `guava`). +Missing one produces `ClassNotFoundException` that reads like a broken target. `scripts/smtool.py` +carries the classpath so you pass it once. + +**Entry-point gotcha.** The main class has moved between versions — `org.jf.*` in older builds, +`com.android.tools.smali.*` in newer ones. If `ClassNotFoundException` names the **main class**, it +is a version mismatch, not a missing jar. + +**jadx is for reading, not for concluding.** Decompiled Java reorders and rewrites control flow; +line numbers and even which branch is which can differ from the bytecode. When a decision depends on +it, verify in smali or at runtime. `jadx` writing `(unknown)`/empty method bodies means it failed, +not that the method is empty. + +**`smali` assembling can abort silently** — the tree produces no dex while the process still exits +0. Never treat exit code as proof of a build. Assert the artifact exists, is fresh, and is non-empty +(`patch-audit.md`). + +## Tier 2 — native + +| Tool | Invoke | Notes | +|---|---|---| +| `radare2` / `rizin` | `r2 -q -c '' file` | scriptable disassembly/analysis; the practical choice when no GUI is available | +| `Ghidra` (headless) | `analyzeHeadless -import -postScript ` | decompiler without a GUI; slower to script but far better output than a raw disassembler | +| `capstone` (python) | library | decoding in your own scripts — see the silent-stop trap below | +| `keystone` (python) | library | assembling a short patch when you are editing bytes by hand | +| `pyelftools` (python) | library | **section-based — unreliable on hardened targets.** Prefer hand-walking `PT_LOAD`/`PT_DYNAMIC`, as `scripts/elf_plt.py` does | +| IDA Pro | GUI | human-driven. Ask for it when decompiler output is genuinely needed; do not pretend a raw disassembly is equivalent | +| `x64dbg` | GUI, Windows | human-driven live debugging of a native Windows target | +| `gdb` / `lldb` | CLI | live debugging where a device or emulator allows it | + +**capstone can stop silently.** Decoding a buffer that does not begin at an instruction boundary may +return a few instructions and then nothing, with no error. A scan built on that reports "no matches" +for a library full of matches. Use byte-pattern search or a resynchronising scan for anything where +completeness matters (`native-tamper-and-suicide.md` §Scanner traps). + +**Disassembler output is a hypothesis.** Fixed-width architectures (aarch64) decode almost any +4-byte window into *some* instruction, so a wrong start offset yields plausible-looking garbage. +Bound your window with a known entry point or a known call site. + +## Tier 3 — runtime and network + +| Tool | Invoke | Notes | +|---|---|---| +| `frida` (host) + `frida-server` (device) | `frida -U -f -l script.js` | **host and device versions must match** — skew produces errors that look like a broken target (`pitfalls.md` P15) | +| `objection` | `objection -g explore` | quick Java-layer poking on top of Frida | +| `mitmproxy` | `mitmproxy --mode regular` | request/response inspection; a device proxy is usually faster to set up than a transparent one | +| `adb` | everything | the primary device interface | + +Version alignment for Frida is a **hard gate**, not a nicety. Check both sides before writing a +script, and check the device architecture — the server binary is per-ABI. + +## Tier 4 — cross-platform runtimes + +| Runtime | Identifier | Tool | +|---|---|---| +| Flutter / Dart | `libflutter.so` + `libapp.so` | `blutter` (needs the **matching Dart version**) — `dart-aot.md` | +| Unity / IL2CPP | `libil2cpp.so` + `global-metadata.dat` | Il2CppDumper + a native decompiler | +| React Native | `libhermes.so` / `index.android.bundle` | Hermes bytecode tooling, or plain JS if the bundle is unminified | +| Cordova / hybrid | WebView + `assets/www` | read the JS directly; Java layer is usually thin | + +**A version mismatch here wastes the most time of any tool in this file.** A Dart decompiler built +for a different engine version produces output that is subtly wrong rather than obviously broken. +Pin the version first (`dart-aot.md` §1) and do not "try it and see". + +## The general trap: a tool's failure is not a finding about the target + +A missing jar, a version skew, a forged section header, and a genuinely absent symbol all produce +**empty or partial output**. The output looks the same; the conclusions are completely different. + +Before reporting "there is nothing there", confirm the tool was actually capable of finding it: + +- Did it read the file at all? (entry count, segment count, non-zero output somewhere) +- Does a **known-present** item show up? (search for something you already know exists — a symbol + you saw in the dynamic table, a string you read in the hex dump) +- Would a different method see it? (byte pattern vs decoded scan) + +**This is `pitfalls.md` P25 and P26 in tool form.** A self-built analyser that fails quietly reads +exactly like a clean target, and it removes routes from consideration for free. + +## Working without a network + +Assume nothing can be downloaded. What still works, in order of usefulness: + +1. Tier 0 in full, plus the Python standard library — enough for structure, strings, hashing, and + most ELF/dex parsing. +2. Whatever jars and binaries are already present. **Locate them once and reuse the paths** rather + than re-searching on every command. +3. A local package cache or an offline mirror, if the environment has one. + +If a tool genuinely cannot be obtained, say which step is blocked and **what a substitute would +give up** — do not silently substitute a weaker method and report its result as equivalent. + +## Recording the toolchain actually used + +When you report, name the tools and versions that produced each finding. Two agents with different +tool versions will reach different conclusions about the same binary, and without the version that +difference is unresolvable later. diff --git a/scripts/apk_diff.py b/scripts/apk_diff.py new file mode 100644 index 0000000..009f0ff --- /dev/null +++ b/scripts/apk_diff.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""Entry-level APK diff: what actually changed between two builds. + +Two jobs, both recurring: + + 1. **Audit a build you did not produce.** Given an original and a + "cracked"/"modded" copy, this lists every entry that differs, every entry + the author ADDED, and every entry they REMOVED. Added entries under + `lib/`, `assets/`, or new dex files are the interesting ones -- that is + where injected payloads live. + + 2. **Verify your own build was surgical.** After repacking, the diff against + the original must contain exactly the entries you intended to touch and + nothing else. A surprise entry is a bug in your pipeline, not in the app. + +It compares *content*, not just sizes: two builds can have identical sizes and +different bytes, and a same-size replacement is exactly the case a careless +comparison misses. + +Usage +----- + python apk_diff.py original.apk modified.apk + python apk_diff.py original.apk modified.apk --hide-identical + python apk_diff.py original.apk modified.apk --only 'classes.*\\.dex|lib/.*' + python apk_diff.py original.apk modified.apk --summary +""" + +import argparse +import hashlib +import re +import sys +import zipfile + +# Directories worth naming explicitly -- this is where injected code lives. +INTERESTING = ("lib/", "assets/", "META-INF/services/") + + +def snapshot(path): + """{entry_name: (size, sha256, compress_type, crc)}""" + out = {} + with zipfile.ZipFile(path) as z: + for it in z.infolist(): + if it.is_dir(): + continue + data = z.read(it.filename) + out[it.filename] = ( + len(data), + hashlib.sha256(data).hexdigest(), + it.compress_type, + it.CRC, + ) + return out + + +def classify(name): + if re.match(r"^classes\d*\.dex$", name): + return "dex" + if name.startswith("META-INF/") and name.endswith((".RSA", ".SF", ".MF", ".DSA", ".EC")): + return "signature" + if name.startswith(INTERESTING): + return "code/assets" + return "other" + + +def main(): + ap = argparse.ArgumentParser(description="Entry-level APK diff.") + ap.add_argument("original") + ap.add_argument("modified") + ap.add_argument("--only", help="regex: restrict output to matching entry names") + ap.add_argument("--hide-identical", action="store_true", + help="omit the identical-count line") + ap.add_argument("--summary", action="store_true", + help="counts only, no per-entry listing") + ap.add_argument("--max", type=int, default=200, + help="max entries listed per section (default 200)") + args = ap.parse_args() + + try: + a = snapshot(args.original) + b = snapshot(args.modified) + except Exception as e: + sys.exit("error reading APK: %s" % e) + + only_a = sorted(set(a) - set(b)) + only_b = sorted(set(b) - set(a)) + common = sorted(set(a) & set(b)) + changed = [n for n in common if a[n][1] != b[n][1]] + + rx = re.compile(args.only) if args.only else None + + def keep(n): + return rx.search(n) if rx else True + + def sha(path, n): + return a[n][1][:16] if n in a else b[n][1][:16] + + print("A (original) %s entries=%d" % (args.original, len(a))) + print("B (modified) %s entries=%d" % (args.modified, len(b))) + print("") + print("only in A (removed) : %d" % len(only_a)) + print("only in B (added) : %d" % len(only_b)) + print("content changed : %d" % len(changed)) + if not args.hide_identical: + print("identical : %d" % (len(common) - len(changed))) + print("") + + # Signature entries always differ; say so once instead of listing them. + sig_only = [n for n in (only_a + only_b) if classify(n) == "signature"] + if sig_only: + print("(note: %d signature entries account for most of the add/remove noise)" + % len(sig_only)) + + if args.summary: + return + + if only_b: + print("\n=== ADDED in B (injection candidates) ===") + shown = 0 + for n in only_b: + if not keep(n) or classify(n) == "signature": + continue + print(" + %-52s %9d B [%s]" % (n, b[n][0], classify(n))) + shown += 1 + if shown >= args.max: + print(" ... (truncated)") + break + + if only_a: + print("\n=== REMOVED from A ===") + shown = 0 + for n in only_a: + if not keep(n) or classify(n) == "signature": + continue + print(" - %-52s %9d B [%s]" % (n, a[n][0], classify(n))) + shown += 1 + if shown >= args.max: + print(" ... (truncated)") + break + + if changed: + print("\n=== CONTENT CHANGED ===") + shown = 0 + for n in changed: + if not keep(n): + continue + sa, sb = a[n], b[n] + cmark = "" if sa[2] == sb[2] else " ctype %d->%d" % (sa[2], sb[2]) + same_size = " (SAME SIZE)" if sa[0] == sb[0] else "" + print(" ~ %-52s A %9d %s B %9d %s%s%s" + % (n, sa[0], sa[1][:16], sb[0], sb[1][:16], same_size, cmark)) + shown += 1 + if shown >= args.max: + print(" ... (truncated)") + break + if any(a[n][0] == b[n][0] for n in changed if keep(n)): + print("\n (SAME SIZE means the change is not a length difference --") + print(" a size-only comparison would have missed it entirely)") + + if not (only_a or only_b or changed): + print("no differences at all") + + +if __name__ == "__main__": + main() diff --git a/scripts/elf_plt.py b/scripts/elf_plt.py new file mode 100644 index 0000000..a65f63f --- /dev/null +++ b/scripts/elf_plt.py @@ -0,0 +1,457 @@ +#!/usr/bin/env python3 +"""PLT stub -> imported symbol mapping, and byte-level .so diffing. + +Built for hardened/rebuilt native libraries where the usual paths lie: + * section headers are often forged, so tools that walk sections (readelf -x, + objdump -d on named sections, pyelftools' section API) return confidently + wrong or empty results; + * a linear disassembler can stop silently on a buffer that does not start at + an instruction boundary, so "no matches" from it is not evidence. + +Therefore this tool walks PT_LOAD / PT_DYNAMIC by hand and decodes the two +stub shapes with its own decoders (no capstone dependency, no silent stops). + +Why you need the stub -> symbol map: when a hardening library terminates the +process, it does so through a PLT stub. To suppress it safely you must know +*which* symbol a given stub resolves to -- inferring from position or from a +comment is how you end up freezing `snprintf` or `pthread_exit` and reporting +it as "the patch did not work". + +Usage +----- + # full stub table + python elf_plt.py libfoo.so + + # what do these stubs call? + python elf_plt.py libfoo.so --query 0x9920 0x9950 + + # find the stub for one symbol (and every caller of it) + python elf_plt.py libfoo.so --symbol kill --callers + + # what changed between the original and a rebuilt library? + python elf_plt.py original.so --diff rebuilt.so + + # same, but also name the symbol each changed stub belongs to + python elf_plt.py original.so --diff rebuilt.so --name-regions +""" + +import argparse +import hashlib +import os +import struct +import sys + +PT_LOAD = 1 +PT_DYNAMIC = 2 + +DT_PLTRELSZ = 2 +DT_PLTGOT = 3 +DT_STRTAB = 5 +DT_SYMTAB = 6 +DT_RELA = 7 +DT_RELASZ = 8 +DT_STRSZ = 10 +DT_SYMENT = 11 +DT_PLTREL = 20 +DT_JMPREL = 23 + +EM_X86_64 = 62 +EM_AARCH64 = 183 + +ARCH_NAME = {EM_X86_64: "x86_64", EM_AARCH64: "aarch64"} + + +# -------------------------------------------------------------------------- +# ELF parsing (program headers only -- never trust the section table here) +# -------------------------------------------------------------------------- + +class Elf: + def __init__(self, data, path=""): + self.data = data + self.path = path + if data[:4] != b"\x7fELF": + raise ValueError("not an ELF file") + if data[4] != 2: + raise ValueError("only 64-bit ELF is supported") + self.machine = struct.unpack_from(" len(elf.data): + return "?" + (st_name,) = struct.unpack_from("= strsz: + return "?" + start = strtab_o + st_name + end = elf.data.find(b"\x00", start) + if end < 0: + return "?" + return elf.data[start:end].decode("utf-8", "replace") + + out = {} + for tag_off, tag_sz in ((DT_JMPREL, DT_PLTRELSZ), (DT_RELA, DT_RELASZ)): + if tag_off not in tags: + continue + base = elf.vaddr_to_off(tags[tag_off]) + if base is None: + continue + n = tags.get(tag_sz, 0) // 24 + for k in range(n): + if base + k * 24 + 16 > len(elf.data): + break + r_offset, r_info = struct.unpack_from("> 32) + return out + + +# -------------------------------------------------------------------------- +# Stub decoders -- written out rather than delegated, so nothing stops early +# -------------------------------------------------------------------------- + +def find_stubs(elf, relocs): + """{stub_vaddr: (symbol, got_vaddr)} for the library's stub table.""" + base, foff, fsz, _ms, _fl = elf.exec_segment() + if elf.machine == EM_X86_64: + return _stubs_x86(elf.data, base, foff, fsz, relocs) + if elf.machine == EM_AARCH64: + return _stubs_arm64(elf.data, base, foff, fsz, relocs) + return {} + + +def _stubs_x86(data, base, foff, fsz, relocs): + """x86_64 PLT stub: `ff 25 ` == jmp qword ptr [rip+disp32].""" + out = {} + i = 0 + while i < fsz - 6: + if data[foff + i] == 0xFF and data[foff + i + 1] == 0x25: + disp = struct.unpack_from(" (target_page, Rd). None if not an adrp.""" + if (word >> 31) & 1 != 1: + return None + if ((word >> 24) & 0x1F) != 0x10: + return None + immlo = (word >> 29) & 0x3 + immhi = (word >> 5) & 0x7FFFF + imm = (immhi << 2) | immlo + imm = _sign_extend(imm, 21) + return (pc & ~0xFFF) + (imm << 12), (word & 0x1F) + + +def _decode_ldr_unsigned(word): + """ldr Rt, [Rn, #imm] (64-bit unsigned offset) -> (Rn, Rt, byte_offset).""" + if (word >> 30) != 0b11: # size == 64-bit + return None + if ((word >> 27) & 0x7) != 0b111: + return None + if ((word >> 24) & 0x3) != 0b01: + return None + imm12 = (word >> 10) & 0xFFF + rn = (word >> 5) & 0x1F + rt = word & 0x1F + return rn, rt, imm12 * 8 + + +def _decode_add_imm(word): + """add Rd, Rn, #imm -> (Rn, Rd, imm). + + Encoding: sf op S 100010 sh imm12 Rn Rd + bits[31]=sf bits[30]=op bits[29]=S bits[28:23]=100010 (6 bits, not 9) + """ + if (word >> 31) & 1 != 1: # sf must be 1 (64-bit form) + return None + if ((word >> 23) & 0x3F) != 0b100010: + return None + if (word >> 22) & 1: # shifted form -- not what stubs use + return None + imm12 = (word >> 10) & 0xFFF + rn = (word >> 5) & 0x1F + rd = word & 0x1F + return rn, rd, imm12 + + +def _decode_br(word): + """br Rn -> Rn, or None.""" + if (word & 0xFFFFFC1F) == 0xD61F0000: + return (word >> 5) & 0x1F + return None + + +def find_branch_callers(elf, target_vaddr, limit=500): + """Addresses of BL/B instructions that branch to target_vaddr.""" + base, foff, fsz, _ms, _fl = elf.exec_segment() + hits = [] + if elf.machine == EM_AARCH64: + off = 0 + while off < fsz - 4: + (w,) = struct.unpack_from("= limit: + break + off += 4 + elif elf.machine == EM_X86_64: + i = 0 + while i < fsz - 5: + if elf.data[foff + i] == 0xE8: + disp = struct.unpack_from("= limit: + break + i += 1 + return hits + + +def data_slice(data, off, n): + return data[off:off + n] + + +# -------------------------------------------------------------------------- +# Commands +# -------------------------------------------------------------------------- + +def load(path): + with open(path, "rb") as fh: + return Elf(fh.read(), path) + + +def cmd_list(elf, args): + relocs = reloc_map(elf) + stubs = find_stubs(elf, relocs) + base, foff, fsz, _ms, _fl = elf.exec_segment() + print("== %s" % elf.path) + print(" arch=%s exec seg: vaddr=0x%x file=0x%x size=0x%x" % (elf.arch, base, foff, fsz)) + print(" imported relocations=%d stubs resolved=%d" % (len(relocs), len(stubs))) + if not stubs: + print(" NOTE: zero stubs resolved. Either the library has no PLT, or the") + print(" hardening removed/forged the relocation tables. Do not read") + print(" this as 'the library imports nothing'.") + for va in sorted(stubs): + sym, got = stubs[va] + print(" 0x%08x -> %-34s (GOT 0x%x)" % (va, sym, got)) + return stubs + + +def cmd_query(elf, args): + relocs = reloc_map(elf) + stubs = find_stubs(elf, relocs) + for a in args.query: + va = int(a, 0) + hit = stubs.get(va) + if hit: + print("0x%x -> %s (GOT 0x%x)" % (va, hit[0], hit[1])) + else: + print("0x%x -> not a resolved stub" % va) + + +def cmd_symbol(elf, args): + relocs = reloc_map(elf) + stubs = find_stubs(elf, relocs) + wanted = args.symbol + found = [(va, sym, got) for va, (sym, got) in stubs.items() if sym == wanted] + if not found: + near = sorted({s for _v, (s, _g) in stubs.items() + if wanted in s or s in wanted})[:10] + print("no stub resolves to %r" % wanted) + if near: + print(" similar: %s" % ", ".join(near)) + return + for va, sym, got in sorted(found): + print("stub 0x%x -> %s (GOT 0x%x)" % (va, sym, got)) + if args.callers: + callers = find_branch_callers(elf, va) + print(" callers of the stub: %d" % len(callers)) + for c in callers: + print(" 0x%x" % c) + print(" NOTE: a stub-level patch covers all of these. A call-site-level") + print(" patch must cover every one of them, plus any caller reached") + print(" through a cached function pointer (not listed here).") + + +def cmd_diff(elf_a, args): + path_b = args.diff + elf_b = load(path_b) + a, b = elf_a.data, elf_b.data + print("== diff") + print(" A %s %d bytes sha256=%s" % (elf_a.path, len(a), hashlib.sha256(a).hexdigest()[:16])) + print(" B %s %d bytes sha256=%s" % (path_b, len(b), hashlib.sha256(b).hexdigest()[:16])) + if len(a) != len(b): + print(" SIZE DIFFERS: %d vs %d -- a length change is itself a finding" + % (len(a), len(b))) + n = min(len(a), len(b)) + runs = [] + i = 0 + while i < n: + if a[i] != b[i]: + j = i + while j < n and a[j] != b[j]: + j += 1 + runs.append((i, j - i)) + i = j + else: + i += 1 + total = sum(r[1] for r in runs) + print(" differing bytes=%d in %d run(s)" % (total, len(runs))) + + regions = {} + if args.name_regions: + relocs_a = reloc_map(elf_a) + stubs_a = find_stubs(elf_a, relocs_a) + + def describe(off): + for va, (sym, _got) in stubs_a.items(): + size = 16 if elf_a.machine == EM_AARCH64 else 16 + if va <= off < va + size: + return "PLT stub for %s" % sym + return "" + + regions = {"describe": describe} + + for off, ln in runs: + note = "" + if regions: + note = regions["describe"](off) + if note: + note = " <- " + note + print(" off=0x%06x len=%-4d A: %s" % (off, ln, a[off:off + ln].hex(" "))) + print(" %s B: %s%s" % (" " * 14, b[off:off + ln].hex(" "), note)) + if args.name_regions and not runs: + print(" (identical)") + + +def main(): + ap = argparse.ArgumentParser( + description="PLT stub -> symbol mapping and .so diffing for hardened libraries.", + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("so", help="path to the ELF (.so)") + ap.add_argument("--query", nargs="+", metavar="ADDR", + help="resolve specific stub addresses (hex ok)") + ap.add_argument("--symbol", metavar="NAME", + help="find the stub that resolves to this symbol") + ap.add_argument("--callers", action="store_true", + help="with --symbol: also list branch call sites") + ap.add_argument("--diff", metavar="OTHER_SO", + help="byte-compare against another library") + ap.add_argument("--name-regions", action="store_true", + help="with --diff: name the symbol each changed stub belongs to") + args = ap.parse_args() + + try: + elf = load(args.so) + except Exception as e: + sys.exit("error: %s" % e) + + if args.diff: + cmd_diff(elf, args) + elif args.query: + cmd_query(elf, args) + elif args.symbol: + cmd_symbol(elf, args) + else: + cmd_list(elf, args) + + +if __name__ == "__main__": + main() diff --git a/scripts/native_crash.py b/scripts/native_crash.py new file mode 100644 index 0000000..820bd40 --- /dev/null +++ b/scripts/native_crash.py @@ -0,0 +1,279 @@ +#!/usr/bin/env python3 +"""Extract native crash blocks from a log or tombstone and locate the fault. + +Native deaths are the expensive kind: no Java stack, often no obvious cause, and +the interesting question ("which library, which offset, is this a real fault or +an arranged one?") is buried in a debug dump that is tedious to read by hand. + +This reads a saved `logcat` capture or a tombstone file and reports, per crash: + + * the signal, fault address, and cause + * the register state that matters (the faulting pointer) + * the backtrace, split into your own libraries vs system libraries + * a verdict hint when the fault address looks *arranged* rather than accidental + +With `--lib NAME=PATH` it also resolves a frame to a local copy of the library +and prints the bytes at that offset, plus a disassembly window when a decoder is +available. + +Why the verdict hint matters: a hardening layer that decides the environment is +hostile rarely calls `kill`. It more often arranges a fault -- load a small +constant, use it as a pointer -- so the death looks like an ordinary bug. The +tell is a fault address that is a small integer combined with a register holding +that same integer. This script flags that shape so you look at the right place +instead of hunting a bug that is not there. + +Usage +----- + python native_crash.py capture.txt + python native_crash.py tombstone_07 + python native_crash.py capture.txt --lib libfoo.so=/path/to/local/libfoo.so + python native_crash.py capture.txt --lib libfoo.so=./libfoo.so --disasm 16 +""" + +import argparse +import os +import re +import struct +import sys + +SIGNAL_RE = re.compile( + r"signal\s+(\d+)\s+\((\w+)\)\s*,?\s*code\s+(\d+)\s*\(([^)]+)\)\s*,?\s*fault addr\s+(0x[0-9a-fA-F]+)") +PID_RE = re.compile(r"pid:\s*(\d+),\s*tid:\s*(\d+),\s*name:\s*(\S+)\s*>>>\s*(\S+)") +PROC_RE = re.compile(r"Cmdline:\s*(\S+)") +UPTIME_RE = re.compile(r"Process uptime:\s*(\d+)") +FRAME_RE = re.compile(r"#(\d+)\s+pc\s+([0-9a-fA-F]+)\s+(\S+)") +REG_RE = re.compile(r"^\s*([xX]\d{1,2}|sp|lr|pc|pst)\s+([0-9a-fA-F]{8,16})") +SYSTEM_LIB_RE = re.compile(r"^/(apex|system|vendor|data/dalvik-cache|memfd)") + + +def is_system_lib(path): + if not path.startswith("/"): + return True + return bool(SYSTEM_LIB_RE.match(path)) + + +def parse(text): + """Yield crash dicts.""" + lines = text.splitlines() + crashes = [] + i = 0 + while i < len(lines): + m = SIGNAL_RE.search(lines[i]) + # some ROMs print the signal line without "F DEBUG"; accept both + if not m: + i += 1 + continue + block = {"signal": int(m.group(1)), "signame": m.group(2), + "si_code": m.group(4), "fault": m.group(5), + "frames": [], "regs": {}, "raw": []} + # walk up a little to pick up pid/cmdline/uptime printed just before + for j in range(max(0, i - 25), i): + l = lines[j] + mm = PID_RE.search(l) + if mm and "pid" not in block: + block["pid"], block["tid"], block["tname"], block["proc"] = mm.groups() + mm = PROC_RE.search(l) + if mm and "proc" not in block: + block["proc"] = mm.group(1) + mm = UPTIME_RE.search(l) + if mm: + block["uptime"] = mm.group(1) + if "Cause:" in l and "cause" not in block: + block["cause"] = l.split("Cause:", 1)[1].strip() + # walk down through the dump + j = i + while j < len(lines) and j < i + 120: + l = lines[j] + if j > i + 3 and SIGNAL_RE.search(l) and j > i + 3: + break + block["raw"].append(l.strip()) + mm = PID_RE.search(l) + if mm and "pid" not in block: + block["pid"], block["tid"], block["tname"], block["proc"] = mm.groups() + mm = PROC_RE.search(l) + if mm and "proc" not in block: + block["proc"] = mm.group(1) + mm = UPTIME_RE.search(l) + if mm: + block["uptime"] = mm.group(1) + if "Cause:" in l and "cause" not in block: + block["cause"] = l.split("Cause:", 1)[1].strip() + mm = FRAME_RE.search(l) + if mm: + # Backtrace lines look like either of: + # #00 pc 00000000000128cc /data/app/.../lib/arm64/libfoo.so + # #02 pc 00000000000cccd0 /apex/.../libc.so (__pthread_start+256) + # The path is group(3); anything after it is a symbol in parens. + path = mm.group(3) + rest = l.split(path, 1)[1].strip() if path in l else "" + sym = "" + m2 = re.search(r"\(([^)]+)\)", rest) + if m2: + sym = m2.group(1) + block["frames"].append({ + "n": mm.group(1), "pc": int(mm.group(2), 16), + "path": path, "symbol": sym, + }) + # Register dumps put several registers on one line, so collect all of + # them rather than only the leading one. + for k, v in re.findall(r"\b([xX]\d{1,2}|sp|lr|pc|pst)\s+([0-9a-fA-F]{8,16})\b", + l.replace("F DEBUG :", " ")): + block["regs"][k] = int(v, 16) + if "backtrace:" in l: + pass + j += 1 + i = j if j > i else i + 1 + crashes.append(block) + return crashes + + +def verdict_hint(crash): + hints = [] + fault = int(crash["fault"], 16) + regs = crash["regs"] + if fault <= 0x100: + holder = [k for k, v in regs.items() if v == fault] + hints.append("fault address is a small integer (%s)." % crash["fault"]) + if holder: + hints.append("register(s) %s hold exactly that value -- this is the shape of an " + "ARRANGED fault, not an accident." % ", ".join(sorted(holder))) + hints.append("check whether the instruction at the faulting pc loaded that constant " + "a few instructions earlier and used it as a pointer.") + if crash["signame"] == "SIGKILL": + hints.append("SIGKILL with no tombstone: this is a deliberate termination, not a bug. " + "No crash dump exists because nothing faulted.") + if crash.get("cause") and "null pointer" in crash["cause"] and fault > 0x100: + hints.append("a genuine null-ish dereference; treat as an ordinary bug unless a small " + "constant is involved.") + return hints + + +def load_libs(specs): + out = {} + for s in specs or []: + if "=" not in s: + sys.exit("--lib expects NAME=PATH, got %r" % s) + name, path = s.split("=", 1) + with open(path, "rb") as fh: + out[name] = fh.read() + return out + + +def maybe_disasm(data, offset, count): + try: + from capstone import Cs, CS_ARCH_ARM64, CS_MODE_ARM, CS_ARCH_X86, CS_MODE_64 + except Exception: + return None + # guess arch from ELF e_machine + if data[:4] != b"\x7fELF": + return None + machine = struct.unpack_from("= count: + break + if not out: + # capstone stopped immediately -- say so rather than implying emptiness + return [" (decoder produced nothing from this offset; a silent stop is not " + "evidence there is no code here)"] + return out + + +def main(): + ap = argparse.ArgumentParser(description="Locate native crashes from a log or tombstone.") + ap.add_argument("logfile", help="logcat capture or tombstone file") + ap.add_argument("--lib", action="append", metavar="NAME=PATH", + help="resolve frames for this library name to a local file") + ap.add_argument("--disasm", type=int, default=0, metavar="N", + help="disassemble N instructions around the faulting pc") + ap.add_argument("--frames", type=int, default=12, help="how many frames to print") + args = ap.parse_args() + + if not os.path.exists(args.logfile): + sys.exit("no such file: %s" % args.logfile) + with open(args.logfile, "r", encoding="utf-8", errors="replace") as fh: + text = fh.read() + + crashes = parse(text) + if not crashes: + print("no native crash block found in %s" % args.logfile) + print("") + print("This is a real negative only if the capture covers the death. Check:") + print(" - was the crash buffer cleared *after* the build under test was installed?") + print(" - does the capture include the tombstone? (a SIGKILL leaves none)") + print(" - did the process die at all, or did it merely restart?") + return + + libs = load_libs(args.lib) + + for idx, c in enumerate(crashes, 1): + print("=" * 72) + print("CRASH %d" % idx) + print(" process : %s" % c.get("proc", "?")) + print(" pid/tid : %s / %s" % (c.get("pid", "?"), c.get("tid", "?"))) + if c.get("uptime"): + print(" uptime : %ss <-- time from launch to death; your test window must " + "exceed this" % c["uptime"]) + print(" signal : %s (%d), si_code=%s" % (c["signame"], c["signal"], c["si_code"])) + print(" fault addr: %s" % c["fault"]) + if c.get("cause"): + print(" cause : %s" % c["cause"]) + + hints = verdict_hint(c) + if hints: + print(" HINTS") + for h in hints: + print(" - %s" % h) + + app_frames = [f for f in c["frames"] if not is_system_lib(f["path"])] + if app_frames: + print(" backtrace (your libraries):") + for f in app_frames[:args.frames]: + print(" #%s pc 0x%x %s%s" + % (f["n"], f["pc"], os.path.basename(f["path"]), + (" (%s)" % f["symbol"]) if f["symbol"] else "")) + else: + print(" backtrace: no frame inside an app library") + print(" -> the fault is in the runtime or a system library; an app-side") + print(" patch is unlikely to be the cause. Look at what called in.") + + if c["frames"]: + print(" full backtrace:") + for f in c["frames"][:args.frames]: + tag = "system" if is_system_lib(f["path"]) else "APP" + print(" #%s [%s] pc 0x%x %s%s" + % (f["n"], tag, f["pc"], (f["path"] or "?"), + (" (%s)" % f["symbol"]) if f["symbol"] else "")) + + # resolve against local libraries + for f in c["frames"]: + base = os.path.basename(f["path"]) + for name, data in libs.items(): + if name in base or base in name: + print(" local %s @ 0x%x:" % (name, f["pc"])) + if args.disasm and f is c["frames"][0]: + for line in maybe_disasm(data, f["pc"], args.disasm) or []: + print(line) + else: + print(" raw: %s" % data[f["pc"]:f["pc"] + 16].hex(" ")) + print("") + + print("=" * 72) + print("RECORD THIS: signal, fault addr, and the time-to-death above. They are the") + print("baseline an attempted fix must be measured against -- without them a later") + print("run cannot distinguish 'fixed' from 'the timing changed'.") + + +if __name__ == "__main__": + main()