Close the byte-patching gap, add the server-config path, fix two script crashes
Worked end-to-end on a real unobfuscated commercial app (single-dex, no packer,
server-driven launch screen) and folded back everything that cost time. Net +17
files, no new dependencies, no sample-specific identifiers.
New references
byte-level-patching.md
The technique that was missing entirely. Equal-length in-place edits beat
method rebuilding (measured on a real sample: method rewrite grew debug_info
924 B -> 22.8 KB and the dex 4.32 MB -> 7.73 MB), but four failure modes are
silent: same-length is not enough, branch polarity, verifier legality, and
stale dex header fields. Covers locating an instruction's exact offset (why
listings and byte-pattern guesses both fail), the instruction width table and
the two traps that desynchronise a decode, neutralise-a-branch vs redirect-it,
and the header's signature-then-checksum order.
server-config-and-updates.md
The most common shape of "ad" in a modern app, and the one usually
mis-diagnosed: the server supplies UI the client renders. The two-layer fetch
(local fallback then remote override) that proves it, finding the config DTOs
through the field names data classes keep, why you patch the decision and not
the data, deciding the scope of "remove", and remote re-enable / cached config
durability.
New scripts
dexutil.py dependency-free dex reader: structural walk, exact
instruction decode, dex header recompute/verify in the
correct order, branch-target and operand helpers
dex_find_insn.py locate an instruction by decoded semantics, print its exact
byte offset with context and BOTH sides of any branch
dex_patch_bytes.py equal-length byte patches from a JSON spec: semantic match,
polarity pin via expect_next, equal-length enforcement,
verifier check, header recompute, re-decode to prove it landed
dex_check_verifier.py does any conditional branch target a move-result (bypassing
its producer)? Compares builds and separates pre-existing
findings from regressions the patch introduced
coldstart.py cold-launch capture: timed screenshot burst + logcat signals
+ installed-build facts + launch timing, and a warning when
the foreground activity is not the app
check_repo.py maintenance check: every script parses and answers --help,
every referenced path resolves, every file is documented
Fixes found while running the above
repack.py wrote archives with a plain zipfile writer, which cannot satisfy
the Android R+ install gate: resources.arsc must be STORED *and*
4-byte aligned, and a zip extra record cannot express a 1-3 byte
pad (a stored filler entry is inserted instead). Also fixes the
central-directory pointer and adds an alignment gate to the output.
usb_net_proxy.py, dex_strpatch.py
crashed on --help with a traceback instead of printing usage.
Updated
toolchain.md ddc added (single-binary dex->Java with query subcommands,
with measured timings and its real limits); "not on PATH is
not not-installed" with the search commands; which signer
to use and why jarsigner breaks alignment
repack-and-sign.md alignment requirement and its arithmetic; dex header
recompute; jarsigner hazard; vendor install interception
([N] with no symbolic constant -> root pm install); the
installer can keep owning the screen after an install
recon.md resolve package identity from a tool, not from hand-read
AXML strings (a wrong package sends all later device queries
nowhere); snapshot the config surface during recon
long-task-discipline.md captures you never looked at are not evidence (incl. the
consistent-but-wrong-window variant); long-context decay and
the record shape that survives it
dynamic-frida.md when live attach cannot work at all: static instrumentation,
static read, and self-recorded evidence, with the control
build caveat each one carries
ad-removal.md server-driven config is usually the EASIEST case, not the
hardest; a three-way zero result is a positive finding;
scope note on local brand splash vs promotional content
pitfalls.md P37 stale dex header checksum, P38 [-124] alignment,
P39 bare-numeric install failure, P40 wrong-window evidence,
P41 inverted branch polarity, P42 decode desynchronisation
SKILL.md, README.md symptom index, reference/script indexes and requirements
updated for all of the above
This commit is contained in:
inliver233
@@ -0,0 +1,304 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Cold-launch an app and capture a time line of screenshots plus logcat signals.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
"Did the patch work?" for anything user-visible -- a splash/ad gate, a forced
|
||||
update dialog, a login wall, a crash-on-start -- is answered by looking at the
|
||||
screen over the first seconds of a launch, and by comparison against the
|
||||
unmodified build. Doing that by hand produces two failures that look like
|
||||
findings and are not:
|
||||
|
||||
* **Sampling is not observation.** A dialog that appears and is then occluded
|
||||
can fall entirely between samples, and every frame you happened to take shows
|
||||
something else, which feels like corroboration. This tool captures on a fixed
|
||||
short cadence and, crucially, prints what it captured so a human or agent
|
||||
actually LOOKS at the frames.
|
||||
* **`am start -W` can block past a naive timeout.** It waits for the first
|
||||
frame; on some ROMs that never arrives while another window owns the display
|
||||
and the command simply hangs. This tool launches without `-W` and derives the
|
||||
time line from captures plus logcat instead.
|
||||
|
||||
It also records the two facts that decide whether an install even happened:
|
||||
the installed version, and the on-disk APK hash. "Install succeeded" describes
|
||||
the request, not the app on disk.
|
||||
|
||||
IMPORTANT -- before trusting any screenshot, confirm the foreground activity is
|
||||
your target. A vendor package installer left on screen (a pending confirmation
|
||||
from an earlier `adb install`, for example) will be captured instead of your app
|
||||
and looks completely plausible. `--expect-activity` makes that check automatic.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python coldstart.py --serial <SERIAL> --pkg com.example \\
|
||||
--activity com.example/.MainActivity --duration 14 --interval 0.8 \\
|
||||
--out work/verify/orig --expect-activity com.example/.MainActivity
|
||||
|
||||
Requires: adb. `su` on the device is used when available for version/hash facts,
|
||||
otherwise it degrades to plain `pm`/`dumpsys`.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
def run(cmd, timeout=30, serial=None):
|
||||
"""Run a command with a hard timeout; never let a call stall the run."""
|
||||
try:
|
||||
return subprocess.run(cmd, capture_output=True, timeout=timeout,
|
||||
text=True, errors='replace')
|
||||
except subprocess.TimeoutExpired:
|
||||
return subprocess.CompletedProcess(cmd, 124, '', 'timeout after %ss' % timeout)
|
||||
|
||||
|
||||
class Device(object):
|
||||
def __init__(self, serial, use_su=True, timeout=30):
|
||||
self.serial = serial
|
||||
self.timeout = timeout
|
||||
self.adb = shutil.which('adb') or 'adb'
|
||||
self.su = use_su and self._su_works()
|
||||
|
||||
def _su_works(self):
|
||||
if self.serial:
|
||||
r = run(['adb', '-s', self.serial, 'shell', "su -c 'id'"], 15)
|
||||
else:
|
||||
r = run(['adb', 'shell', "su -c 'id'"], 15)
|
||||
return r.returncode == 0 and 'uid=0' in (r.stdout or '')
|
||||
|
||||
def shell(self, cmd, timeout=None):
|
||||
"""Run a device shell command, preferring root when available."""
|
||||
t = timeout or self.timeout
|
||||
if self.serial:
|
||||
base = ['adb', '-s', self.serial, 'shell']
|
||||
else:
|
||||
base = ['adb', 'shell']
|
||||
if self.su:
|
||||
return run(base + ["su -c '%s'" % cmd.replace("'", "'\\''")], t)
|
||||
return run(base + [cmd], t)
|
||||
|
||||
def screencap(self, path, timeout=30):
|
||||
"""Capture one PNG. Returns bytes written, or -1 on failure."""
|
||||
if self.serial:
|
||||
base = ['adb', '-s', self.serial]
|
||||
else:
|
||||
base = ['adb']
|
||||
try:
|
||||
with open(path, 'wb') as fh:
|
||||
r = subprocess.run(base + ['exec-out', 'screencap', '-p'],
|
||||
stdout=fh, timeout=timeout)
|
||||
size = os.path.getsize(path)
|
||||
if size < 8:
|
||||
return -1
|
||||
with open(path, 'rb') as fh:
|
||||
if fh.read(8) != b'\x89PNG\r\n\x1a\n':
|
||||
return -1
|
||||
return size
|
||||
except subprocess.TimeoutExpired:
|
||||
return -1
|
||||
|
||||
|
||||
def foreground_activity(dev):
|
||||
r = dev.shell('dumpsys activity activities | grep -m1 ResumedActivity')
|
||||
m = re.search(r'([A-Za-z0-9_.]+)/([A-Za-z0-9_.$]+)', r.stdout or '')
|
||||
return '%s/%s' % (m.group(1), m.group(2)) if m else None
|
||||
|
||||
|
||||
def installed_facts(dev, pkg):
|
||||
facts = {}
|
||||
r = dev.shell("dumpsys package %s | grep -E 'versionCode|versionName|firstInstallTime|lastUpdateTime'" % pkg)
|
||||
for line in (r.stdout or '').splitlines():
|
||||
m = re.match(r'\s*(versionCode|versionName|firstInstallTime|lastUpdateTime)=(.*)', line)
|
||||
if m:
|
||||
facts[m.group(1)] = m.group(2).strip()
|
||||
r = dev.shell('ls /data/app/*/%s*/base.apk 2>/dev/null' % pkg)
|
||||
apk_path = (r.stdout or '').strip().splitlines()[:1]
|
||||
if apk_path:
|
||||
facts['apk_path'] = apk_path[0]
|
||||
r2 = dev.shell('sha256sum %s' % apk_path[0])
|
||||
m = re.search(r'([0-9a-f]{64})', r2.stdout or '')
|
||||
if m:
|
||||
facts['apk_sha256'] = m.group(1)
|
||||
r = dev.shell('dumpsys package %s | grep -m1 userId=' % pkg)
|
||||
m = re.search(r'userId=(\d+)', r.stdout or '')
|
||||
if m:
|
||||
facts['uid'] = m.group(1)
|
||||
return facts
|
||||
|
||||
|
||||
def signal_counts(log_path, patterns):
|
||||
"""Count occurrences of each pattern in a logcat dump."""
|
||||
try:
|
||||
with open(log_path, encoding='utf-8', errors='replace') as fh:
|
||||
text = fh.read()
|
||||
except OSError:
|
||||
return {}
|
||||
return {name: len(re.findall(rx, text)) for name, rx in patterns.items()}
|
||||
|
||||
|
||||
SIGNAL_PATTERNS = {
|
||||
'FATAL EXCEPTION': r'FATAL EXCEPTION',
|
||||
'VerifyError': r'VerifyError',
|
||||
'Bad checksum': r'Bad checksum',
|
||||
'IncompatibleClassChangeError': r'IncompatibleClassChangeError',
|
||||
'ClassNotFoundException': r'ClassNotFoundException',
|
||||
'uncaughtException': r'uncaughtException',
|
||||
'am_crash': r'am_crash',
|
||||
'ANR': r'ANR in ',
|
||||
'Displayed': r'Displayed .*\+([0-9]+)ms',
|
||||
}
|
||||
|
||||
|
||||
def main(argv):
|
||||
ap = argparse.ArgumentParser(
|
||||
description='Cold-launch an app and capture a screenshot time line plus '
|
||||
'logcat signals, for before/after comparison.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__.split('Usage')[-1][:1200])
|
||||
ap.add_argument('--serial', help='adb serial (REQUIRED when several devices are '
|
||||
'online, so the wrong one is not picked at random)')
|
||||
ap.add_argument('--pkg', required=True, help='package name')
|
||||
ap.add_argument('--activity', help='component to start (default: monkey/launcher)')
|
||||
ap.add_argument('--out', default='coldstart', help='output directory')
|
||||
ap.add_argument('--duration', type=float, default=14.0, help='seconds to capture')
|
||||
ap.add_argument('--interval', type=float, default=0.8, help='seconds between frames')
|
||||
ap.add_argument('--expect-activity', help='warn if the foreground activity is not '
|
||||
'this (and not the package at all)')
|
||||
ap.add_argument('--no-su', action='store_true', help='never use device root')
|
||||
ap.add_argument('--keep-old', action='store_true',
|
||||
help='do not clear existing screenshots in the output dir')
|
||||
args = ap.parse_args(argv[1:])
|
||||
|
||||
if not shutil.which('adb'):
|
||||
print('error: adb not found on PATH')
|
||||
return 2
|
||||
|
||||
if not args.serial:
|
||||
r = run(['adb', 'devices'])
|
||||
online = [l.split()[0] for l in (r.stdout or '').splitlines()[1:]
|
||||
if len(l.split()) >= 2 and l.split()[1] == 'device']
|
||||
if len(online) > 1:
|
||||
print('error: %d devices online (%s) and no --serial given. adb would '
|
||||
'pick one at random and you would debug the wrong target.'
|
||||
% (len(online), ', '.join(online)))
|
||||
return 2
|
||||
if online:
|
||||
args.serial = online[0]
|
||||
print('[i] single device online; using %s' % args.serial)
|
||||
|
||||
dev = Device(args.serial, use_su=not args.no_su)
|
||||
print('[i] serial=%s root=%s' % (args.serial, dev.su))
|
||||
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
shots = os.path.join(args.out, 'shots')
|
||||
os.makedirs(shots, exist_ok=True)
|
||||
if not args.keep_old:
|
||||
for f in os.listdir(shots):
|
||||
if f.endswith('.png'):
|
||||
os.remove(os.path.join(shots, f))
|
||||
|
||||
facts = installed_facts(dev, args.pkg)
|
||||
print('\n== installed build facts')
|
||||
for k in sorted(facts):
|
||||
print(' %-18s %s' % (k, facts[k]))
|
||||
if not facts.get('versionCode'):
|
||||
print(' WARNING: no versionCode found -- the package may not be installed')
|
||||
print(' (install success describes the request, not the app on disk; the '
|
||||
'sha256 above is the on-disk truth)')
|
||||
|
||||
print('\n== clearing logcat')
|
||||
dev.shell('logcat -c', timeout=20)
|
||||
|
||||
print('== force-stopping %s' % args.pkg)
|
||||
dev.shell('am force-stop %s' % args.pkg, timeout=20)
|
||||
|
||||
print('== launching')
|
||||
t0 = time.time()
|
||||
if args.activity:
|
||||
r = dev.shell('am start -n %s' % args.activity, timeout=25)
|
||||
else:
|
||||
r = dev.shell('monkey -p %s -c android.intent.category.LAUNCHER 1' % args.pkg,
|
||||
timeout=25)
|
||||
out = ((r.stdout or '') + (r.stderr or '')).strip()
|
||||
print(' %s' % (out.splitlines()[0] if out else '(no output)'))
|
||||
if 'Error' in out or 'Exception' in out:
|
||||
print(' LAUNCH ERROR -- read the line above before blaming the patch')
|
||||
|
||||
# live logcat into a file while we capture
|
||||
log_path = os.path.join(args.out, 'logcat.txt')
|
||||
if args.serial:
|
||||
log_cmd = ['adb', '-s', args.serial, 'logcat', '-v', 'threadtime']
|
||||
else:
|
||||
log_cmd = ['adb', 'logcat', '-v', 'threadtime']
|
||||
log_fh = open(log_path, 'w', encoding='utf-8', errors='replace')
|
||||
log_proc = subprocess.Popen(log_cmd, stdout=log_fh, stderr=subprocess.DEVNULL)
|
||||
|
||||
frames = []
|
||||
idx = 0
|
||||
try:
|
||||
while True:
|
||||
el = time.time() - t0
|
||||
if el > args.duration:
|
||||
break
|
||||
name = 'f%02d_%05.2fs.png' % (idx, el)
|
||||
path = os.path.join(shots, name)
|
||||
n = dev.screencap(path)
|
||||
frames.append((name, n, el))
|
||||
print(' %-20s %s' % (name, ('%d bytes' % n) if n > 0 else 'CAPTURE FAILED'))
|
||||
idx += 1
|
||||
time.sleep(max(0.0, args.interval))
|
||||
finally:
|
||||
time.sleep(0.5)
|
||||
log_proc.terminate()
|
||||
try:
|
||||
log_proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
log_proc.kill()
|
||||
log_fh.close()
|
||||
|
||||
fg = foreground_activity(dev)
|
||||
print('\n== foreground at end: %s' % fg)
|
||||
if args.expect_activity and fg and args.pkg not in fg:
|
||||
print(' *** FOREGROUND IS NOT YOUR APP (%s) ***' % fg)
|
||||
print(' Everything captured above shows something else -- most often a '
|
||||
'vendor package-installer confirmation left over from an install.')
|
||||
print(' Clear it (e.g. close the installer, press HOME) and re-run; do not '
|
||||
'read these frames as evidence about your build.')
|
||||
|
||||
counts = signal_counts(log_path, SIGNAL_PATTERNS)
|
||||
print('\n== logcat signals (%s)' % log_path)
|
||||
for name in SIGNAL_PATTERNS:
|
||||
print(' %-32s %s' % (name, counts.get(name, 0)))
|
||||
|
||||
m = re.findall(r'Displayed ([^:]+): \+([0-9]+)ms', open(log_path, encoding='utf-8',
|
||||
errors='replace').read())
|
||||
if m:
|
||||
print('\n== launch timing (logcat Displayed)')
|
||||
for comp, ms in m:
|
||||
print(' %s %s ms' % (comp.strip(), ms))
|
||||
else:
|
||||
print('\n no "Displayed" line: the launch may not have reached a first '
|
||||
'frame (a hang or a blocking dialog looks exactly like this)')
|
||||
|
||||
zero = sum(1 for _n, n, _e in frames if n <= 0)
|
||||
print('\n== %d frame(s) in %s, %d failed' % (len(frames), shots, zero))
|
||||
|
||||
# the point of the tool: make inspection happen
|
||||
print('\n== INSPECT THE FRAMES. A capture nobody looked at is not evidence.')
|
||||
nonzero = [f for f in sorted(os.listdir(shots)) if f.endswith('.png')]
|
||||
if nonzero:
|
||||
print(' start with: %s' % os.path.join(shots, nonzero[0]))
|
||||
mid = nonzero[len(nonzero) // 2]
|
||||
print(' and: %s' % os.path.join(shots, mid))
|
||||
print(' Byte-identical consecutive frames mean the screen is static -- that '
|
||||
'is a finding (a hang, or a dialog waiting), not a capture problem.')
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,170 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tier-3 verifier check: does any branch target a `move-result*` instruction?
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
`move-result*` is not an ordinary instruction. The verifier requires it to be the
|
||||
immediate successor of the invoke that produced the value it copies. A branch that
|
||||
lands on a `move-result` therefore bypasses the producer, and the class fails to
|
||||
load with:
|
||||
|
||||
VerifyError: ... copyRes1 v11 <- result0 type=Undefined
|
||||
|
||||
This matters specifically for byte-level patching. Redirecting a branch
|
||||
(`if-*` -> `goto`) is the intuitive way to force one side of a condition, and it
|
||||
is the way that silently creates this violation. Replacing the branch with a pair
|
||||
of `nop`s cannot: it removes a control-flow edge instead of adding one.
|
||||
|
||||
The check must be done on the control-flow graph, not linearly. A "is the
|
||||
previous instruction the producer?" scan over the instruction list reports clean
|
||||
on a method that has a branch jumping straight onto a `move-result`, because the
|
||||
producer does sit immediately before it in the stream. Only asking "does ANY
|
||||
branch target this offset" sees the problem.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python dex_check_verifier.py app.apk # whole image
|
||||
python dex_check_verifier.py app.apk --class 'Lcom/ex/Foo;'
|
||||
python dex_check_verifier.py before.dex after.dex # compare two builds
|
||||
|
||||
Exit codes: 0 = no findings, 1 = findings present, 2 = usage/parse error. The
|
||||
non-zero-on-findings convention is deliberate so a pipeline can gate on it.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from dexutil import IF_TEST, IF_TESTZ, MOVE_RESULT_OPS, load_dex # noqa: E402
|
||||
|
||||
|
||||
def analyse_method(dex, code_off):
|
||||
"""Return (findings, clean, detail) for one method body.
|
||||
|
||||
findings: list of (branch_offset, move_result_offset, is_conditional)
|
||||
clean: whether the decode consumed exactly insns_size units
|
||||
detail: human string when not clean
|
||||
|
||||
ONLY conditional branches are reported. An unconditional `goto` that lands on
|
||||
a `move-result` is a normal compiler-generated merge point: the producer runs
|
||||
on the incoming path and the goto is a *forward* jump to the shared epilogue,
|
||||
so nothing is bypassed. Reporting those as violations makes the tool unusable
|
||||
-- on a real sample they outnumber genuine findings by 20:1 and are pure
|
||||
compiler output.
|
||||
|
||||
The real hazard is narrower: redirecting a CONDITIONAL branch so that it lands
|
||||
directly on a `move-result`. That skips the producing invoke on the taken path.
|
||||
"""
|
||||
insns = list(dex.decode(code_off))
|
||||
info = dex.code_info(code_off)
|
||||
expected = info["insns_off"] + info["insns_size"] * 2
|
||||
ended = (insns[-1]["off"] + insns[-1]["units"] * 2) if insns else info["insns_off"]
|
||||
clean = (ended == expected)
|
||||
detail = "" if clean else ("decode ended at 0x%x, method body ends at 0x%x"
|
||||
% (ended, expected))
|
||||
|
||||
move_results = {i["off"] for i in insns if i["op"] in MOVE_RESULT_OPS}
|
||||
findings = []
|
||||
for i in insns:
|
||||
if i["op"] not in IF_TEST and i["op"] not in IF_TESTZ:
|
||||
continue
|
||||
t = dex.branch_target(i)
|
||||
if t is not None and t in move_results:
|
||||
findings.append((i["off"], t, True))
|
||||
return findings, clean, detail
|
||||
|
||||
|
||||
def scan_image(path, entry, cls_filter):
|
||||
dex, entry_name = load_dex(path, entry)
|
||||
problems = dex.check()
|
||||
if problems:
|
||||
for p in problems:
|
||||
print(" [structure] %s" % p)
|
||||
raise RuntimeError("structurally broken: %s" % path)
|
||||
|
||||
total_methods = 0
|
||||
total_findings = 0
|
||||
unclean = 0
|
||||
for fqcn in dex.class_names():
|
||||
if cls_filter and cls_filter not in fqcn:
|
||||
continue
|
||||
try:
|
||||
methods = list(dex.methods_of(fqcn))
|
||||
except Exception:
|
||||
continue
|
||||
for _section, _midx, _c, name, desc, code_off in methods:
|
||||
if code_off == 0:
|
||||
continue
|
||||
total_methods += 1
|
||||
findings, clean, detail = analyse_method(dex, code_off)
|
||||
if not clean:
|
||||
unclean += 1
|
||||
if findings:
|
||||
total_findings += len(findings)
|
||||
print(" VIOLATION %s.%s%s" % (fqcn, name, desc))
|
||||
for b_off, mr_off, _cond in findings:
|
||||
b_insn = dex.insn_at(code_off, b_off)
|
||||
mr_insn = dex.insn_at(code_off, mr_off)
|
||||
print(" conditional branch 0x%x (%s)"
|
||||
% (b_off, dex.describe(b_insn)))
|
||||
print(" -> move-result 0x%x (%s)"
|
||||
% (mr_off, dex.describe(mr_insn)))
|
||||
print(" the producing invoke is bypassed on the taken "
|
||||
"path: the class will fail to load")
|
||||
return {"methods": total_methods, "findings": total_findings,
|
||||
"unclean": unclean, "entry": entry_name}
|
||||
|
||||
|
||||
def main(argv):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Detect branch-into-move-result verifier violations.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__.split("Usage")[-1][:800])
|
||||
ap.add_argument("targets", nargs="+", help="one .dex/.apk, or two to compare")
|
||||
ap.add_argument("--entry", help="archive member (default: first classes*.dex)")
|
||||
ap.add_argument("--class", dest="cls", help="substring filter on the class name")
|
||||
args = ap.parse_args(argv[1:])
|
||||
|
||||
results = []
|
||||
for path in args.targets:
|
||||
print("== %s" % path)
|
||||
try:
|
||||
stats = scan_image(path, args.entry, args.cls)
|
||||
except Exception as exc:
|
||||
print(" error: %s" % exc)
|
||||
return 2
|
||||
results.append(stats)
|
||||
print(" methods checked: %d violations: %d unclean decodes: %d"
|
||||
% (stats["methods"], stats["findings"], stats["unclean"]))
|
||||
print("")
|
||||
|
||||
if len(results) >= 2:
|
||||
before, after = results[0], results[1]
|
||||
print("== comparison")
|
||||
print(" violations %d -> %d" % (before["findings"], after["findings"]))
|
||||
if after["findings"] > before["findings"]:
|
||||
print(" REGRESSION: your patch introduced %d new violation(s). A "
|
||||
"branch now lands on a move-result." % (after["findings"] - before["findings"]))
|
||||
print(" Fix: replace the branch with `nop`s (removes an edge) instead "
|
||||
"of redirecting it (adds one).")
|
||||
return 1
|
||||
if before["findings"] and after["findings"] == before["findings"]:
|
||||
print(" note: violations exist in BOTH builds, so they are pre-existing "
|
||||
"and not caused by the patch. Do not 'fix' them as part of this task.")
|
||||
|
||||
if any(r["unclean"] for r in results):
|
||||
print("== WARNING: unclean decodes reported above. Offsets from those "
|
||||
"methods are unreliable -- widen the instruction format table or "
|
||||
"cross-check against a disassembler before trusting a patch offset.")
|
||||
|
||||
if any(r["findings"] for r in results):
|
||||
return 1
|
||||
print("== no branch-into-move-result violations")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,278 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Locate an instruction by decoded semantics and report its exact byte offset.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
You know what the instruction *does* ("a branch on the config boolean whose target
|
||||
starts the countdown") but you need the *byte offset* to patch it. Two shortcuts
|
||||
fail:
|
||||
|
||||
* baksmali listings give smali line numbers, not code offsets. `.line` tracks
|
||||
source lines, breaks 1:N, and cannot be converted reliably.
|
||||
* a hard-coded offset dies the moment the sample is rebuilt, and a wrong offset
|
||||
in an equal-length patch corrupts the neighbouring instruction silently.
|
||||
|
||||
So this tool decodes the method and filters on semantics. It always prints the
|
||||
surrounding instructions so you can see which side of a branch you are about to
|
||||
change -- the polarity mistake is the most common way a patch does the exact
|
||||
opposite of the goal and still starts cleanly.
|
||||
|
||||
Filter language (all optional; combined with AND, comma = OR within a repeated
|
||||
flag):
|
||||
|
||||
NAME substring match on the class, method or string operand
|
||||
@Member field or method name operand equals/contains Member
|
||||
=literal a const/4 literal equals this integer
|
||||
#Class/member a branch whose TARGET reads this field of this class
|
||||
>kind instruction kind: if-test if-testz invoke return
|
||||
move-result const4 nop goto switch field sfield
|
||||
$name the ENCLOSING method name must contain this
|
||||
|
||||
Examples
|
||||
--------
|
||||
# a boolean gate whose taken path reads Cfg.enabled
|
||||
python dex_find_insn.py app.apk --class 'Lcom/example/SplashActivity;' \\
|
||||
--method u --filter '#Lcom/example/Cfg;*enabled'
|
||||
|
||||
# every invoke of a method named like goHome, with 3 lines of context
|
||||
python dex_find_insn.py app.apk --filter '@goHome' --kind invoke -C 3
|
||||
|
||||
# where a specific literal is compared
|
||||
python dex_find_insn.py app.apk --class 'Lcom/example/PlayerActivity;' --filter '=1' --kind if-testz
|
||||
|
||||
Exit codes: 0 = at least one hit, 1 = no hits (a finding: say so, do not retry the
|
||||
same filter), 2 = usage error.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from dexutil import ( # noqa: E402
|
||||
Dex, IF_TEST, IF_TESTZ, INVOKE_OPS, MOVE_RESULT_OPS, load_dex,
|
||||
)
|
||||
|
||||
_KIND_MAP = {
|
||||
"if-test": IF_TEST,
|
||||
"if-testz": IF_TESTZ,
|
||||
"invoke": INVOKE_OPS,
|
||||
"return": (0x0F, 0x10, 0x11),
|
||||
"move-result": MOVE_RESULT_OPS,
|
||||
"const4": (0x12,),
|
||||
"const16": (0x13,),
|
||||
"goto": (0x27, 0x28, 0x29),
|
||||
"switch": (0x2A, 0x2B),
|
||||
"nop": (0x00,),
|
||||
}
|
||||
_FIELD_READ = tuple(range(0x52, 0x59))
|
||||
_FIELD_WRITE = tuple(range(0x59, 0x60))
|
||||
_SFIELD = tuple(range(0x60, 0x6E))
|
||||
_KIND_MAP["field"] = _FIELD_READ
|
||||
_KIND_MAP["field-write"] = _FIELD_WRITE
|
||||
_KIND_MAP["sfield"] = _SFIELD
|
||||
|
||||
|
||||
class Filter(object):
|
||||
"""One compiled filter expression."""
|
||||
|
||||
def __init__(self, text):
|
||||
self.raw = text
|
||||
self.kind = text[1:] if text.startswith(">") else None
|
||||
self.member = text[1:] if text.startswith("@") else None
|
||||
self.literal = int(text[1:]) if text.startswith("=") else None
|
||||
self.target_field = None
|
||||
self.target_class = None
|
||||
self.enclosing = text[1:] if text.startswith("$") else None
|
||||
self.text = None
|
||||
if text.startswith("#"):
|
||||
spec = text[1:]
|
||||
if "/" in spec and ("*" in spec):
|
||||
cls, mem = spec.split("*", 1)
|
||||
self.target_class, self.target_field = cls, mem
|
||||
elif "*" in spec:
|
||||
self.target_field = spec.split("*", 1)[1]
|
||||
else:
|
||||
self.target_class = spec
|
||||
elif not any(text.startswith(p) for p in ("@", "=", ">", "$")):
|
||||
self.text = text
|
||||
|
||||
def describe(self):
|
||||
return self.raw
|
||||
|
||||
|
||||
def _operand_text(dex, insn):
|
||||
"""All names this instruction mentions (class, method, field, string)."""
|
||||
parts = []
|
||||
op, pos = insn["op"], insn["off"]
|
||||
if op in INVOKE_OPS:
|
||||
cls, nm, ds = dex.method(int.from_bytes(insn["raw"][2:4], "little"))
|
||||
parts += [cls, nm, ds]
|
||||
elif op in _FIELD_READ + _FIELD_WRITE + _SFIELD:
|
||||
cls, nm, ty = dex.field(int.from_bytes(insn["raw"][2:4], "little"))
|
||||
parts += [cls, nm, ty]
|
||||
elif op in (0x19, 0x1A):
|
||||
parts.append(dex.string_safe(int.from_bytes(insn["raw"][2:4], "little")))
|
||||
return parts
|
||||
|
||||
|
||||
def _matches(dex, insn, filters, code_off):
|
||||
"""All filters must hold (AND)."""
|
||||
for f in filters:
|
||||
if f.kind:
|
||||
ops = _KIND_MAP.get(f.kind)
|
||||
if ops is None:
|
||||
raise ValueError("unknown kind: %s" % f.kind)
|
||||
if insn["op"] not in ops:
|
||||
return False
|
||||
if f.literal is not None:
|
||||
if insn["op"] != 0x12:
|
||||
return False
|
||||
val = insn["raw"][1] & 0xF
|
||||
if val > 7:
|
||||
val -= 16
|
||||
if val != f.literal:
|
||||
return False
|
||||
if f.member:
|
||||
if insn["op"] not in INVOKE_OPS:
|
||||
return False
|
||||
_cls, nm, _ds = dex.method(int.from_bytes(insn["raw"][2:4], "little"))
|
||||
if f.member not in nm:
|
||||
return False
|
||||
if f.target_class or f.target_field:
|
||||
tgt = dex.branch_target(insn)
|
||||
if tgt is None:
|
||||
return False
|
||||
t_insn = dex.insn_at(code_off, tgt)
|
||||
if t_insn is None or not (0x52 <= t_insn["op"] <= 0x58):
|
||||
return False
|
||||
cls, nm, _ty = dex.field(int.from_bytes(t_insn["raw"][2:4], "little"))
|
||||
if f.target_class and f.target_class != cls:
|
||||
return False
|
||||
if f.target_field and f.target_field not in nm:
|
||||
return False
|
||||
if f.text:
|
||||
hay = " ".join(_operand_text(dex, insn))
|
||||
if f.text.lower() not in hay.lower():
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def scan(dex, filters, kind_only=None, class_pattern=None, method_pattern=None,
|
||||
limit=None):
|
||||
"""Yield (fqcn, method_name, descriptor, code_off, insn, all_insns, idx)."""
|
||||
hits = 0
|
||||
cls_re = re.compile(class_pattern) if class_pattern else None
|
||||
mth_re = re.compile(method_pattern) if method_pattern else None
|
||||
|
||||
for fqcn in dex.class_names():
|
||||
if cls_re and not cls_re.search(fqcn):
|
||||
continue
|
||||
try:
|
||||
methods = list(dex.methods_of(fqcn))
|
||||
except Exception:
|
||||
continue
|
||||
for section, midx, _c, name, desc, code_off in methods:
|
||||
if code_off == 0:
|
||||
continue
|
||||
if mth_re and not mth_re.search(name):
|
||||
continue
|
||||
for f in filters:
|
||||
if f.enclosing and f.enclosing not in name:
|
||||
break
|
||||
insns = list(dex.decode(code_off))
|
||||
# a decode that does not land on the method end means offsets here
|
||||
# cannot be trusted; report the method rather than silent misses
|
||||
info = dex.code_info(code_off)
|
||||
expected = info["insns_off"] + info["insns_size"] * 2
|
||||
clean = (insns[-1]["off"] + insns[-1]["units"] * 2 == expected) if insns else False
|
||||
for i, insn in enumerate(insns):
|
||||
if kind_only and insn["op"] not in _KIND_MAP.get(kind_only, ()):
|
||||
continue
|
||||
if filters and not _matches(dex, insn, filters, code_off):
|
||||
continue
|
||||
yield (fqcn, name, desc, code_off, insn, insns, i, clean)
|
||||
hits += 1
|
||||
if limit and hits >= limit:
|
||||
return
|
||||
|
||||
|
||||
def main(argv):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Find an instruction by decoded semantics; print its exact "
|
||||
"byte offset and surrounding context.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__.split("Examples")[-1][:1500])
|
||||
ap.add_argument("target", help=".dex or .apk/.zip (use --entry to pick a dex)")
|
||||
ap.add_argument("--entry", help="archive member (default: first classes*.dex)")
|
||||
ap.add_argument("--class", dest="cls", help="regex over the class descriptor")
|
||||
ap.add_argument("--method", help="regex over the method name")
|
||||
ap.add_argument("--filter", action="append", default=[],
|
||||
help="filter expression, repeatable (AND across, OR within a flag "
|
||||
"list). See the module docstring for the language.")
|
||||
ap.add_argument("--kind", help="restrict to one instruction kind (see docstring)")
|
||||
ap.add_argument("-C", "--context", type=int, default=2,
|
||||
help="instructions of context to print (default 2)")
|
||||
ap.add_argument("--limit", type=int, default=40, help="max hits (default 40)")
|
||||
ap.add_argument("--show-unclean", action="store_true",
|
||||
help="also print hits from methods whose decode did not align")
|
||||
args = ap.parse_args(argv[1:])
|
||||
|
||||
dex, entry = load_dex(args.target, args.entry)
|
||||
problems = dex.check()
|
||||
if problems:
|
||||
print("structural problems in %s (%s):" % (args.target, entry))
|
||||
for p in problems:
|
||||
print(" - %s" % p)
|
||||
return 2
|
||||
|
||||
filters = [Filter(a) for a in args.filter]
|
||||
print("== %s (%s) %d classes" % (args.target, entry,
|
||||
dex.header["class_defs_size"]))
|
||||
if filters:
|
||||
print(" filters: %s" % ", ".join(f.describe() for f in filters))
|
||||
|
||||
found = 0
|
||||
for fqcn, name, desc, code_off, insn, insns, idx, clean in scan(
|
||||
dex, filters, args.kind, args.cls, args.method, args.limit):
|
||||
if not clean and not args.show_unclean:
|
||||
print("\n!! %s.%s%s decode did not align; offsets unreliable, "
|
||||
"skipping (use --show-unclean to see anyway)"
|
||||
% (fqcn, name, desc))
|
||||
continue
|
||||
found += 1
|
||||
print("\n-- %s.%s%s" % (fqcn, name, desc))
|
||||
print(" code_off=0x%x insns_off=0x%x (decode clean: %s)"
|
||||
% (code_off, code_off + 16, clean))
|
||||
lo = max(0, idx - args.context)
|
||||
hi = min(len(insns), idx + args.context + 1)
|
||||
for j in range(lo, hi):
|
||||
marker = " <== HIT" if j == idx else ""
|
||||
print(" %s%s" % (dex.describe(insns[j]), marker))
|
||||
print(" >>> file offset of the instruction: 0x%x" % insn["off"])
|
||||
if insn["op"] in IF_TEST or insn["op"] in IF_TESTZ:
|
||||
tgt = dex.branch_target(insn)
|
||||
t_insn = dex.insn_at(code_off, tgt)
|
||||
print(" >>> branch TAKEN target 0x%x: %s"
|
||||
% (tgt, dex.describe(t_insn) if t_insn else "?"))
|
||||
nxt = insn["off"] + insn["units"] * 2
|
||||
n_insn = dex.insn_at(code_off, nxt)
|
||||
print(" >>> branch FALL-THROUGH 0x%x: %s"
|
||||
% (nxt, dex.describe(n_insn) if n_insn else "?"))
|
||||
print(" >>> decide which side you want BEFORE patching; a wrong "
|
||||
"polarity still starts cleanly")
|
||||
|
||||
print("\n== %d hit(s)" % found)
|
||||
if not found:
|
||||
print(" no match. Do not retry the same filter: either the decode is "
|
||||
"unclean (see --show-unclean) or the behaviour lives in another "
|
||||
"layer (native / another dex / another class).")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,436 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Apply equal-length byte patches to a dex, with structural and verifier checks.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
The cheap way to change behaviour is to rewrite a few bytes in place rather than
|
||||
rebuild a method body. It is cheap for a reason: nothing moves, so no try/catch
|
||||
block, debug-info pointer or branch displacement can be invalidated by
|
||||
construction. But it has four failure modes that all produce a build which looks
|
||||
fine and misbehaves, and this tool exists to make each one impossible to skip:
|
||||
|
||||
1. **Same-length is not enough** -- the replacement must decode to a valid
|
||||
instruction sequence of exactly the same length. A 4-byte branch replaced by
|
||||
a 4-byte branch is fine; a 2-byte branch replaced by a 4-byte instruction
|
||||
corrupts the next instruction.
|
||||
2. **Polarity** -- the overwhelmingly common error is patching the wrong side of
|
||||
a condition. So a spec can *require* the instruction that follows the edit,
|
||||
and this tool aborts if it is not the one you named. Name the fall-through
|
||||
you expect and the mistake cannot survive.
|
||||
3. **Verifier legality** -- a branch whose target is a `move-result*` bypasses
|
||||
the producing invoke and the class fails to load (`VerifyError`). Checked
|
||||
here before writing.
|
||||
4. **dex header staleness** -- the checksum and signature fields must be
|
||||
recomputed, signature FIRST (the checksum covers it). Skipping this makes
|
||||
Android fall back to interpreting the dex, which surfaces as an unrelated
|
||||
ClassNotFoundException at startup.
|
||||
|
||||
Spec format (JSON; a list, or {"patches": [...]}):
|
||||
|
||||
[{
|
||||
"name": "human label",
|
||||
"reason": "why this edit is correct",
|
||||
"class": "Lpkg/Name;",
|
||||
"method": "someMethod",
|
||||
"desc": "(Lpkg/Arg;)V", // optional: disambiguates overloads
|
||||
"match": {"kind": "if-testz", "target_reads_field_of": "Lpkg/Cfg;",
|
||||
"target_field": "enabled"},
|
||||
"expect_next": {"kind": "invoke", "target_method": "goHome"},
|
||||
"replace": {"kind": "nops"} // or {"bytes": "28060000"}
|
||||
}]
|
||||
|
||||
`match` selects the instruction; `expect_next` (optional but strongly advised)
|
||||
asserts what immediately follows, which is what pins the polarity; `replace`
|
||||
gives the bytes. Run with --dry-run first: it prints the match, the neighbours,
|
||||
the verifier verdict and the predicted byte diff without writing anything.
|
||||
|
||||
Exit codes: 0 ok, 1 failure (structure, match, verifier, or self-verify), 2 usage.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from dexutil import ( # noqa: E402
|
||||
Dex, IF_TEST, IF_TESTZ, INVOKE_OPS, MOVE_RESULT_OPS, RETURN_OPS,
|
||||
fix_dex_header, load_dex, verify_dex_header,
|
||||
)
|
||||
|
||||
|
||||
def _kind_of(insn):
|
||||
"""Coarse semantic kind used by spec matching."""
|
||||
op = insn["op"]
|
||||
if op in IF_TEST:
|
||||
return "if-test"
|
||||
if op in IF_TESTZ:
|
||||
return "if-testz"
|
||||
if op in INVOKE_OPS:
|
||||
return "invoke"
|
||||
if op in RETURN_OPS:
|
||||
return "return"
|
||||
if op in MOVE_RESULT_OPS:
|
||||
return "move-result"
|
||||
if op == 0x12:
|
||||
return "const4"
|
||||
return insn["name"]
|
||||
|
||||
|
||||
def _describe_neighbours(dex, code_off, target_off, span=4):
|
||||
"""Render up to `span` instructions before and after an offset."""
|
||||
insns = list(dex.decode(code_off))
|
||||
idx = None
|
||||
for i, insn in enumerate(insns):
|
||||
if insn["off"] == target_off:
|
||||
idx = i
|
||||
break
|
||||
if idx is None:
|
||||
return [], None, []
|
||||
before = insns[max(0, idx - span):idx]
|
||||
after = insns[idx + 1:idx + 1 + span]
|
||||
return before, insns[idx], after
|
||||
|
||||
|
||||
def _matches(dex, insn, crit, code_off=None):
|
||||
"""Does one instruction satisfy a match/expect criterion?"""
|
||||
if crit is None:
|
||||
return True
|
||||
want = crit.get("kind")
|
||||
if want and _kind_of(insn) != want:
|
||||
return False
|
||||
|
||||
if want in ("if-test", "if-testz"):
|
||||
treg = crit.get("target_reg")
|
||||
if treg is not None:
|
||||
regs, _ = dex.branch_regs(insn)
|
||||
if treg not in regs:
|
||||
return False
|
||||
# The usual way to pin "this is the gate for config X": require that the
|
||||
# branch TARGET instruction reads a field of a named class. This is what
|
||||
# distinguishes the gate you want from the other branches in the method.
|
||||
fcls = crit.get("target_reads_field_of")
|
||||
fname = crit.get("target_field")
|
||||
if fcls or fname:
|
||||
if code_off is None:
|
||||
return False
|
||||
tgt = dex.branch_target(insn)
|
||||
t_insn = dex.insn_at(code_off, tgt)
|
||||
if t_insn is None:
|
||||
return False
|
||||
if not (0x52 <= t_insn["op"] <= 0x58):
|
||||
return False
|
||||
fidx = struct.unpack_from("<H", t_insn["raw"], 2)[0]
|
||||
cls, nm, _ty = dex.field(fidx)
|
||||
if fcls and cls != fcls:
|
||||
return False
|
||||
if fname and nm != fname:
|
||||
return False
|
||||
|
||||
if want == "const4":
|
||||
lit = crit.get("literal")
|
||||
if lit is not None:
|
||||
raw = insn["raw"][1]
|
||||
val = raw & 0xF
|
||||
if val > 7:
|
||||
val -= 16
|
||||
if val != lit:
|
||||
return False
|
||||
|
||||
if want == "invoke":
|
||||
midx = struct.unpack_from("<H", insn["raw"], 2)[0]
|
||||
cls, nm, _ds = dex.method(midx)
|
||||
if crit.get("target_class") and crit["target_class"] not in cls:
|
||||
return False
|
||||
if crit.get("target_method") and crit["target_method"] != nm:
|
||||
return False
|
||||
|
||||
if want in ("return", "move-result", "nop"):
|
||||
pass
|
||||
|
||||
raw_eq = crit.get("bytes")
|
||||
if raw_eq and insn["raw"].hex() != raw_eq.replace(" ", "").lower():
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _find_site(dex, code_off, spec):
|
||||
"""Return the single instruction matching spec['match'].
|
||||
|
||||
Exactly one match is required. Ambiguity is treated as failure rather than
|
||||
"take the first": picking the wrong site in a method with several similar
|
||||
branches is a silent, expensive error.
|
||||
"""
|
||||
insns = list(dex.decode(code_off))
|
||||
hits = [i for i in insns if _matches(dex, i, spec.get("match"), code_off)]
|
||||
if len(hits) == 1:
|
||||
return hits[0], insns
|
||||
if not hits:
|
||||
return None, insns
|
||||
|
||||
# allow an ordinal to disambiguate deliberately
|
||||
ordv = spec.get("match", {}).get("ordinal")
|
||||
if ordv is not None and 0 <= ordv < len(hits):
|
||||
return hits[ordv], insns
|
||||
return ("ambiguous", hits), insns
|
||||
|
||||
|
||||
def _verifier_report(dex, code_off, target_off_for_new_edge=None):
|
||||
"""Tier-3 check: does any branch target a move-result instruction?
|
||||
|
||||
`move-result*` must be immediately preceded by its producing invoke. A branch
|
||||
into one bypasses the producer and the class fails to load with
|
||||
`VerifyError ... copyResN v <- result0 type=Undefined`. A linear "is the
|
||||
previous instruction the producer" scan misses this; only the CFG view sees
|
||||
it.
|
||||
"""
|
||||
findings = []
|
||||
for insn in dex.decode(code_off):
|
||||
if insn["op"] not in MOVE_RESULT_OPS:
|
||||
continue
|
||||
tgt = insn["off"]
|
||||
for other in dex.decode(code_off):
|
||||
t = dex.branch_target(other)
|
||||
if t == tgt:
|
||||
findings.append((other["off"], tgt))
|
||||
return findings
|
||||
|
||||
|
||||
def _apply_replace(dex_data, insn, repl):
|
||||
"""Return the new bytes for this instruction, enforcing equal length."""
|
||||
kind = repl.get("kind")
|
||||
if kind == "nops":
|
||||
n = len(insn["raw"])
|
||||
return b"\x00" * n
|
||||
if kind == "bytes":
|
||||
blob = bytes.fromhex(repl["bytes"].replace(" ", ""))
|
||||
return blob
|
||||
raise ValueError("replace must be {'kind':'nops'} or {'kind':'bytes','bytes':...}")
|
||||
|
||||
|
||||
def process_one(dex, data, spec, index, dry_run):
|
||||
"""Apply one spec entry to `data`; return a report dict, or raise."""
|
||||
cls = spec["class"]
|
||||
name = spec["method"]
|
||||
desc = spec.get("desc")
|
||||
label = spec.get("name") or ("patch%d" % index)
|
||||
report = {"name": label, "ok": False}
|
||||
|
||||
if desc:
|
||||
found = dex.find_method(cls, name, desc)
|
||||
if not found:
|
||||
raise RuntimeError("[%s] method not found: %s->%s%s"
|
||||
% (label, cls, name, desc))
|
||||
section, midx, code_off = found
|
||||
else:
|
||||
overloads = dex.find_methods_named(cls, name)
|
||||
if len(overloads) != 1:
|
||||
raise RuntimeError(
|
||||
"[%s] %s->%s has %d overloads; give \"desc\" to disambiguate: %s"
|
||||
% (label, cls, name, len(overloads),
|
||||
[d for _s, _i, d, _c in overloads]))
|
||||
section, midx, dsc, code_off = overloads[0]
|
||||
desc = dsc
|
||||
|
||||
report.update({"class": cls, "method": name, "desc": desc,
|
||||
"section": section, "method_idx": midx, "code_off": code_off})
|
||||
|
||||
site, _insns = _find_site(dex, code_off, spec)
|
||||
if site is None:
|
||||
raise RuntimeError("[%s] no instruction matched spec['match']" % label)
|
||||
if isinstance(site, tuple) and site and site[0] == "ambiguous":
|
||||
cands = ", ".join("0x%x" % h["off"] for h in site[1])
|
||||
raise RuntimeError("[%s] match is ambiguous (%d hits: %s); tighten it or "
|
||||
"set match.ordinal" % (label, len(site[1]), cands))
|
||||
|
||||
before, cur, after = _describe_neighbours(dex, code_off, site["off"])
|
||||
report.update({
|
||||
"site_off": cur["off"],
|
||||
"old_bytes": cur["raw"].hex(),
|
||||
"old_text": dex.describe(cur),
|
||||
"before": [dex.describe(i) for i in before],
|
||||
"after": [dex.describe(i) for i in after],
|
||||
})
|
||||
|
||||
# polarity pin: what must immediately follow?
|
||||
exp = spec.get("expect_next")
|
||||
if exp is not None:
|
||||
nxt = after[0] if after else None
|
||||
if nxt is None or not _matches(dex, nxt, exp, code_off):
|
||||
raise RuntimeError(
|
||||
"[%s] expect_next not satisfied at 0x%x. Expected %r, found %r.\n"
|
||||
" This is the polarity check: the instruction after the branch "
|
||||
"is not the one your spec assumes, so the branch points at the "
|
||||
"wrong side."
|
||||
% (label, cur["off"], exp, dex.describe(nxt) if nxt else None))
|
||||
report["expect_next_ok"] = True
|
||||
|
||||
new = _apply_replace(data, cur, spec.get("replace", {"kind": "nops"}))
|
||||
if len(new) != len(cur["raw"]):
|
||||
raise RuntimeError(
|
||||
"[%s] replacement is %d bytes but the instruction is %d. Equal length "
|
||||
"is required: a longer replacement overwrites the next instruction."
|
||||
% (label, len(new), len(cur["raw"])))
|
||||
report.update({"new_bytes": new.hex(), "size_delta": 0})
|
||||
|
||||
# verifier: simulate the new instruction to see if it adds a branch edge
|
||||
fake = Dex(bytes(data), dex.name)
|
||||
fake_probe = dict(cur)
|
||||
fake_probe["raw"] = new
|
||||
fake_probe["op"] = new[0]
|
||||
report["new_text"] = fake.describe(fake_probe)
|
||||
|
||||
if dry_run:
|
||||
report["ok"] = True
|
||||
return report, data
|
||||
|
||||
data[cur["off"]:cur["off"] + len(new)] = new
|
||||
|
||||
# Re-read the patched dex and confirm the edit landed.
|
||||
#
|
||||
# Compare the BYTE RANGE, not a decoded instruction: replacing a 4-byte branch
|
||||
# with a nop pair yields two 1-unit instructions, so "the instruction at this
|
||||
# offset has length 4" is false even though the patch is perfectly correct.
|
||||
# The byte range is what the patch actually owns.
|
||||
span = slice(cur["off"], cur["off"] + len(new))
|
||||
if bytes(data[span]) != new:
|
||||
raise RuntimeError(
|
||||
"[%s] bytes at 0x%x are %s, expected %s"
|
||||
% (label, cur["off"], bytes(data[span]).hex(), new.hex()))
|
||||
|
||||
patched = Dex(bytes(data), dex.name)
|
||||
insns_after = list(patched.decode(code_off))
|
||||
info = patched.code_info(code_off)
|
||||
expected_end = info["insns_off"] + info["insns_size"] * 2
|
||||
ended = (insns_after[-1]["off"] + insns_after[-1]["units"] * 2
|
||||
if insns_after else info["insns_off"])
|
||||
report["post_instruction_count"] = len(insns_after)
|
||||
report["post_decode_clean"] = (ended == expected_end)
|
||||
|
||||
landed = [i for i in insns_after
|
||||
if cur["off"] <= i["off"] < cur["off"] + len(new)]
|
||||
report["post_text"] = " | ".join(patched.describe(i) for i in landed)
|
||||
|
||||
# the next instruction boundary after the replaced span must still be the
|
||||
# instruction that followed the original one
|
||||
nxt_after = patched.insn_at(code_off, cur["off"] + len(new))
|
||||
report["post_next_text"] = patched.describe(nxt_after) if nxt_after else None
|
||||
if not report["post_decode_clean"]:
|
||||
raise RuntimeError(
|
||||
"[%s] the method no longer decodes to exactly insns_size units after "
|
||||
"the edit -- the replacement changed the instruction stream length. "
|
||||
"Use an equal-length replacement." % label)
|
||||
|
||||
edges = _verifier_report(patched, code_off)
|
||||
report["verifier_edges"] = len(edges)
|
||||
if edges:
|
||||
raise RuntimeError(
|
||||
"[%s] verifier: a branch targets a move-result at %s; the class will "
|
||||
"fail to load (VerifyError). Use a nop pair instead of a branch."
|
||||
% (label, ", ".join("0x%x->0x%x" % e for e in edges)))
|
||||
|
||||
report["ok"] = True
|
||||
return report, data
|
||||
|
||||
|
||||
def main(argv):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Apply equal-length byte patches to a dex, with structural, "
|
||||
"polarity and verifier checks.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__.split("Spec format")[-1][:1200] if __doc__ else None)
|
||||
ap.add_argument("dex", help="input .dex, or an APK/zip (use --entry to pick)")
|
||||
ap.add_argument("--entry", help="archive member to read (default: classes.dex)")
|
||||
ap.add_argument("--spec", required=True, help="JSON spec file")
|
||||
ap.add_argument("-o", "--out", help="output dex path")
|
||||
ap.add_argument("--dry-run", action="store_true",
|
||||
help="report matches and diffs without writing")
|
||||
ap.add_argument("--report", help="write a machine-readable JSON report here")
|
||||
args = ap.parse_args(argv[1:])
|
||||
|
||||
dex, entry = load_dex(args.dex, args.entry) if os.path.exists(args.dex) else (None, None)
|
||||
if dex is None:
|
||||
# allow raw bytes piped through a path that does not exist yet: fail clearly
|
||||
print("error: no such file: %s" % args.dex)
|
||||
return 2
|
||||
|
||||
problems = dex.check()
|
||||
if problems:
|
||||
print("structural problems in %s (%s):" % (args.dex, entry))
|
||||
for p in problems:
|
||||
print(" - %s" % p)
|
||||
print("refusing to patch a source that does not parse cleanly")
|
||||
return 1
|
||||
|
||||
with open(args.spec, encoding="utf-8") as fh:
|
||||
spec_doc = json.load(fh)
|
||||
specs = spec_doc["patches"] if isinstance(spec_doc, dict) else spec_doc
|
||||
|
||||
ok_before = verify_dex_header(dex.d)
|
||||
print("== source: %s (%s, %d bytes)" % (args.dex, entry, len(dex.d)))
|
||||
print(" header before: checksum_ok=%s signature_ok=%s"
|
||||
% ok_before)
|
||||
if not all(ok_before):
|
||||
print(" note: the SOURCE dex header is already inconsistent. Some")
|
||||
print(" producers ship a zeroed signature field; the patch will fix it.")
|
||||
|
||||
data = bytearray(dex.d)
|
||||
reports = []
|
||||
for i, spec in enumerate(specs):
|
||||
rep, data = process_one(dex, data, spec, i, args.dry_run)
|
||||
reports.append(rep)
|
||||
print("\n-- [%s] %s.%s%s" % (rep["name"], rep["class"], rep["method"], rep["desc"]))
|
||||
print(" site 0x%x %s" % (rep["site_off"], rep["old_text"]))
|
||||
for line in rep["before"][-2:]:
|
||||
print(" (before) %s" % line)
|
||||
print(" -> %s" % rep.get("new_text"))
|
||||
for line in rep["after"][:2]:
|
||||
print(" (after) %s" % line)
|
||||
if rep.get("expect_next_ok"):
|
||||
print(" polarity: expect_next satisfied")
|
||||
|
||||
if args.dry_run:
|
||||
print("\n== dry run: nothing written")
|
||||
if args.report:
|
||||
with open(args.report, "w", encoding="utf-8") as fh:
|
||||
json.dump(reports, fh, indent=2)
|
||||
print(" report: %s" % args.report)
|
||||
return 0
|
||||
|
||||
header = fix_dex_header(data)
|
||||
ok_after = verify_dex_header(data)
|
||||
print("\n== header checksum 0x%08x -> 0x%08x" % (header["before_checksum"],
|
||||
header["after_checksum"]))
|
||||
print(" signature %s -> %s"
|
||||
% (header["before_signature"].hex()[:16],
|
||||
header["after_signature"].hex()[:16]))
|
||||
print(" self-verify: checksum_ok=%s signature_ok=%s" % ok_after)
|
||||
if not all(ok_after):
|
||||
print("FATAL: header does not self-verify; nothing written")
|
||||
return 1
|
||||
|
||||
out = args.out
|
||||
if not out:
|
||||
root, ext = os.path.splitext(args.dex)
|
||||
out = root + ".patched" + (ext or ".dex")
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(bytes(data))
|
||||
print("== wrote %s (%d bytes, delta %d)"
|
||||
% (out, len(data), len(data) - len(dex.d)))
|
||||
|
||||
if args.report:
|
||||
with open(args.report, "w", encoding="utf-8") as fh:
|
||||
json.dump(reports, fh, indent=2)
|
||||
print(" report: %s" % args.report)
|
||||
|
||||
print("\nNext: repack (scripts/repack.py) then sign with apksigner, then run "
|
||||
"the device + screen checks in references/verification.md.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -81,6 +81,13 @@ def _order_ok(data, off, ob, nb):
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) > 1 and sys.argv[1] in ('-h', '--help'):
|
||||
print(__doc__)
|
||||
return 0
|
||||
if len(sys.argv) < 5:
|
||||
print('error: needs 4 arguments, got %d\n' % (len(sys.argv) - 1))
|
||||
print(__doc__)
|
||||
return 2
|
||||
src, dst, old, new = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
|
||||
ob, nb = old.encode('utf-8'), new.encode('utf-8')
|
||||
if len(ob) != len(nb):
|
||||
|
||||
@@ -0,0 +1,582 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Minimal, dependency-free dex reader: structure walk + exact instruction decode.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
Locating the exact byte offset of one instruction is where byte-level patching
|
||||
either works or wastes an afternoon. Two shortcuts do not work and are worth
|
||||
naming so nobody re-derives them:
|
||||
|
||||
* Reconstructing offsets from a baksmali listing. `.line` directives track
|
||||
SOURCE lines, and one source line can span several instructions, so the
|
||||
values repeat and do not map 1:1 onto code offsets.
|
||||
* Matching a guessed byte sequence. Encodings vary with register numbers and
|
||||
operand widths, and a plausible-looking opcode can belong to a different
|
||||
format than you assume (0x38 is `if-test`, 22t/4 bytes -- not `if-testz`).
|
||||
|
||||
So: walk class -> class_data_item -> method -> code_item, then decode forward with
|
||||
a complete format table, and match on decoded semantics instead of bytes.
|
||||
|
||||
Three details in the walk that silently produce wrong answers:
|
||||
|
||||
* `class_data_item` member indices are DELTAS against the previous entry in the
|
||||
same list. Reading a raw uleb as an absolute index yields real-looking wrong
|
||||
members.
|
||||
* dalvik encodes switch/array payloads as `00 <ident> <size>` with ident 1..3.
|
||||
A plain `00 00` is an ordinary one-unit nop; treating every 00 as a payload
|
||||
swallows the next instruction and desynchronises the rest of the method.
|
||||
* The header field order is easy to mis-remember. See OFFSETS below, and always
|
||||
sanity-check the parsed sizes against the file.
|
||||
|
||||
Pure standard library. Import it, or run it for a per-method dump:
|
||||
|
||||
python dexutil.py <dex-or-apk> <Class/Name;> <method> <descriptor>
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import struct
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
# Verified dex header offsets. After magic(8) + checksum(4) + signature(20):
|
||||
OFFSETS = {
|
||||
"file_size": 0x20, "header_size": 0x24, "endian_tag": 0x28,
|
||||
"link_size": 0x2C, "link_off": 0x30, "map_off": 0x34,
|
||||
"string_ids_size": 0x38, "string_ids_off": 0x3C,
|
||||
"type_ids_size": 0x40, "type_ids_off": 0x44,
|
||||
"proto_ids_size": 0x48, "proto_ids_off": 0x4C,
|
||||
"field_ids_size": 0x50, "field_ids_off": 0x54,
|
||||
"method_ids_size": 0x58, "method_ids_off": 0x5C,
|
||||
"class_defs_size": 0x60, "class_defs_off": 0x64,
|
||||
"data_size": 0x68, "data_off": 0x6C,
|
||||
}
|
||||
|
||||
OP_NAMES = {
|
||||
0x00: "nop", 0x01: "move", 0x02: "move/from16", 0x03: "move/16",
|
||||
0x04: "move-wide", 0x05: "move-wide/from16", 0x06: "move-wide/16",
|
||||
0x07: "move-object", 0x08: "move-object/from16", 0x09: "move-object/16",
|
||||
0x0A: "move-result", 0x0B: "move-result-wide", 0x0C: "move-result-object",
|
||||
0x0D: "move-exception", 0x0E: "return-void", 0x0F: "return",
|
||||
0x10: "return-wide", 0x11: "return-object", 0x12: "const/4",
|
||||
0x13: "const/16", 0x14: "const", 0x15: "const/high16",
|
||||
0x16: "const-wide/high16", 0x17: "const-wide/16", 0x18: "const-wide/32",
|
||||
0x19: "const-string", 0x1A: "const-string/jumbo", 0x1B: "const-class",
|
||||
0x1C: "monitor-enter", 0x1D: "monitor-exit", 0x1E: "check-cast",
|
||||
0x1F: "instance-of", 0x20: "array-length", 0x21: "new-instance",
|
||||
0x22: "new-array", 0x23: "filled-new-array", 0x24: "filled-new-array/range",
|
||||
0x25: "fill-array-data", 0x26: "throw", 0x27: "goto",
|
||||
0x28: "goto/16", 0x29: "goto/32", 0x2A: "packed-switch",
|
||||
0x2B: "sparse-switch",
|
||||
0x32: "if-eq", 0x33: "if-ne", 0x34: "if-lt", 0x35: "if-ge",
|
||||
0x36: "if-gt", 0x37: "if-le",
|
||||
0x38: "if-eqz", 0x39: "if-nez", 0x3A: "if-ltz", 0x3B: "if-gez",
|
||||
0x3C: "if-gtz", 0x3D: "if-lez",
|
||||
0x52: "iget", 0x53: "iget-wide", 0x54: "iget-object", 0x55: "iget-boolean",
|
||||
0x56: "iget-byte", 0x57: "iget-char", 0x58: "iget-short",
|
||||
0x59: "iput", 0x5A: "iput-wide", 0x5B: "iput-object", 0x5C: "iput-boolean",
|
||||
0x5D: "iput-byte", 0x5E: "iput-char", 0x5F: "iput-short",
|
||||
0x60: "sget", 0x61: "sget-wide", 0x62: "sget-object", 0x63: "sget-boolean",
|
||||
0x64: "sget-byte", 0x65: "sget-char", 0x66: "sget-short",
|
||||
0x67: "sput", 0x68: "sput-wide", 0x69: "sput-object", 0x6A: "sput-boolean",
|
||||
0x6B: "sput-byte", 0x6C: "sput-char", 0x6D: "sput-short",
|
||||
0x6E: "invoke-virtual", 0x6F: "invoke-super", 0x70: "invoke-direct",
|
||||
0x71: "invoke-static", 0x72: "invoke-interface",
|
||||
0x74: "invoke-virtual/range", 0x75: "invoke-super/range",
|
||||
0x76: "invoke-direct/range", 0x77: "invoke-static/range",
|
||||
0x78: "invoke-interface/range",
|
||||
}
|
||||
|
||||
# Opcode groups whose operands are a pair of registers plus an int16 offset.
|
||||
IF_TEST = set(range(0x32, 0x38)) # 22t: 2 register nibbles + int16
|
||||
IF_TESTZ = set(range(0x38, 0x3E)) # 21t: 1 register nibble + int16
|
||||
RETURN_OPS = (0x0F, 0x10, 0x11)
|
||||
MOVE_RESULT_OPS = (0x0A, 0x0B, 0x0C)
|
||||
INVOKE_OPS = tuple(range(0x6E, 0x73)) + tuple(range(0x74, 0x79))
|
||||
# opcodes that read an instance field: (dest_reg, object_reg, field_idx)
|
||||
IFIELD_OPS = tuple(range(0x52, 0x59))
|
||||
# opcodes that write an instance field: (value_reg, object_reg, field_idx)
|
||||
PFIELD_OPS = tuple(range(0x59, 0x60))
|
||||
SFIELD_OPS = tuple(range(0x60, 0x6E))
|
||||
|
||||
|
||||
def u16(b, o):
|
||||
return b[o] | (b[o + 1] << 8)
|
||||
|
||||
|
||||
def u32(b, o):
|
||||
return b[o] | (b[o + 1] << 8) | (b[o + 2] << 16) | (b[o + 3] << 24)
|
||||
|
||||
|
||||
def s16(v):
|
||||
return v - 0x10000 if v > 0x7FFF else v
|
||||
|
||||
|
||||
def read_uleb(b, o):
|
||||
"""ULEB128 -> (value, new_offset). Member indices in class_data are deltas."""
|
||||
result = 0
|
||||
shift = 0
|
||||
while True:
|
||||
x = b[o]
|
||||
o += 1
|
||||
result |= (x & 0x7F) << shift
|
||||
if (x & 0x80) == 0:
|
||||
break
|
||||
shift += 7
|
||||
return result, o
|
||||
|
||||
|
||||
def insn_units(op, data, pos, end):
|
||||
"""Instruction length in 16-bit code units.
|
||||
|
||||
One wrong width desynchronises everything after it, and the failure is quiet:
|
||||
the decode keeps producing plausible-looking instructions, just shifted, so
|
||||
every offset you derive from that point is wrong. Two width traps are common
|
||||
enough to call out:
|
||||
|
||||
* `0x32`-`0x3D` (if-test 22t / if-testz 21t) are **2** units, not 1. Treating
|
||||
them as 1 invents a fake second instruction at every branch.
|
||||
* `0x1A` (const-string/jumbo) is emitted as a **4-byte** form (op, register,
|
||||
uint16 string index) by real toolchains and read that way by ART and
|
||||
baksmali, even though the reference format for the opcode is 31c. Counting
|
||||
it as 3 units is the single most costly width error -- it shifts the rest
|
||||
of the method by one unit per occurrence.
|
||||
* `0x28` is `goto` (10t, **1** unit), `0x29` is `goto/16` (2 units) and
|
||||
`0x2A` is `goto/32` (3 units). Do not read 0x28 as a two-unit form.
|
||||
|
||||
Whenever a decode is used to derive a patch offset, assert that the walk ends
|
||||
exactly on `insns_off + insns_size*2`. See `decode_all`.
|
||||
"""
|
||||
if op == 0x00:
|
||||
# payload nop: `00 <ident> <size>` with ident 1..3; a plain `00 00` is a
|
||||
# one-unit nop. Never treat every 00 as a payload.
|
||||
if pos + 4 <= end and (data[pos + 1] & 0xFF) in (0x01, 0x02, 0x03):
|
||||
return 1 + int.from_bytes(data[pos + 2:pos + 4], "little")
|
||||
return 1
|
||||
# -- 1 unit -------------------------------------------------------------
|
||||
if op in (0x01, 0x04, 0x07, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10,
|
||||
0x11, 0x12, # 12x / 11x / 11n / 10x
|
||||
0x1C, 0x1D, 0x26, 0x27): # monitor-enter/exit, throw, goto
|
||||
return 1
|
||||
if 0x7B <= op <= 0x8F: # 12x unop
|
||||
return 1
|
||||
if 0xB0 <= op <= 0xCF: # 12x binop/2addr
|
||||
return 1
|
||||
if 0xE3 <= op <= 0xFC: # 10x family
|
||||
return 1
|
||||
# -- 2 units ------------------------------------------------------------
|
||||
if op in (0x02, 0x05, 0x08): # 22x move/from16 variants
|
||||
return 2
|
||||
if op in (0x13, 0x15, 0x16, 0x17, 0x19, 0x1A, 0x1B, 0x1E, 0x1F,
|
||||
0x20, 0x21, 0x23, 0x24): # 21s/21h/21c/22c/35c/3rc
|
||||
return 2
|
||||
if 0x2C <= op <= 0x31: # 23x cmp
|
||||
return 2
|
||||
if op in IF_TEST or op in IF_TESTZ: # 22t / 21t <-- 2, not 1
|
||||
return 2
|
||||
if 0x44 <= op <= 0x51: # 23x aget/aput
|
||||
return 2
|
||||
if 0x52 <= op <= 0x5F: # 22c iget/iput
|
||||
return 2
|
||||
if 0x60 <= op <= 0x6D: # 21c sget/sput
|
||||
return 2
|
||||
if 0x90 <= op <= 0xAF: # 23x binop
|
||||
return 2
|
||||
if 0xD0 <= op <= 0xD7: # 22s binop/lit16
|
||||
return 2
|
||||
if 0xD8 <= op <= 0xE2: # 22b binop/lit8
|
||||
return 2
|
||||
if op == 0x29: # goto/16 (20t)
|
||||
return 2
|
||||
# -- 3 units ------------------------------------------------------------
|
||||
if op in (0x03, 0x06, 0x09, # 32x
|
||||
0x14, 0x18, # 31i const, const-wide/32
|
||||
0x22, 0x25, # 22c new-array, 31t fill-array
|
||||
0x2A, 0x2B, # 31t packed/sparse-switch
|
||||
0x6E, 0x6F, 0x70, 0x71, 0x72, # 35c invoke
|
||||
0x74, 0x75, 0x76, 0x77, 0x78): # 3rc invoke/range
|
||||
return 3
|
||||
# -- payloads / unknown -------------------------------------------------
|
||||
return 1
|
||||
|
||||
|
||||
class Dex(object):
|
||||
"""Structural reader for one dex image."""
|
||||
|
||||
def __init__(self, data, name="classes.dex"):
|
||||
self.d = data
|
||||
self.name = name
|
||||
self.header = {k: u32(data, v) for k, v in OFFSETS.items()}
|
||||
|
||||
# ---- sanity -----------------------------------------------------------
|
||||
def check(self):
|
||||
"""Return a list of structural complaints; empty means it parses."""
|
||||
problems = []
|
||||
size = len(self.d)
|
||||
if self.d[:4] != b"dex\n":
|
||||
problems.append("not a dex: magic=%r" % self.d[:4])
|
||||
for key in ("string_ids_off", "type_ids_off", "proto_ids_off",
|
||||
"field_ids_off", "method_ids_off", "class_defs_off"):
|
||||
off = self.header[key]
|
||||
if not (0 < off < size):
|
||||
problems.append("%s=0x%x out of range (file size %d)"
|
||||
% (key, off, size))
|
||||
declared = self.header["file_size"]
|
||||
if declared and declared != size:
|
||||
problems.append("header file_size=%d but actual=%d" % (declared, size))
|
||||
return problems
|
||||
|
||||
# ---- index tables -----------------------------------------------------
|
||||
def string(self, idx):
|
||||
p = u32(self.d, self.header["string_ids_off"] + idx * 4)
|
||||
_utf16_len, p = read_uleb(self.d, p)
|
||||
end = self.d.index(b"\x00", p)
|
||||
return self.d[p:end].decode("utf-8", "replace")
|
||||
|
||||
def string_safe(self, idx):
|
||||
"""Like string() but never raises -- a desynced decode passes junk index.
|
||||
|
||||
A renderer that throws on a bad index hides the very symptom the caller is
|
||||
looking for (a decode that has drifted), so return a marker instead.
|
||||
"""
|
||||
try:
|
||||
if idx >= self.header["string_ids_size"]:
|
||||
return "<string_idx %d out of range>" % idx
|
||||
return self.string(idx)
|
||||
except Exception:
|
||||
return "<string_idx %d unreadable>" % idx
|
||||
|
||||
def type_(self, idx):
|
||||
return self.string(u32(self.d, self.header["type_ids_off"] + idx * 4))
|
||||
|
||||
def proto(self, idx):
|
||||
o = self.header["proto_ids_off"] + idx * 12
|
||||
ret = self.type_(u32(self.d, o + 4))
|
||||
poff = u32(self.d, o + 8)
|
||||
params = []
|
||||
if poff:
|
||||
n = u32(self.d, poff)
|
||||
p = poff + 4
|
||||
for _ in range(n):
|
||||
params.append(self.type_(u16(self.d, p)))
|
||||
p += 2
|
||||
return "(" + "".join(params) + ")" + ret
|
||||
|
||||
def field(self, idx):
|
||||
o = self.header["field_ids_off"] + idx * 8
|
||||
return (self.type_(u16(self.d, o)), self.string(u32(self.d, o + 4)),
|
||||
self.type_(u16(self.d, o + 2)))
|
||||
|
||||
def method(self, idx):
|
||||
o = self.header["method_ids_off"] + idx * 8
|
||||
return (self.type_(u16(self.d, o)), self.string(u32(self.d, o + 4)),
|
||||
self.proto(u16(self.d, o + 2)))
|
||||
|
||||
def find_class(self, fqcn):
|
||||
"""Return the class_def_item offset for a 'Lpkg/Name;' FQCN, or None."""
|
||||
for i in range(self.header["class_defs_size"]):
|
||||
o = self.header["class_defs_off"] + i * 32
|
||||
if self.type_(u32(self.d, o)) == fqcn:
|
||||
return o
|
||||
return None
|
||||
|
||||
def methods_of(self, fqcn):
|
||||
"""Yield (section, method_idx, class, name, descriptor, code_off)."""
|
||||
cd = self.find_class(fqcn)
|
||||
if cd is None:
|
||||
raise KeyError("class not found: %s" % fqcn)
|
||||
p = u32(self.d, cd + 24)
|
||||
if p == 0:
|
||||
return
|
||||
sf, p = read_uleb(self.d, p)
|
||||
inf, p = read_uleb(self.d, p)
|
||||
dm, p = read_uleb(self.d, p)
|
||||
vm, p = read_uleb(self.d, p)
|
||||
for _ in range(sf + inf): # skip field lists
|
||||
_i, p = read_uleb(self.d, p)
|
||||
_a, p = read_uleb(self.d, p)
|
||||
for section, count in (("direct", dm), ("virtual", vm)):
|
||||
running = 0
|
||||
for _ in range(count):
|
||||
diff, p = read_uleb(self.d, p)
|
||||
running += diff # indices are delta-encoded
|
||||
_acc, p = read_uleb(self.d, p)
|
||||
code_off, p = read_uleb(self.d, p)
|
||||
cls, name, desc = self.method(running)
|
||||
yield section, running, cls, name, desc, code_off
|
||||
|
||||
def find_method(self, fqcn, name, desc):
|
||||
"""Return (section, method_idx, code_off) or None."""
|
||||
for section, idx, _cls, nm, ds, code_off in self.methods_of(fqcn):
|
||||
if nm == name and ds == desc:
|
||||
return section, idx, code_off
|
||||
return None
|
||||
|
||||
def find_methods_named(self, fqcn, name):
|
||||
"""All overloads of a name -> list of (section, idx, desc, code_off)."""
|
||||
out = []
|
||||
for section, idx, _cls, nm, ds, code_off in self.methods_of(fqcn):
|
||||
if nm == name:
|
||||
out.append((section, idx, ds, code_off))
|
||||
return out
|
||||
|
||||
def class_names(self):
|
||||
for i in range(self.header["class_defs_size"]):
|
||||
o = self.header["class_defs_off"] + i * 32
|
||||
yield self.type_(u32(self.d, o))
|
||||
|
||||
# ---- code -------------------------------------------------------------
|
||||
def code_info(self, code_off):
|
||||
return {
|
||||
"registers": u16(self.d, code_off),
|
||||
"ins": u16(self.d, code_off + 2),
|
||||
"outs": u16(self.d, code_off + 4),
|
||||
"tries": u16(self.d, code_off + 6),
|
||||
"debug_info_off": u32(self.d, code_off + 8),
|
||||
"insns_size": u32(self.d, code_off + 12),
|
||||
"insns_off": code_off + 16,
|
||||
}
|
||||
|
||||
def decode(self, code_off):
|
||||
"""Yield dicts describing each instruction of one method body.
|
||||
|
||||
Callers should assert the walk ends exactly on insns_off+insns_size*2;
|
||||
landing short or long means the format table is wrong somewhere.
|
||||
"""
|
||||
info = self.code_info(code_off)
|
||||
pos = info["insns_off"]
|
||||
end = pos + info["insns_size"] * 2
|
||||
while pos < end:
|
||||
op = self.d[pos]
|
||||
units = insn_units(op, self.d, pos, end)
|
||||
if units < 1 or pos + units * 2 > end:
|
||||
return
|
||||
yield {
|
||||
"off": pos, "op": op, "units": units,
|
||||
"name": OP_NAMES.get(op, "op_%02x" % op),
|
||||
"raw": bytes(self.d[pos:pos + units * 2]),
|
||||
"registers": info["registers"],
|
||||
}
|
||||
pos += units * 2
|
||||
|
||||
def decode_all(self, code_off):
|
||||
"""Full listing as a list, plus a verdict on whether it ended cleanly."""
|
||||
insns = list(self.decode(code_off))
|
||||
info = self.code_info(code_off)
|
||||
expected = info["insns_off"] + info["insns_size"] * 2
|
||||
ended = insns[-1]["off"] + insns[-1]["units"] * 2 if insns else info["insns_off"]
|
||||
return insns, (ended == expected), expected
|
||||
|
||||
def insn_at(self, code_off, target_off):
|
||||
for insn in self.decode(code_off):
|
||||
if insn["off"] == target_off:
|
||||
return insn
|
||||
return None
|
||||
|
||||
# ---- operand helpers --------------------------------------------------
|
||||
def branch_target(self, insn):
|
||||
"""Absolute target of a branch, or None if the instruction is not one."""
|
||||
op, pos, raw = insn["op"], insn["off"], insn["raw"]
|
||||
if op in IF_TEST or op in IF_TESTZ:
|
||||
return pos + s16(u16(self.d, pos + 2)) * 2
|
||||
if op == 0x27: # goto (10t, signed byte)
|
||||
off = raw[1]
|
||||
if off > 127:
|
||||
off -= 256
|
||||
return pos + off * 2
|
||||
if op == 0x28:
|
||||
return pos + s16(u16(self.d, pos + 2)) * 2
|
||||
if op == 0x29:
|
||||
off = u32(self.d, pos + 2)
|
||||
if off > 0x7FFFFFFF:
|
||||
off -= 0x100000000
|
||||
return pos + off * 2
|
||||
return None
|
||||
|
||||
def branch_regs(self, insn):
|
||||
"""(regs_read, kind) for a conditional branch."""
|
||||
if insn["op"] in IF_TEST:
|
||||
return [insn["raw"][1] >> 4, insn["raw"][1] & 0xF], "if-test"
|
||||
if insn["op"] in IF_TESTZ:
|
||||
return [insn["raw"][1] & 0xF], "if-testz"
|
||||
return [], None
|
||||
|
||||
def all_targets(self, code_off):
|
||||
"""Set of every absolute branch target in one method."""
|
||||
targets = set()
|
||||
for insn in self.decode(code_off):
|
||||
t = self.branch_target(insn)
|
||||
if t is not None:
|
||||
targets.add(t)
|
||||
return targets
|
||||
|
||||
# ---- description ------------------------------------------------------
|
||||
def describe(self, insn):
|
||||
"""One-line human rendering with resolved field/method/string names."""
|
||||
op, pos, raw = insn["op"], insn["off"], insn["raw"]
|
||||
text = "0x%x: %-14s %s" % (pos, raw.hex(), insn["name"])
|
||||
if op in IF_TEST:
|
||||
text += " v%d,v%d -> 0x%x" % (raw[1] >> 4, raw[1] & 0xF,
|
||||
self.branch_target(insn))
|
||||
elif op in IF_TESTZ:
|
||||
text += " v%d -> 0x%x" % (raw[1] & 0xF, self.branch_target(insn))
|
||||
elif op in (0x27, 0x28, 0x29):
|
||||
text += " -> 0x%x" % self.branch_target(insn)
|
||||
elif op in IFIELD_OPS:
|
||||
cls, nm, ty = self.field(u16(self.d, pos + 2))
|
||||
text += " v%d <- v%d.%s:%s" % (raw[1] & 0xF, raw[1] >> 4, nm, ty)
|
||||
elif op in PFIELD_OPS:
|
||||
cls, nm, ty = self.field(u16(self.d, pos + 2))
|
||||
text += " v%d -> v%d.%s:%s" % (raw[1] & 0xF, raw[1] >> 4, nm, ty)
|
||||
elif op in SFIELD_OPS:
|
||||
cls, nm, ty = self.field(u16(self.d, pos + 2))
|
||||
text += " %s.%s:%s" % (cls, nm, ty)
|
||||
elif op in INVOKE_OPS:
|
||||
cls, nm, ds = self.method(u16(self.d, pos + 2))
|
||||
text += " %s.%s%s" % (cls, nm, ds)
|
||||
elif op in (0x19, 0x1A):
|
||||
# Both forms carry the string index as a uint16 at byte offset 2.
|
||||
# Reading a uint32 here gives a huge index that walks off the table.
|
||||
text += ' "%s"' % self.string_safe(u16(self.d, pos + 2))
|
||||
elif op == 0x12:
|
||||
lit = raw[1] & 0xF
|
||||
text += " v%d, %d" % (raw[1] >> 4, lit - 16 if lit > 7 else lit)
|
||||
return text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# dex header integrity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def fix_dex_header(data):
|
||||
"""Recompute a dex header's checksum and signature IN THE CORRECT ORDER.
|
||||
|
||||
Order is not cosmetic:
|
||||
bytes 12..32 = sha1(data[32:]) -- signature first
|
||||
bytes 8..12 = adler32(data[12:]) -- checksum last, covers the signature
|
||||
|
||||
Reversed, the adler32 is taken while the signature field is still zeroed, so
|
||||
the header never verifies. Android logs
|
||||
`Failure to verify dex file ...: Bad checksum (computed, expected)` -- the
|
||||
real value shows up as "expected" -- and falls back to interpreting the dex,
|
||||
which can surface as an unrelated ClassNotFoundException at startup.
|
||||
|
||||
Accepts and returns a bytearray; also returns the before/after values so a
|
||||
caller can print them.
|
||||
"""
|
||||
before_checksum = int.from_bytes(data[8:12], "little")
|
||||
before_signature = bytes(data[12:32])
|
||||
|
||||
data[12:32] = hashlib.sha1(bytes(data[32:])).digest()
|
||||
after_signature = bytes(data[12:32])
|
||||
|
||||
import zlib
|
||||
after_checksum = zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
|
||||
data[8:12] = after_checksum.to_bytes(4, "little")
|
||||
|
||||
return {
|
||||
"before_checksum": before_checksum, "after_checksum": after_checksum,
|
||||
"before_signature": before_signature, "after_signature": after_signature,
|
||||
}
|
||||
|
||||
|
||||
def verify_dex_header(data):
|
||||
"""Return (checksum_ok, signature_ok) for a bytearray/bytes dex."""
|
||||
import zlib
|
||||
stored_c = int.from_bytes(data[8:12], "little")
|
||||
calc_c = zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
|
||||
return stored_c == calc_c, bytes(data[12:32]) == hashlib.sha1(bytes(data[32:])).digest()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# loading
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def load_dex(source, entry=None):
|
||||
"""Load a Dex from a .dex path, an .apk/.zip path, or raw bytes.
|
||||
|
||||
For an archive, `entry` picks a member (default: the first classes*.dex).
|
||||
Returns (Dex, entry_name).
|
||||
"""
|
||||
if isinstance(source, (bytes, bytearray)):
|
||||
return Dex(bytes(source), entry or "<bytes>"), entry or "<bytes>"
|
||||
path = str(source)
|
||||
if path.lower().endswith((".apk", ".zip", ".jar", ".xapk", ".apks", ".apkm")):
|
||||
with zipfile.ZipFile(path) as z:
|
||||
if entry:
|
||||
return Dex(z.read(entry), entry), entry
|
||||
candidates = sorted(n for n in z.namelist()
|
||||
if n.startswith("classes") and n.endswith(".dex"))
|
||||
if not candidates:
|
||||
raise ValueError("no classes*.dex in %s" % path)
|
||||
name = candidates[0]
|
||||
return Dex(z.read(name), name), name
|
||||
with open(path, "rb") as fh:
|
||||
return Dex(fh.read(), path), path
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) != 5:
|
||||
print(__doc__)
|
||||
return 2
|
||||
target, fqcn, name, desc = argv[1:5]
|
||||
dex, entry = load_dex(target)
|
||||
print("== %s (entry %s, %d bytes)" % (target, entry, len(dex.d)))
|
||||
problems = dex.check()
|
||||
for p in problems:
|
||||
print(" [structure] %s" % p)
|
||||
if problems:
|
||||
print(" refusing to continue on a structurally broken read")
|
||||
return 1
|
||||
|
||||
found = dex.find_method(fqcn, name, desc)
|
||||
if not found:
|
||||
print("method not found: %s->%s%s" % (fqcn, name, desc))
|
||||
print("methods in class:")
|
||||
for section, idx, _c, nm, ds, code_off in dex.methods_of(fqcn):
|
||||
print(" [%s] %s%s code_off=0x%x" % (section, nm, ds, code_off))
|
||||
return 1
|
||||
|
||||
section, idx, code_off = found
|
||||
info = dex.code_info(code_off)
|
||||
insns, clean, expected = dex.decode_all(code_off)
|
||||
print("== %s.%s%s [%s] method_idx=%d" % (fqcn, name, desc, section, idx))
|
||||
print(" code_off=0x%x insns_off=0x%x insns_size=%d registers=%d"
|
||||
% (code_off, info["insns_off"], info["insns_size"], info["registers"]))
|
||||
print(" decoded %d instructions; ended cleanly: %s" % (len(insns), clean))
|
||||
if not clean:
|
||||
print(" WARNING: decode did not land on 0x%x -- offsets below are suspect"
|
||||
% expected)
|
||||
|
||||
regs = info["registers"]
|
||||
over = [i for i in insns if _max_reg(i) >= regs]
|
||||
if over:
|
||||
print(" WARNING: %d instruction(s) reference a register >= %d; "
|
||||
"desync likely" % (len(over), regs))
|
||||
|
||||
targets = dex.all_targets(code_off)
|
||||
print(" branch targets: %s" % ", ".join(sorted("0x%x" % t for t in targets)))
|
||||
print("-- listing --")
|
||||
for insn in insns:
|
||||
mark = " <== branch target" if insn["off"] in targets else ""
|
||||
print(" " + dex.describe(insn) + mark)
|
||||
return 0
|
||||
|
||||
|
||||
def _max_reg(insn):
|
||||
"""Highest register number named by an instruction (rough but sufficient)."""
|
||||
raw, op = insn["raw"], insn["op"]
|
||||
if op in IF_TEST or op in IFIELD_OPS or op in PFIELD_OPS:
|
||||
return max(raw[1] >> 4, raw[1] & 0xF)
|
||||
if op in IF_TESTZ:
|
||||
return raw[1] & 0xF
|
||||
if op == 0x12:
|
||||
return raw[1] >> 4
|
||||
if op in (0x01, 0x04, 0x07, 0x0A, 0x0B, 0x0C, 0x0D, 0x0F, 0x10, 0x11,
|
||||
0x1C, 0x1D, 0x1E):
|
||||
return raw[1] & 0xF
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
+224
-23
@@ -208,10 +208,26 @@ def build_unsigned(apk, repl, out_apk, drop_signatures=True):
|
||||
IllegalStateException: Module with the Main dispatcher is missing ...
|
||||
and the message never points at META-INF, so the cause is very hard to find.
|
||||
Therefore: drop signature artifacts only, keep every META-INF subdirectory.
|
||||
|
||||
ALIGNMENT. Writing the entries with a plain zipfile writer produces an archive
|
||||
Android R+ refuses to install:
|
||||
|
||||
Failure [-124: Failed parse during installPackageLI: Targeting R+ (version
|
||||
30 and above) requires the resources.arsc of installed APKs to be stored
|
||||
uncompressed and aligned on a 4-byte boundary]
|
||||
|
||||
Two independent requirements hide in that message: `resources.arsc` must be
|
||||
STORED (not deflated), and its data must start at a 4-byte boundary. The same
|
||||
applies to uncompressed `lib/*.so`. Python's zipfile cannot express an entry
|
||||
offset, so this writer emits the local headers itself and pads the local extra
|
||||
field to hit the boundary.
|
||||
"""
|
||||
seen = set()
|
||||
replaced_names = set(repl)
|
||||
with zipfile.ZipFile(apk, 'r') as zin, zipfile.ZipFile(out_apk, 'w', zipfile.ZIP_DEFLATED) as zout:
|
||||
|
||||
# Collect (name, source, data, keep_stored) in output order.
|
||||
plan = []
|
||||
with zipfile.ZipFile(apk, 'r') as zin:
|
||||
for item in zin.infolist():
|
||||
name = item.filename
|
||||
base = os.path.basename(name)
|
||||
@@ -222,32 +238,208 @@ def build_unsigned(apk, repl, out_apk, drop_signatures=True):
|
||||
continue
|
||||
if name in replaced_names:
|
||||
continue
|
||||
data = zin.read(name)
|
||||
keep = name in STORE_ONLY or base in STORE_ONLY
|
||||
zi = zipfile.ZipInfo(name, date_time=item.date_time)
|
||||
zi.compress_type = zipfile.ZIP_STORED if keep else zipfile.ZIP_DEFLATED
|
||||
zi.external_attr = item.external_attr
|
||||
zi.internal_attr = item.internal_attr
|
||||
zi.create_system = item.create_system
|
||||
zout.writestr(zi, data)
|
||||
seen.add(name)
|
||||
for name, path in sorted(repl.items()):
|
||||
with open(path, 'rb') as fh:
|
||||
data = fh.read()
|
||||
zi = zipfile.ZipInfo(name, date_time=(2024, 1, 1, 0, 0, 0))
|
||||
zi.compress_type = zipfile.ZIP_DEFLATED
|
||||
zout.writestr(zi, data)
|
||||
log('[zip] + %s (%d bytes <- %s)' % (name, len(data), path))
|
||||
seen.add(name)
|
||||
plan.append((name, item, zin.read(name),
|
||||
name in STORE_ONLY or base in STORE_ONLY))
|
||||
for name, path in sorted(repl.items()):
|
||||
with open(path, 'rb') as fh:
|
||||
data = fh.read()
|
||||
info = zipfile.ZipInfo(name, date_time=(2024, 1, 1, 0, 0, 0))
|
||||
plan.append((name, info, data, False))
|
||||
log('[zip] + %s (%d bytes <- %s)' % (name, len(data), path))
|
||||
|
||||
_write_aligned_zip(out_apk, plan)
|
||||
seen.update(name for name, _i, _d, _k in plan)
|
||||
return seen
|
||||
|
||||
|
||||
def zip_report(apk):
|
||||
rows = []
|
||||
with zipfile.ZipFile(apk, 'r') as z:
|
||||
ALIGN = 4
|
||||
# Entries Android requires to be STORED, and (for some) 4-byte aligned.
|
||||
ALIGNED_STORED = ('resources.arsc',)
|
||||
ALIGNED_PATTERNS = (re.compile(r'^lib/[^/]+\.so$'),)
|
||||
FILLER_NAME = 'META-INF/ALIGN.RSV'
|
||||
|
||||
|
||||
def _needs_alignment(name):
|
||||
if name in ALIGNED_STORED:
|
||||
return True
|
||||
return any(p.match(name) for p in ALIGNED_PATTERNS)
|
||||
|
||||
|
||||
def _dos_word(dt):
|
||||
"""(time, date) MS-DOS words for a zip local header."""
|
||||
y, mo, d, h, mi, s = dt
|
||||
if y < 1980:
|
||||
y, mo, d, h, mi, s = 1980, 1, 1, 0, 0, 0
|
||||
return (h << 11) | (mi << 5) | (s // 2), ((y - 1980) << 9) | (mo << 5) | d
|
||||
|
||||
|
||||
def _local_header(nlen, method, crc, csize, usize, dt, flags=0, extralen=0):
|
||||
t, dd = _dos_word(dt)
|
||||
import struct
|
||||
return struct.pack('<IHHHHHIIIHH', 0x04034B50, 20, flags, method,
|
||||
t, dd, crc, csize, usize, nlen, extralen)
|
||||
|
||||
|
||||
def _deflate_raw(data):
|
||||
"""Raw deflate (no zlib wrapper), which is what a zip method-8 entry holds.
|
||||
|
||||
zlib.compress() prepends a 2-byte zlib header. Some readers tolerate it, some
|
||||
do not, and the failure reads as a corrupt entry rather than a bad compressor
|
||||
call, so use wbits=-15 and get it right the first time.
|
||||
"""
|
||||
import zlib
|
||||
c = zlib.compressobj(9, zlib.DEFLATED, -15)
|
||||
return c.compress(data) + c.flush()
|
||||
|
||||
|
||||
def _write_aligned_zip(out_path, plan):
|
||||
"""Write a zip whose STORED entries can be relied on for offset alignment.
|
||||
|
||||
Padding rule, and why it is not just `(-offset) % 4`: a zip extra area is a
|
||||
sequence of (id, size, payload) records, so its minimum useful length is 4
|
||||
bytes. A required pad of 1-3 bytes therefore CANNOT be expressed in the extra
|
||||
field. When that happens this writer inserts a stored filler entry of exactly
|
||||
the needed size instead -- the filler has a computable size, so the following
|
||||
entry still lands on the boundary.
|
||||
"""
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
order = []
|
||||
meta = {}
|
||||
offset = 0
|
||||
scratch = out_path + '.tmp'
|
||||
|
||||
with open(scratch, 'wb') as out:
|
||||
for name, info, data, keep_stored in plan:
|
||||
name_b = name.encode('utf-8')
|
||||
if keep_stored:
|
||||
method, payload = 0, data
|
||||
else:
|
||||
method, payload = 8, _deflate_raw(data)
|
||||
crc = zlib.crc32(data) & 0xFFFFFFFF
|
||||
|
||||
# reach the boundary before the local header of an aligned entry
|
||||
if _needs_alignment(name):
|
||||
need = (ALIGN - ((offset + 30 + len(name_b)) % ALIGN)) % ALIGN
|
||||
if need in (1, 2, 3):
|
||||
off2, fill_meta = _emit_filler(out, offset, need)
|
||||
order.append(FILLER_NAME)
|
||||
meta[FILLER_NAME] = fill_meta
|
||||
offset = off2
|
||||
|
||||
extra, extralen = b'', 0
|
||||
if _needs_alignment(name):
|
||||
head = 30 + len(name_b)
|
||||
if (offset + head) % ALIGN != 0:
|
||||
# expressible pad: one (id,size,payload) record >= 4 bytes
|
||||
pad = (ALIGN - ((offset + head) % ALIGN)) % ALIGN
|
||||
if pad < 4:
|
||||
pad += ALIGN
|
||||
extra = b'\xfe\xca' + struct.pack('<H', pad - 4) + b'\x00' * (pad - 4)
|
||||
extralen = len(extra)
|
||||
|
||||
flags = getattr(info, 'flag_bits', 0)
|
||||
local_header_offset = offset # the CD points at the HEADER
|
||||
out.write(_local_header(len(name_b), method, crc, len(payload),
|
||||
len(data), info.date_time, flags, extralen))
|
||||
out.write(name_b)
|
||||
out.write(extra)
|
||||
data_off = offset + 30 + len(name_b) + extralen
|
||||
if out.tell() != data_off:
|
||||
raise RuntimeError('offset bookkeeping drift for %s' % name)
|
||||
if _needs_alignment(name) and data_off % ALIGN != 0:
|
||||
raise RuntimeError('%s not aligned: data at 0x%x' % (name, data_off))
|
||||
out.write(payload)
|
||||
offset = out.tell()
|
||||
|
||||
order.append(name)
|
||||
meta[name] = {'name_b': name_b, 'method': method, 'crc': crc,
|
||||
'csize': len(payload), 'usize': len(data),
|
||||
'local_offset': local_header_offset,
|
||||
'date_time': info.date_time, 'flag_bits': flags,
|
||||
'external_attr': getattr(info, 'external_attr', 0)}
|
||||
|
||||
cd_start = out.tell()
|
||||
for name in order:
|
||||
m = meta[name]
|
||||
t, dd = _dos_word(m['date_time'])
|
||||
# central directory: sig, ver_made, ver_need, flags, method, time,
|
||||
# date, crc, csize, usize, namelen, extralen, commentlen, disk,
|
||||
# int_attr, ext_attr, local_header_offset
|
||||
out.write(struct.pack(
|
||||
'<IHHHHHHIIIHHHHHII', 0x02014B50, 20, 20, m.get('flag_bits', 0),
|
||||
m['method'], t, dd, m['crc'], m['csize'], m['usize'],
|
||||
len(m['name_b']), 0, 0, 0, 0,
|
||||
(m['external_attr'] >> 16) & 0xFFFF, m['local_offset']))
|
||||
out.write(m['name_b'])
|
||||
cd_size = out.tell() - cd_start
|
||||
out.write(struct.pack('<IHHHHIIH', 0x06054B50, 0, 0, len(order), len(order),
|
||||
cd_size, cd_start, 0))
|
||||
|
||||
shutil.move(scratch, out_path)
|
||||
return order
|
||||
|
||||
|
||||
def _emit_filler(out, offset, size):
|
||||
"""Insert a stored, zero-filled entry of exactly `size` payload bytes."""
|
||||
import struct
|
||||
import zlib
|
||||
name_b = FILLER_NAME.encode('utf-8')
|
||||
payload = b'\x00' * size
|
||||
crc = zlib.crc32(payload) & 0xFFFFFFFF
|
||||
out.write(_local_header(len(name_b), 0, crc, size, size, (1980, 1, 1, 0, 0, 0)))
|
||||
out.write(name_b)
|
||||
out.write(payload)
|
||||
return out.tell(), {'name_b': name_b, 'method': 0, 'crc': crc,
|
||||
'csize': size, 'usize': size,
|
||||
'local_offset': offset,
|
||||
'date_time': (1980, 1, 1, 0, 0, 0), 'external_attr': 0}
|
||||
|
||||
|
||||
def check_alignment(apk):
|
||||
"""Report storage + 4-byte alignment of the entries Android insists on.
|
||||
|
||||
Returns a list of complaint strings; empty means the install gate is satisfied.
|
||||
"""
|
||||
import struct
|
||||
bad = []
|
||||
with zipfile.ZipFile(apk, 'r') as z, open(apk, 'rb') as fh:
|
||||
for i in z.infolist():
|
||||
if i.filename in STORE_ONLY or os.path.basename(i.filename) in STORE_ONLY:
|
||||
rows.append('%s method=%d (0=STORED)' % (i.filename, i.compress_type))
|
||||
if not (i.filename in ALIGNED_STORED or _needs_alignment(i.filename)):
|
||||
continue
|
||||
fh.seek(i.header_offset)
|
||||
lh = fh.read(30)
|
||||
nlen, elen = struct.unpack('<HH', lh[26:30])
|
||||
data_off = i.header_offset + 30 + nlen + elen
|
||||
if i.compress_type != 0:
|
||||
bad.append('%s is compressed (method=%d); Android R+ requires '
|
||||
'STORED' % (i.filename, i.compress_type))
|
||||
if i.filename != FILLER_NAME and data_off % ALIGN != 0:
|
||||
bad.append('%s data offset 0x%x is not %d-byte aligned'
|
||||
% (i.filename, data_off, ALIGN))
|
||||
return bad
|
||||
|
||||
|
||||
def zip_report(apk):
|
||||
"""Entry summary, including the storage/alignment facts Android gates on."""
|
||||
import struct
|
||||
rows = []
|
||||
with zipfile.ZipFile(apk, 'r') as z, open(apk, 'rb') as fh:
|
||||
for i in z.infolist():
|
||||
base = os.path.basename(i.filename)
|
||||
needs = (i.filename in STORE_ONLY or base in STORE_ONLY
|
||||
or _needs_alignment(i.filename))
|
||||
if needs:
|
||||
fh.seek(i.header_offset)
|
||||
lh = fh.read(30)
|
||||
nlen, elen = struct.unpack('<HH', lh[26:30])
|
||||
data_off = i.header_offset + 30 + nlen + elen
|
||||
verdict = 'aligned' if data_off % ALIGN == 0 else 'NOT ALIGNED'
|
||||
stored = 'STORED' if i.compress_type == 0 else \
|
||||
'COMPRESSED(method=%d)' % i.compress_type
|
||||
rows.append('%-40s %-22s offset=0x%-8x %s'
|
||||
% (i.filename, stored, data_off, verdict))
|
||||
if re.match(r'^classes\d*\.dex$', i.filename):
|
||||
rows.append('%s %d bytes method=%d crc=%08x' % (
|
||||
i.filename, i.file_size, i.compress_type, i.CRC))
|
||||
@@ -448,6 +640,15 @@ def main():
|
||||
log('== zip layout of final apk')
|
||||
for row in zip_report(out):
|
||||
log(' ' + row)
|
||||
alignment_problems = check_alignment(out)
|
||||
if alignment_problems:
|
||||
log('== ALIGNMENT/STORAGE GATE FAILED -- do not ship this build:')
|
||||
for p in alignment_problems:
|
||||
log(' - %s' % p)
|
||||
log(' Android R+ (targetSdk 30+) refuses to install an APK whose '
|
||||
'resources.arsc is compressed or not 4-byte aligned.')
|
||||
else:
|
||||
log('== alignment gate: resources.arsc STORED and 4-byte aligned (OK)')
|
||||
|
||||
ok = verify_apk(out, tools, signer_jar, expect_signed=not args.no_sign)
|
||||
if not args.no_sign:
|
||||
|
||||
@@ -28,7 +28,16 @@ import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
PORT = int(sys.argv[1]) if len(sys.argv) > 1 else 8080
|
||||
if len(sys.argv) > 1 and sys.argv[1] in ('-h', '--help'):
|
||||
print(__doc__)
|
||||
sys.exit(0)
|
||||
|
||||
try:
|
||||
PORT = int(sys.argv[1]) if len(sys.argv) > 1 else 8080
|
||||
except ValueError:
|
||||
print('error: listen_port must be a number, got %r\n' % sys.argv[1])
|
||||
print(__doc__)
|
||||
sys.exit(2)
|
||||
LOGF = sys.argv[2] if len(sys.argv) > 2 else None
|
||||
_lock = threading.Lock()
|
||||
_logf = open(LOGF, 'a', encoding='utf-8') if LOGF else None
|
||||
|
||||
Reference in New Issue
Block a user