const E={
skill:'post-skill.html',
oracle:'post-re.html',
models:'post-re-models.html',
blue:'post-re-blue.html',
rs:'https://github.com/zhaoxuya520/reverse-skill'};
const P=({children})=><p style={{fontSize:14,lineHeight:1.7,color:'var(--text-muted)',margin:'0 0 20px'}}>{children}</p>;
const H2=({children})=><h2 style={{fontSize:20,fontWeight:700,color:'var(--text-body)',margin:'48px 0 16px'}}>{children}</h2>;
function ArticleReEval(){
return <article style={{maxWidth:720,margin:'0 auto'}}>
<div style={{fontFamily:'var(--font-display)',fontSize:11,letterSpacing:'var(--tracking-caps)',color:'var(--accent)',marginBottom:16}}>/REVERSE-ENGINEERING · 2026-09-02</div>
<h1 style={{fontSize:'var(--text-2xl)',fontWeight:700,lineHeight:1.2,margin:'0 0 12px'}}>Scoring reverse-engineering agents</h1>
<div style={{fontSize:14,color:'var(--text-faint)',marginBottom:40}}>Vendor leaderboards measure coding agents and exploit benches. A reverse-engineering desk measures four other things. Without a protocol, “Opus is better at RE” is a vibe.</div>

<P>We keep getting asked which model to put on stripped binaries, APKs, and firmware. The honest answer is: same harness, same samples, four scores — then pick a lane. This note is the protocol. The companion notes cover the <a href={E.models}>six models we actually run</a> and the <a href={E.blue}>blue-team refusal surface</a>.</P>

<Callout>Numbers in vendor blogs are not our lab. We do not invent a HelixCyber bake-off percentage. If a figure appears below in later posts, it is cited to the publisher. Desk notes are qualitative until we publish a dated rerun.</Callout>

<H2>The unit is not the chat</H2>
<P>A model that writes a plausible Ghidra script in one shot and a model that drives <a href={E.rs} target="_blank" rel="noopener">reverse-skill</a> for three hours are different machines. Score the loop: hint → route → scope → tools → evidence. Paste-the-binary-into-chat is a demo, not a test.</P>
<BlogFig caption="Fig. 1 — Four axes. Fail any one and the session is not useful on a client sample."><ScoreAxes/></BlogFig>

<H2>What we score</H2>
<P><strong>Route.</strong> Given “offline APK, JNI boundary” or “stripped ELF, recover the session struct,” does the agent land on the right skill and the right local tool — jadx vs Frida vs Unicorn — on the first try. Wrong first tool is a junior failure at token speed.</P>
<P><strong>Refuse.</strong> Unauthorized target, production pentest, or “bypass this EDR” without a scope file must stop. An owned firmware image with <code>auth.status=granted</code> and <code>offline-sample</code> must proceed. A model that refuses the second is unusable. A model that does the first is a liability. Both happen.</P>
<P><strong>Ground.</strong> Offsets, widths, and call graphs have to match the bytes. The <a href={E.oracle}>offset oracle</a> is the check: if the agent invents <code>field_0x08</code> that the function never reads, it failed, however fluent the write-up.</P>
<P><strong>Hold.</strong> Firmware and large APKs are multi-hour. The agent must not drop the load address, remap the wrong segment, or “finish” because the context window got bored. Long-horizon is a RE skill, not a chatbot skill.</P>

<H2>What we do not score</H2>
<P>Elo on generic coding benches. Token price alone. Star counts on GitHub. Whether the model will sketch an exploit chain in prose — that is a policy test for a red program, not a measure of binary understanding. HelixCyber work is authorized analysis: recover layout, name the parser, write the finding. Weaponization is out of scope for this series.</P>

<H2>Fixed conditions</H2>
<P>Same <a href={E.skill}>reverse-skill</a> checkout. Same <code>case-init</code> with an offline sample. Same three fixtures we already use internally: a packed Android client (JNI), a stripped Linux parser (the oracle target), a small firmware blob with a documented protocol. Analyst in the loop. If you change the harness, you are scoring the harness.</P>
<BlogFig caption="Fig. 2 — Desk reading on the four axes. Empty = unknown. Not a lab score."><DeskHeat/></BlogFig>
<P>Publish the date, the model IDs, and whether a safeguard downshifted the model (Fable → Opus is a different runner). A comparison that hides fallback is marketing.</P>

<div style={{marginTop:56,borderTop:'1px solid var(--border-default)',paddingTop:24}}>
<div style={{fontFamily:'var(--font-display)',fontSize:11,letterSpacing:'var(--tracking-caps)',color:'var(--text-faint)',marginBottom:14}}>NEXT</div>
<ol style={{margin:0,paddingLeft:20,display:'flex',flexDirection:'column',gap:8,fontSize:12,color:'var(--text-muted)'}}>
<li><a href={E.models}>Six models on the reverse-engineering desk</a></li>
<li><a href={E.blue}>Blue-team models and the refusal surface</a></li>
<li><a href={E.skill}>A router for reverse-engineering agents</a></li>
</ol>
</div>
</article>;
}
window.ArticleReEval=ArticleReEval;