DecompileBench
A comprehensive benchmark for evaluating traditional and LLM-based decompilers in real-world scenarios.
0 projects
About DecompileBench
DecompileBench is a framework designed to evaluate the performance of decompilation processes using both traditional decompilers and large language models (LLMs). It provides a suite of scripts to extract functions from real-world projects (via `oss-fuzz`), compile them into datasets with varying optimization levels (O0 to Os), and interface with a scalable decompiler service. The framework supports traditional tools like Hex-Rays, Ghidra, and BinaryNinja, alongside LLM-based refinement using models such as GPT-4o-mini. Evaluation is conducted through multiple metrics, including Reexecutable Success Rate (RSR), Coverage Expansion Rate (CER), and an LLM-based arena evaluation that computes Elo scores across 12 dimensions of code quality.
Summarised from the official page linked above.