Agentless: The Power of Deterministic Abstraction

Localization is the primary failure mode for autonomous agents; they frequently "wander" into irrelevant parts of the codebase, consuming their context window with non-essential files. Agentless bypasses this entirely by presenting the model with a carefully engineered Skeleton Format - a structural abstraction of the file system that retains only class signatures, function definitions, and module-level comments.

The File Skeleton Format used in Agentless to provide structural context without token overflow.

The localization process is strictly hierarchical and divided into three non-iterative steps:

1. File-Level Localization: The LLM is provided with a compressed tree-like directory structure of the repository. It is instructed to identify a small subset of highly suspicious files that likely contain the bug.
2. Class/Function-Level Localization: For the selected files, Agentless generates the "Skeleton Format". By stripping out the internal implementation details of functions and leaving only the signatures and docstrings, the token count is drastically reduced. The LLM then narrows the search to specific classes or functions within these skeletons.
3. Line-Level Localization: Finally, the model is given the complete, uncompressed source code of the identified classes or functions. It is tasked with pinpointing the exact line numbers that require modification.

This hierarchical reduction forces the model to pinpoint the bug through a deterministic sequence. It proves that for structured domains, a rigid map and progressive disclosure of complexity is far more valuable than a flexible, unconstrained reasoning loop.

Once the bug is localized, the repair phase begins. Instead of asking the LLM to rewrite entire functions or files - which frequently leads to indentation errors and lost context - Agentless mandates a strict Search and Replace diff format. The model must output a SEARCH block containing the exact original lines from the repository, followed by a REPLACE block with the intended fix.

To maximize the probability of success, the system generates multiple candidate patches ($n=21$ by default) by sampling the LLM at a high temperature. However, selecting the correct patch from this pool presents a significant challenge: LLMs often generate functionally identical code with superficial variations in whitespace, variable naming, or comments. To solve this, Agentless introduces AST Normalization.

Every generated patch is parsed into an Abstract Syntax Tree (AST) using Python's ast module. The system algorithmically traverses the tree to strip out all non-functional elements - docstrings, inline comments, and arbitrary formatting - and then unparses the AST back into a canonical text format using astunparse. This allows the system to perform Majority Voting on the true semantic intent of the fix, ensuring that surface-level differences don't split the vote. The surviving, most semantically dominant patch is then moved to the validation phase.

The validation phase in Agentless integrates classical software engineering tools as a hard gate for AI output. Candidate patches are first filtered through a basic syntax checker to ensure compilation success. The surviving patches are then applied to the repository and subjected to a regression test suite. Any patch that breaks existing repository tests is immediately discarded. Only the patches that pass these deterministic filters are passed back to the LLM for a final re-ranking and selection.

Beyond the architectural innovations, the UIUC researchers conducted a manual audit of the widely used SWE-bench Lite dataset, uncovering significant data contamination. They found that 4.3% of the issues contained the exact "ground truth" patch in the issue description (data leakage), while another 9.3% lacked sufficient information in the issue text for any human or machine to logically deduce a solution. By filtering these out, they created SWE-bench Lite-S (the "Sane" subset).

This discovery highlights a critical evaluation gap for the research community: we are often measuring agent performance on a noisy channel. Agentless proves that the current bottleneck in AI software engineering is not necessarily the LLM's reasoning capability, but the precision of localization, the semantic density of the context, and the cleanliness of our evaluation metrics. As context windows expand to millions of tokens, the necessity of abstractions like the "Skeleton Format" may evolve, but the requirement for AST-based semantic verification and deterministic testing remains the primary guardrail for reliable deployment.