Issue To Code Semantic Mapping

via “issue-to-code semantic mapping”

AI coding agent benchmark — real GitHub issues, end-to-end evaluation, the standard for code agents.

Unique: Grounds semantic mappings in actual GitHub issue resolutions rather than synthetic or manually-annotated relationships, ensuring that the mappings reflect real developer decisions about which code to modify. The mappings are validated by test passage, creating an objective ground truth.

vs others: More authentic than manually-labeled datasets because mappings come from real issue resolutions, and more objective than human-annotated relevance because test passage provides ground truth for which files actually needed modification.