paper
arXiv cs.CL
November 4th, 2026 at 5:00 AM

MediaWiki Code2Code Search: Neural Retrieval for the Semantic Discovery of Open-Source Software Entities

arXiv:2607.26766v2 Announce Type: replace-cross Abstract: Code search in large-scale ecosystems is often hindered by the lexical gap between user queries and implementation details, alongside the trade-off between the low latency of traditional Information Retrieval (IR) and the precision of Deep Learning (DL). We present MediaWiki Code2Code Search, a neural retrieval system for semantic code-to-code discovery. By indexing 1.29 million structural entities (functions, types, and templates) across 2,500+ MediaWiki repositories, our system enables retrieval based on computational intent rather than surface tokens. We employ a split-build architecture, decoupling GPU-intensive offline indexing from a CPU-only serving layer; our FAISS IVF-PQ index occupies 168.6 MiB: a 96.6\% reduction compared to a flat float32 baseline, and achieves a median query latency of 1.85 seconds on commodity hardware, satisfying the 6 GiB RAM constraint of Wikimedia Toolforge. In a blinded evaluation graded by a three-model LLM jury (inter-judge Krippendorff's $\alpha = 0.74$), our system improves ranking quality over a BM25 baseline across a 27-query benchmark, raising the jury-mean nDCG@10 (to 0.81 from 0.63), a positive and consistent gain across all judges. Gains are most pronounced in name-obfuscated and cross-language tasks where lexical methods fail. The system is available at https://code2codesearch.toolforge.org under the Apache 2.0 licence and provides an open RESTful API.

#ai
#llm
#open_source

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 3.01

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2607.26766