QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction

arXiv:2610.30180 · cs.IR, q-bio.QM · Submitted 2026-10-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction".

Marcus: Here's the gist of the paper. The gist The QProtFuse method proposes query-guided layer fusion to adaptively fuse multilayer protein features and jointly learns protein and annotation retrieval through intra-modal and cross-modal contrastive objectives to improve retrieval-augmented protein function prediction.

Ines: Predicting protein functions from amino acid sequences is essential to understanding biological processes, yet existing retrieval methods primarily expand external signals while overlooking the rich knowledge encoded within a single protein language model (PLM) The paper investigates whether different PLM layers exhibit distinct strengths in protein retrieval and how this internal knowledge can be effectively used to improve protein RAG Layerwise Probing and Fusion Strategy The study first probed the layerwise retrieval performance of ESM-2 on Prot-Inst-OOD tasks, showing that "the four subtasks achieve their best retrieval performance using features from different intermediate layers" This observation led to the proposal of QProtFuse, which "leverages the query to adaptively assign layer-wise weights and fuse multi-layer features".

Marcus: The fusion process involves grouping consecutive PLM layers and then conditioning this fusion on a frozen text encoder that maps queries to group weights via an MLP.

Ines: Joint Contrastive Learning Objectives QProtFuse is trained with a dual-objective contrastive learning framework consisting of two complementary objectives: intra-modal protein-to-protein (PP) alignment and cross-modal protein-to-annotation (PA) alignment.

Paper summary: Marcus: Looking at the authors of this paper, Mianzhi Pan, Peishuo Liu, Yawen Ouyang, Botian Wang, Yiming Rong, Hao Zhou. They are all working in areas related to AI and bioinformatics research at places like Nanjing University and Tsinghua University > it shows how these kinds of cross-disciplinary teams are tackling these complex problems >

Ines: The paper QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction is significant because it moves away from the old idea of just using the last layer of a protein model > it establishes that the way we fuse information internally matters more than just picking a single component >

Yuki: For someone looking at this, what does this mean practically? It suggests that when you use a protein language model for function prediction, you shouldn't assume every layer is equally good at finding the right biological matches >

Marcus: It means we should think about how to guide the fusion process with a query, which lets us adaptively select which layers contribute most to answering our specific biological question >

Ines: So it’s about making the model smarter about its own structure through this query-guided fusion and complementary training objectives > that’s what they show helps improve downstream RAG function descriptions for models like Qwen3-8B and DeepSeek-V4-Pro >

Yuki: It supports the idea that internal knowledge, when accessed correctly, can significantly improve our ability to predict biological functions from sequences > it’s about leveraging the structure of the model itself rather than just treating it as a black box >

Conclusion: Ines: So, we’ve seen how QProtFuse adapts protein model layers using queries to boost function prediction retrieval, right?

Marcus: Yeah, it’s about moving past just picking one layer and instead using that query to guide the whole fusion process between the different layers of the protein language model.

Yuki: From a population perspective, this suggests that different parts of the protein model might be specialized for different kinds of biological questions, which is interesting because we often treat them as a single entity.

Ines: Exactly. The authors show that this query-guided fusion helps them jointly learn how to retrieve both the protein and its biological annotations, using these two contrastive objectives they built in.

Marcus: And the results are pretty solid on Prot-Inst-OOD, where they got a Recall at one of sixteen point four percent with the reranking setup. That’s a real lift over just using the final layer alone.

Yuki: That sixteen point four percent recall, when you look at it through a population lens, it means we’re getting better at linking sequence features to actual known biological functions in a way that respects the complexity of the model structure.

Ines: But what does this actually mean for us? It means that for function prediction using large language models, you don't just rely on whatever layer happens to be best; you can dynamically adjust how those layers work together based on the specific question you’re asking.

Marcus: And when we look at the downstream LLM-as-a-judge scores, like Qwen3 and DeepSeek V4 Pro getting up to six point two and six point six eight, that tells us this retrieval improvement actually translates into better function descriptions for those bigger models.

Yuki: It suggests that exploiting the internal knowledge of a single protein language model in this adaptive way is a more effective strategy than just feeding it everything at once.

Ines: So, QProtFuse shows how leveraging the model's internal structure intelligently can give us much stronger results when we’re trying to predict function from an amino acid sequence.

Marcus: It opens up a new way to think about fine-tuning these models for biological tasks by considering their architecture dynamically instead of statically.

Yuki: Next, we have to look at how this adaptive fusion performs on different types of protein structures or maybe even different disease states, which is where we really test the robustness of this approach.

National Key Laboratory for Novel Software Technology & School of Artificial Intelligence, Nanjing University, China · Institute of AI Industry Research (AIR), Tsinghua University 3Shanghai AI Lab · Chemistry and Biomedicine Innovation Center (ChemBIC), Nanjing University · University of Chinese Academy of Sciences, Beijing

cs.IR, q-bio.QM

Submitted: 2026-10-19

Updated: 2026-10-19

Code: https://github.com/panmianzhi/QProtFuse

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist The QProtFuse method proposes query-guided layer fusion to adaptively fuse multilayer protein features and jointly learns protein and annotation retrieval through intra-modal and cross-modal

Key concepts

Layerwise Probing
This involves testing different intermediate layers of a PLM to see which ones are best for retrieving specific protein information. The study found that different subtasks performed better when using features from various intermediate layers, proving that relying only on the final layer is not optimal.
QProtFuse
This proposed method uses a query to dynamically determine the importance (weights) of different PLM layers and fuses their features. It combines these multi-layer features using this query-guided fusion to create a richer representation for retrieval.
Intra-modal Contrastive Learning (PP)
This objective focuses on aligning protein features with other proteins. It trains the model to distinguish functionally related proteins based on a given query, learning how to better separate similar biological entities within the protein data itself.
Cross-modal Contrastive Learning (PA)
This objective aligns protein features with external annotations (descriptions). It pushes the protein features closer to their corresponding descriptions while pushing unrelated proteins away from them, improving retrieval based on external knowledge.

Summary

The gist The QProtFuse method proposes query-guided layer fusion to adaptively fuse multilayer protein features and jointly learns protein and annotation retrieval through intra-modal and cross-modal contrastive objectives to improve retrieval-augmented protein function prediction.

Motivation

Predicting protein functions from amino acid sequences is essential to understanding biological processes, yet existing retrieval methods primarily expand external signals while overlooking the rich knowledge encoded within a single protein language model (PLM) The paper investigates whether different PLM layers exhibit distinct strengths in protein retrieval and how this internal knowledge can be effectively used to improve protein RAG

Layerwise Probing and Fusion Strategy

The study first probed the layerwise retrieval performance of ESM-2 on Prot-Inst-OOD tasks, showing that the four subtasks achieve their best retrieval performance using features from different intermediate layers This observation led to the proposal of QProtFuse, which leverages the query to adaptively assign layer-wise weights and fuse multi-layer features. The fusion process involves grouping consecutive PLM layers and then conditioning this fusion on a frozen text encoder that maps queries to group weights via an MLP.

Joint Contrastive Learning Objectives

QProtFuse is trained with a dual-objective contrastive learning framework consisting of two complementary objectives: intra-modal protein-to-protein (PP) alignment and cross-modal protein-to-annotation (PA) alignment. The PP objective learns to distinguish functionally related proteins under the query by optimizing the expected SigLIP loss over training queries and candidates. The PA objective draws protein features toward matching annotations and pushes them from unrelated descriptions by aligning query-conditioned protein features with candidate annotation features.

Inference Modes

Joint training enables QProtFuse to support three complementary modes during inference: protein retrieval, annotation retrieval, and protein retrieval with annotation reranking. Protein retrieval uses the PP branch, which is responsive to the biological question, while annotation retrieval utilizes the PA branch. Furthermore, protein retrieval with annotation reranking uses PP to identify proteins and then reranks them using PA scores to refine the order based on how well annotations match.

Experimental Results

On Prot-Inst-OOD, QProtFuse achieved superior performance compared with all baselines, reaching a Recall@1 of 16.4% and Recall@10 of 25.9% with reranking configuration. This retrieval gain improved downstream function descriptions, yielding the highest LLM-as-a-judge scores for both Qwen3-8B (6.22/10) and DeepSeek-V4-Pro (6.68/10). Ablation analyses confirmed that query guidance improves retrieval effectiveness and demonstrated that joint training improves both retrieval modes over their respective single-objective models, with a larger gain in PA.

Conclusion

QProtFuse successfully demonstrates the value of query-guided layer fusion and complementary training objectives for exploiting a PLM’s internal knowledge to achieve SOTA retrieval and RAG-based protein function description performance. The findings support the adaptive use of a single PLM’s internal knowledge as an effective approach to retrieval-augmented protein function prediction.

--- Page 5 ---

The paper investigates whether different PLM layers exhibit distinct strengths in protein retrieval and how this internal knowledge can be effectively used to improve protein RAG As shown in Figure 1(b), the four subtasks achieve their best retrieval performance using features from different intermediate layers. These observations answer Q1 and show that relying solely on finallayer features is suboptimal.

--- Page 6 ---

Inspired by multi-layer visual feature fusion in multimodal LLMs (Yao et al., 2024; Lin et al., 2025, 2026), we propose QProtFuse (Queryguided multi-layer Protein Feature Fusion) for retrieval-augmented protein function prediction. QProtFuse leverages the query to adaptively assign layer-wise weights and fuse multi-layer features.

--- Page 7 ---

We extend layer fusion to PLMs and adapt it to retrieval through joint protein-to-protein and proteinto-description contrastive objectives. We train QProtFuse with two complementary objectives: intra-modal protein-toprotein (PP) alignment and cross-modal proteinto-annotation (PA) alignment.

--- Page 9 ---

QProtFuse achieves the strongest protein and annotation retrieval per8. The findings highlight adaptive use of a single PLM’s internal knowledge as an effective approach to retrievalaugmented protein function prediction.

--- Page 14 ---

QProtFuse outperforms reranked ProtQueSt on all three metrics and achieves the highest MRR and R@1 among the Hybrid methods. The results highlight the value of query-guided retrieval for generation quality beyond lexical entity overlap.

--- Page 5 ---

QProtFuse achieves superior protein and annotation retrieval performance compared with all baselines, reaching a Recall@1 of 16.4% and Recall@10 of 25.9% with our reranking configuration.

--- Page 4 ---

QProtFuse also consistently outperforms RAPM and ProtQueSt in PP retrieval, as well as ProtQueSt in PA retrieval. These gains over methods integrating additional external signals highlight the value of exploiting the rich inner knowledge within a single PLM.

--- Page 4 ---

We use OpenAI’s text-embedding-3-large (3072-D) for query and annotation embedding. We set τquery = 0.65, τfn = 0.70, and H = 128, and initialize the learnable SigLIP temperature τ and bias b to 0.07 and −10, respectively.

--- Page 4 ---

QProtFuse achieves the highest MRR and R@1 among the Hybrid methods. We use this reranking configuration for downstream RAG.

--- Page 7 ---

QProtFuse outperforms reranked ProtQueSt on all three metrics and achieves the highest MRR and R@1 among the Hybrid methods. We use this reranking configuration for downstream RAG.

--- Page 5 ---

QProtFuse outperforms reranked ProtQueSt on all three metrics and achieves the highest MRR and R@1 among the Hybrid methods. We use this reranking configuration for downstream RAG.

Improvements for AI systems

  1. Adaptive feature fusion for protein retrieval allows different PLM layers to be weighted based on biological queries, as shown by QProtFuse using a query-dependent weights mechanism. This enables the system to select features from different depths of the PLM that are most relevant for a specific task, addressing the limitation that relying solely on final-layer features is suboptimal.

  2. Joint contrastive objectives—intra-modal protein-to-protein and cross-modal protein-to-annotation alignment—jointly train the retriever to select such evidence by optimizing both intra-modal PP alignment and cross-modal PA alignment. This allows the improved AI system to perform superior retrieval for both proteins and annotations simultaneously, achieving a Recall at 1 of 16.4% on Prot-Inst-OOD.

  3. The inference framework supports three complementary modes: protein retrieval, annotation retrieval, and protein retrieval with annotation reranking, all using the same checkpoint. This enables the improved system to generate protein retrieval with annotation reranking, which refines the order of candidates based on how well annotations match the query protein and question to improve downstream RAG.

Abstract

Predicting protein functions from amino acid sequences is essential to understanding biological processes. Large language models (LLMs) offer flexible function descriptions to natural-language queries, yet their limited protein understanding motivates retrieval-augmented generation (RAG). Existing protein RAG methods primarily expand external retrieval signals while overlooking the rich knowledge encoded within a single protein language model (PLM). We show that retrieval performance varies across PLM layers and biological tasks, making conventional final-layer selection suboptimal. To this end, we propose QProtFuse, which uses queries to adaptively fuse multi-layer protein features and jointly learns protein and annotation retrieval through intra-modal and cross-modal contrastive objectives. On Prot-Inst-OOD, QProtFuse achieves the best protein and annotation retrieval scores, reaching 16.4% Recall@1 with annotation reranking. These gains improve downstream RAG-based function descriptions, yielding the highest LLM-as-a-judge scores for both Qwen3-8B (6.22/10) and DeepSeek-V4-Pro (6.68/10). Ablations and weight analyses demonstrate the value of query-guided fusion and complementary training objectives for exploiting a PLM's internal knowledge.

Sources

Related papers