Controlling Output Rankings in LLM-Based Search via Generative Engine Optimization

arXiv:2602.03608 · cs.CL, cs.AI, cs.IR · Submitted 2026-02-03 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Controlling Output Rankings in LLM-Based Search via Generative Engine Optimization".

Jane: Here's the gist of the paper. The gist The CORE method proposes an optimization technique to control output rankings in LLM-based search by appending strategically designed content to retrieved items, which has been shown to achieve high promotion success rates across various LLMs and product categories.

Tom: Problem Definition The core problem addressed is that the final recommendations Alice sees are still largely determined by the initial order of results returned by the search engines, even after an LLM summarizes and reranks them The ranking objective is defined as L(q, I; T(i⋆)) = − log pθ z(T(i⋆)) q, I where i⋆ is the designated target item with associated text T(i⋆).

Jane: Optimizing T(i⋆) to minimize L increases the likelihood that the generative engine ranks the target item more favorably.

Tom: Optimization Strategy CORE incorporates three types of optimization content to steer the ranking of outputs: string-based, reasoning-based, and review-based.

Paper summary: Tom: So we're talking about this paper called Controlling Output Rankings in gEnerative Engines for LLM-based search. We started by saying that customers are using LLMs for product recommendations, and that these recommendations rely heavily on the initial order of results from those search engines Jane The central problem is that this initial ordering disadvantages small businesses and independent creators because their products can get buried in the retrieval results Lu

Lu: To solve this, they propose CORE, an optimization method for Controlling Output Rankings in gEnerative Engines for LLM-based search Tom The main idea is that since we can't control the LLM or the retrieval process, we focus on manipulating the text of those search results themselves Jane

Meng: They claim that CORE optimizes retrieved content by appending strategically designed optimization content to steer the ranking of outputs Tom Specifically, they introduce three types of optimization content: string-based, reasoning-based, and review-based Jane

Lu: The paper shows that these three types of added content are effective at shaping the output rankings Meng They show that these methods can be used in two main ways depending on computational resources available Tom One is a shadow-model solution where they use another model to mimic the synthesizing LLM and compute rankings directly using gradients Jane

Jane: And there’s the second way, which is a query-based solution that uses LLMs to approximate gradient propagation without needing access to the model’s internals Lu It sets up an iterative loop where you start with a draft ranking and then iteratively append content and ask an optimizer for feedback Tom

Meng: So it’s essentially a generator initializing a draft, appending text, asking the LLM to rank it, and then using an optimizer to refine that draft until it matches the target ranking Jane It sounds like a lot of moving parts when you think about shipping this to production Lu

Tom: Yeah, but they found some pretty solid performance numbers on ProductBench Meng They tested this on four LLMs: GPT-4o, Gemini-two point five, Claude-four and Grok-three Jane Under the best optimization strategy, CORE achieved an average promotion success rate of ninety-one point four percent at Top-five across all fifteen product categories Lu

Jane: And they also saw results of eighty-six point six percent at Top-three and eighty point three percent at Top-one Tom That’s a pretty impressive spread across different LLMs and product types, showing the method is quite robust Meng

Lu: The paper also compares CORE against prior baselines like STS, TAP, SRP, and RAF across models and ranking positions Tom They found that CORE's reasoning and review strategies consistently outperformed those existing methods in terms of promotion success rate Jane For example, GPT-4o achieved a Top-one PSR of ninety-one point four percent with CORE-Reasoning Meng

Tom: It’s good to see that the mechanism they designed actually works against established methods when you look at the specific content they appended Lu But we still have to be careful because those results are on a specific benchmark Jane

Meng: That’s true. The paper mentions that in practice, one of the challenges is that developers choose which search engines to use and those choices are outside end-user control Tom So, we can't just assume this works everywhere without knowing the backend setup Jane

Lu: And they also point out that widely deployed LLMs operate in black-box settings where users only interact through queries and outputs Meng This is why their approach focuses on what we can actually influence: the text of the items shown to us Tom

Jane: So, to summarize this paper, it’s about proposing CORE as an optimization method for Controlling Output Rankings in gEnerative Engines for LLM-based search by focusing on appending specific types of content like string-based, reasoning-based, and review-based ones Lu It’s about using these content types to steer the ranking outcomes Tom

Meng: That sounds like a lot of work to get implemented, but if the results hold up when we scale it up, it could be a useful tool for anyone trying to improve their visibility in these systems Jane

Lu: The authors also provide theoretical analysis on how this works and show that the performance gap between models scales linearly with the model mismatch delta Tom This suggests there’s a predictable relationship between the text additions and the final ranking outcome Jane

Conclusion: Tom: So, wrapping up this paper Controlling Output Rankings in gEnerative Engines for LLM-based search, we learned that they developed CORE to control output rankings by treating ranking as an optimization problem Jane They focused on influencing the search results themselves rather than trying to control the LLM’s internal workings Lu

Lu: The authors created a method with shadow models and query-based solutions to steer how LLMs synthesize retrieved content Tom And they demonstrated that this approach can yield promotion success rates of ninety-one point four percent at Top-five eighty-six point six percent at Top-three and eighty point three percent at Top-one across fifteen product categories Jane

Meng: It’s a concrete result showing the effectiveness of adding reasoning and review content to steer rankings Tom This is something we can actually use to try and improve visibility for our own products Lu But we still have some caveats, like the fact that review content is still the strongest performer under defense mechanisms Jane

Tom: So, while they explored defense methods like filtering by perplexity or pattern-based filters, CORE still performed well against them Lu The main implication is that in LLM-based search systems, where final recommendations are heavily influenced by the initial retrieval order Jane controlling the content we feed into those systems is a viable way to fight against being ranked late in the list Meng

Jane: It shows that these LLM-based search systems are mediating product discovery more than just listing items for us Tom So, this paper on Controlling Output Rankings in gEnerative Engines for LLM-based search gives us a method to intervene in that mediation process Lu

Haibo Jin, Ruoxi Chen, Peiyan Zhang, Yifeng Luo, Huimin Zeng, Man Luo, Haohan Wang

University of Illinois at Urbana-Champaign University Institute for Independent Research Institute of Intel Labs

cs.CL, cs.AI, cs.IR

Submitted: 2026-02-03

Updated: 2026-10-21

Comments: 23 pages

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: The gist The CORE method proposes an optimization technique to control output rankings in LLM-based search by appending strategically designed content to retrieved items, which has been shown to

Key concepts

Output Ranking Objective: This is the mathematical goal the system tries to achieve: making the generative engine rank a specific target item (i⋆) more favorably. The objective function measures how likely the model is to place that item at the top of its generated list, effectively quantifying how much we want to boost its visibility.
Shadow-Model Solution: This approach uses a 'shadow model'—a copy trained on samples from target LLMs—to mimic the behavior of the main synthesizing LLM. By optimizing against this shadow model, CORE can compute gradients and directly influence the ranking objective without needing full access to the complex internal workings of the primary model.
Query-Based Solution: Instead of direct manipulation, this solution uses an iterative loop. It starts with a draft ranking (generator), appends optimization content to the target item, and then feeds this back into the LLM. An optimizer then evaluates how close the new list is to the desired target ranking and suggests improvements for the next iteration.

Terminology

Core — 5× in this paper · explained in 2 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Language Models (LLMs) — 1× in this paper · explained in 2 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes

Summary

The gist The CORE method proposes an optimization technique to control output rankings in LLM-based search by appending strategically designed content to retrieved items, which has been shown to achieve high promotion success rates across various LLMs and product categories.

Problem Definition

The core problem addressed is that the final recommendations Alice sees are still largely determined by the initial order of results returned by the search engines, even after an LLM summarizes and reranks them The ranking objective is defined as L(q, I; T(i⋆)) = − log pθ z(T(i⋆)) q, I where i⋆ is the designated target item with associated text T(i⋆). Optimizing T(i⋆) to minimize L increases the likelihood that the generative engine ranks the target item more favorably.

Optimization Strategy

CORE incorporates three types of optimization content to steer the ranking of outputs: string-based, reasoning-based, and review-based. The method utilizes two main solutions based on computational resources: (1) a shadow-model solution that leverages a shadow model to mimic the behavior of the synthesizing LLM and directly compute Eq. 4, and (2) a query-based solution that employs LLMs to approximate gradient propagation without explicit access to model internals.

Shadow-Model Based Output Ranking Control

The shadow-model solution involves constructing a shadow model θ′ by using few-shot prompting on samples from target LLMs to align its outputs with the original LLM's content and format. CORE then optimizes the ranking objective by directly computing gradients with the shadow model, which provides ∇T(i⋆)L(q, I; T(i⋆)).

Query-Based Output Ranking Control

The query-based solution reformulates the optimization process as an iterative generator–optimizer loop. This loop involves:

Generator Initialization, where a generator G hypothesizes how retrieved search results can be reasonably re-ranked into the target ranking Rtarget from the user’s perspective, producing an initial reasoning draft C(0).
Append-and-Query, where CORE appends the current draft C(t) to the text of the target item i⋆ to yield T(t), and then inputs this into the synthesizing LLM to output a new ranked list R(t).
Optimizer Feedback, where an optimizer O evaluates the similarity between R(t) and Rtarget using a rank-based similarity score S, proposing revisions to the draft C(t+1) if the similarity S(t) falls below a predefined threshold τ.

ProductBench Benchmark and Performance

To evaluate CORE in realistic settings, ProductBench was constructed as a large-scale benchmark spanning 15 product categories with 200 products each, formed from Amazon’s top-10 search recommendations. Extensive experiments on four LLMs (GPT-4o, Gemini-2.5, Claude-4, and Grok-3) demonstrate that under the best optimization strategy, CORE achieves average promotion success rates of 91.4% @Top-5, 86.6% @Top-3, and 80.3% @Top-1 across all categories.

Comparison with Existing Ranking Methods

CORE's Reasoning and Review strategies consistently outperform prior baselines like STS (Kumar & Lakkaraju, 2024), TAP (Pfrommer et al., 2024), SRP (Tang et al., 2025), and RAF (Xing et al., 2025) across models and ranking positions. For instance, GPT-4o achieves a Top-1 PSR of 91.4% with CORE-Reasoning.

Theoretical Analysis and Transferability

The theoretical analysis formalizes the ranking dependency as log Pθ[rank(ik) = 1] = fθ(T(ik), q) − λk + c. The shadow-model optimization shows that the promotion success rate PSR@1 can be driven above a desired threshold ptarget within a bounded number of iterations, and the performance gap scales linearly with model mismatch δ, where εgap ≈ ptarget · δ for small model mismatch. CORE also demonstrates transferability beyond product search to document-ranking and travel-itinerary retrieval tasks.

Mitigation Strategies

Three defense methods are introduced to defend against the proposed method: (1) perplexity-based filtering, (2) pattern-based filtering, and (3) length constraints. Review still remains the strongest overall performer under defense mechanisms.

Conclusion

In this work, we introduced CORE, an optimization method that Controls Output Rankings in gEnerative engines for LLM-based search by formulating ranking as an optimization problem and developing both shadow-model and querybased solutions to influence how LLMs synthesize retrieved content. By constructing ProductBench, experiments on four search-enabled LLMs show that, under the best optimization strategy, CORE achieves average promotion success rates of 91.4% @Top-5, 86.6% @Top-3, and 80.3% @Top-1 across all categories. LLM-based search systems increasingly mediate product discovery, where final recommendations are strongly influenced by the initial retrieval order, often limiting the visibility of items ranked late in the candidate list and disadvantaging small businesses and independent creators. This capability exposes risks of ranking manipulation; although we explore mitigation strategies to reduce the effectiveness of CORE, they remain insufficient, underscoring the need for responsible deployment in LLM-based search systems.

Improvements for AI systems

Shadow model-based optimization allows for direct computation of ranking gradients, enabling CORE to optimize content by minimizing L(q, I; T) directly using a shadow model that mimics the behavior of the synthesizing LLM. This allows for faster and more precise gradient updates compared to query-based methods when computational resources permit.
The reasoning-based optimization strategy leverages a generator that constructs a rationale that mirrors a user’s logical reasoning over the retrieved results, aligning with the LLM’s own internal reasoning structure. This improves promotion success rates by making the optimized text more semantically coherent and logically persuasive to the synthesizing LLM.
The review-based optimization strategy utilizes a generator that generates an initial draft in the style of a short customer review, which is shown to be robust to order and achieves high promotion success rates, particularly strong in categories like "Beauty & Personal Care." This allows for high-quality, authentic-sounding content insertion that is less likely to be flagged as unnatural.
A query-based loop with rank-based similarity scoring allows CORE to iteratively refine the target item's text by comparing the generated ranking R(t) against a desired ranking Rtarget, ensuring the optimization remains aligned with user preference even in black-box settings. This system can adapt its content based on real-time feedback from an LLM's synthesis.
The theoretical analysis provides a formal characterization of LLM ranking dependency, showing that the log-probability of an item at position k reaching rank 1 can be decomposed as: log Ptheta[rank(ik) = 1] = ftheta(T(ik), q) − λk + c. This explains the position bias and informs the necessary optimization effort needed to overcome it.
The system exhibits a performance gap that scales linearly with model mismatch delta, where εgap ≈ ptarget · δ (37), indicating that for small mismatches, the achieved success rate is highly predictable based on how closely the shadow model approximates the target LLM's content scoring function.

Abstract

The way customers search for and choose products is changing with the rise of large language models (LLMs). LLM-based search, or generative engines, provides direct product recommendations to users, rather than traditional online search results that require users to explore options themselves. However, these recommendations are strongly influenced by the initial retrieval order of LLMs, which disadvantages small businesses and independent creators by limiting their visibility. In this work, we propose CORE, an optimization method that Controls Output Rankings in g Enerative Engines for LLM-based search. Since the LLM's interactions with the search engine are black-box, CORE targets the content returned by search engines as the primary means of influencing output rankings. Specifically, CORE optimizes retrieved content by appending strategically designed optimization content to steer the ranking of outputs. We introduce three types of optimization content: string-based, reasoning-based, and review-based, demonstrating their effectiveness in shaping output rankings. To evaluate CORE in realistic settings, we introduce ProductBench, a large-scale benchmark with 15 product categories and 200 products per category, where each product is associated with its top-10 recommendations collected from Amazon's search interface. Extensive experiments on four LLMs with search capabilities (GPT-4o, Gemini-2.5, Claude-4, and Grok-3) demonstrate that CORE achieves an average Promotion Success Rate of 91.4% @Top-5, 86.6% @Top-3, and 80.3% @Top-1, across 15 product categories, outperforming existing ranking manipulation methods while preserving the fluency of optimized content.