ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation

arXiv:2512.04350 · cs.CL · Submitted 2025-12-04 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation".

Jane: Here's the gist of the paper. The gist: CLUSTERFUSION is a hybrid framework that treats Large Language Models as the clustering core, guided by lightweight embedding methods to achieve state-of-the-art performance on standard and domain-specific text clustering tasks.

Tom: How it works CLUSTERFUSION rethinks the relationship between traditional clustering and LLMs by treating the LLM as the clustering core through two main tasks: topic summarization and topic assignment, which is described in section 2 CLUSTERFUSION Framework.

Jane: This design enables direct incorporation of domain knowledge and user preferences by using prompts to specify what kind of clusters should be formed (e.g., whether ambiguous utterances such as “yes,” “no,” and “unsure” should be separated or grouped) and how individual records with domain terminology should be interpreted (page 1).

Tom: The framework proceeds in three stages: embedding-guided subset partition, LLM-driven topic summarization, and LLM-based topic assignment (page 2).

Paper summary: Tom: So we're talking about this paper called "ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation". It looks like they're trying to solve a problem where traditional clustering methods with pre-trained embeddings aren't working well in specific areas without needing a lot of fine-tuning.

Jane: Yeah, so the big idea here is that large language models, or LLMs, have really good contextual reasoning abilities, but most research has only used them as extra tools to tweak embeddings or adjust where the clusters are drawn.

Lu: This paper proposes something different by treating the LLM as the main clustering core itself and guiding it with some lightweight embedding methods. It's a hybrid approach that combines both ideas.

Meng: So, how does this framework actually work? We need to see what they're doing step by step.

Tom: Right, they break it down into three stages: first you have embedding-guided subset partition, then there's LLM-driven topic summarization, and finally LLM-based topic assignment. That’s the structure of CLUSTERFUSION.

Jane: And what this design aims to do is allow them to directly put domain knowledge and user preferences into the process by using prompts to tell the AI how it should group things or interpret specific terms.

Lu: The first stage is embedding-guided subset partition, where they use embeddings to organize the data into a manageable sample, and then they sort those samples in a couple of ways before sending them to the LLM.

Tom: And that sorting is important because it's not just random; they test two strategies: one based on which cluster a record belongs to, and another based on how similar the embedding is to the first sample.

Meng: What about the numbers? How do they handle keeping things balanced when you’re sampling, especially for smaller groups?

Jane: They use balanced sampling where if a group has enough samples, they take them without replacement to keep diversity high, but if a group is smaller than the target sample size, they sample with replacement to hit that target size.

Lu: Then there's the sorting part again. They get an ordering called pi cluster based on the predicted group index and pi sim based on embedding similarity, and they choose whichever one works better for readability before concatenating them into a final sorted sample D′S.

Tom: That pre-organization is key because they found that it really improves the quality of the topic summaries the LLM generates later on.

Jane: So, once you have that sorted sample, in the second stage, they take those records and feed them into a prompt for an LLM to generate candidate cluster topics along with brief definitions.

Meng: And what’s really interesting is how they use that prompt to inject domain knowledge directly into the summarization step.

Lu: It lets them specify things like whether ambiguous words, such as "yes," "no," or "unsure," should be split up or grouped together in the final topic list.

Paper summary: Tom: And then finally, in the third stage, topic assignment where every single record from the original dataset gets checked against those candidate topics by a separate LLM call.

Jane: This is where they use a second prompt to classify each record into one of those generated topics and they actually rerun the call if the AI gives them an invalid label.

Lu: The results show that this method matches or surpasses state-of-the-art methods on standard benchmarks, and it also shows substantial improvements on newly constructed domain-specific datasets.

Meng: I'm looking at those numbers now. On the OpenAI Codex dataset, they show accuracy improving from forty-four point five to sixty-six point zero which is a forty eight percent increase over that baseline, which is pretty significant for a practical deployment model like we're building here.

Tom: It’s not just about the raw accuracy; qualitatively, they showed that you can inject domain expertise into both the summarization and assignment prompts to improve how well the AI understands specific terminology.

Jane: And they also showed that ordering matters for the summary quality, with performance improving by twenty two percent on the Tweet dataset just because they pre-organized the records either by cluster assignments or by embedding similarity.

Lu: They also pointed out a limitation: for very large datasets, you might still need a divide-and-conquer strategy due to context window limits. And they also noted that it assumes you already know how many clusters, K, which isn't always realistic.

Meng: From an engineering standpoint, the authors say this framework is training-free and cost-efficient compared to methods that learn representations like SCCL or DSE, so it’s good for production if we can manage the inference costs.

Tom: It’s definitely a simpler design than those representation learning approaches, which means less overhead on our side when we're trying to get something running reliably.

Jane: So, thinking about the whole picture of "ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation", what do you guys think is the main implication here for how people approach text clustering in specialized fields?

Lu: It suggests that instead of just relying on pre-trained embeddings to define clusters, we can use those embeddings as a guide to structure the data before letting the powerful contextual reasoning of an LLM actually do the heavy lifting of defining those topics.

Meng: For us, it means we can bypass some of that expensive fine-tuning and get high performance just by carefully crafting prompts with domain context.

Tom: It seems like they’ve found a way to make LLMs work as the core clustering engine instead of just a helper module for embedding refinement.

Jane: And for someone who doesn't read the paper, it means you can use your knowledge of the data structure to tell the AI exactly how you want it organized, making those complex tasks much more accessible.

Conclusion: Tom: So we're wrapping up our look at ClusterFusion, which is titled "ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation."

Jane: Basically, this paper is about using Large Language Models as the main engine for clustering instead of just using them as a side tool.

Lu: Right. They treat the LLM like the core process—the thing that does the actual grouping—but they guide it with some simpler embedding methods.

Meng: I'm curious about what this means in practice for us, Tom, if we're trying to deploy something at scale without massive retraining costs.

Tom: Well, they show results on three standard datasets and two new ones, like OpenAI Codex and Adobe Lightroom. And the numbers are pretty solid; they hit state-of-the-art accuracy across the board.

Jane: What’s really interesting is how they inject domain knowledge directly into the process using prompts for both summarizing topics and assigning records to them.

Lu: They also tested different ways to order the data before feeding it to the LLM, and that pre-organization made a noticeable difference in summary quality.

Meng: So, what's the real cost here? Is this something we can actually run reliably on our systems right now?

Tom: The authors say it’s training-free and cost-efficient compared to other representation learning methods, which is good news for deployment.

Jane: But they do have a few caveats, like needing prior knowledge of how many clusters you want before you start.

Lu: And they also flag that for really big datasets, the context length of the LLM could still be an issue if we don't use a smarter way to break things down.

Meng: That sounds like a hurdle for production, but if the accuracy gains are this much, I might see it as worth investigating further.

Tom: Exactly. It suggests that instead of just tweaking embeddings, we can leverage the LLM’s reasoning power directly for clustering tasks in specialized fields.

Jane: It shifts the focus from how good our initial embeddings are to how well we can prompt the AI with our specific domain needs and preferences.

Yiming Xu, Yuan Yuan, Vijay Viswanathan, Graham Neubig

Adobe · Carnegie Mellon University

cs.CL

Submitted: 2025-12-04

Updated: 2026-10-20

Code: https://github.com/YimingXu1213/clusterFusion

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: The gist: CLUSTERFUSION is a hybrid framework that treats Large Language Models as the clustering core, guided by lightweight embedding methods to achieve state-of-the-art performance on standard and

Key concepts

Embedding-Guided Subset Partition: This initial stage uses traditional embeddings to organize a large dataset into a smaller, representative sample. It involves grouping records using an embedding model, then sampling them carefully to ensure diversity across different groups. Sorting is also applied based on either cluster membership or similarity to improve the quality of the input for the LLM.
LLM-Driven Topic Summarization: In this stage, a selected subset of data is fed into an LLM with a prompt instructing it to abstract the text into concise candidate topics and brief definitions. This leverages the LLM's contextual understanding, allowing domain knowledge or user preferences to be explicitly included in the prompt.
LLM-Based Topic Assignment: The final stage involves using separate calls to an LLM to assign every record in the original dataset to one of the pre-generated candidate topics. This step ensures each piece of text is accurately categorized based on the topics summarized by the LLM, with a mechanism to re-run if an invalid label is produced.

Terminology

Clustering — 4× in this paper · explained in 2 episodes
Benchmark — 2× in this paper · explained in 2 episodes
Embeddings — 2× in this paper · explained in 2 episodes
Adaptability — 1× in this paper
Adaptation — 1× in this paper
Core — 1× in this paper · explained in 2 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Language Models (LLMs) — 1× in this paper · explained in 2 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 9 episodes

Summary

How it works

CLUSTERFUSION rethinks the relationship between traditional clustering and LLMs by treating the LLM as the clustering core through two main tasks: topic summarization and topic assignment, which is described in section 2 CLUSTERFUSION Framework. This design enables direct incorporation of domain knowledge and user preferences by using prompts to specify what kind of clusters should be formed (e.g., whether ambiguous utterances such as “yes,” “no,” and “unsure” should be separated or grouped) and how individual records with domain terminology should be interpreted (page 1). The framework proceeds in three stages: embedding-guided subset partition, LLM-driven topic summarization, and LLM-based topic assignment (page 2).

Embedding-Guided Subset Partition

The first stage leverages embedding-based methods to transform the input dataset into a manageable, representative, and well-organized sample (page 3). This stage consists of two substeps: Grouping and Balanced Sampling, followed by Sorting.

Grouping and Balanced Sampling involves computing embedding representations Z = Embedder(DN) (page 3), applying a traditional clustering algorithm such as KMeans to partition the embeddings into M groups G = Group(Z, M) (page 3), and then constructing a representative sample DS of size S by performing balanced sampling where for groups with sufficient samples (Gm ≥ ⌊S/M⌋, we sample without replacement to ensure diversity, and for smaller groups (Gm < ⌊S/M⌋, we sample with replacement to meet the target sample size (page 3).
Sorting involves arranging the sampled records DS = Sample(Gm, ⌊S/M⌋) before concatenation using two strategies: Cluster-based ordering πcluster = arg sort(group(xsi)) (page 3), or Similarity-based ordering πsim = arg sort(sim(zs1, zsi)) (page 3). The final sorted sample is then D′S = Sample(DS) where π ∈ ⌊πcluster, πsim⌋ (page 3). This pre-organization improves readability and reduces fragmentation, enabling the LLM to generate more consistent and higher-quality topic summaries (page 3).

Summarization: Topic Extraction

The second stage involves topic extraction where the sampled and sorted records are concatenated into a prompt for the LLM, which is instructed to abstract the corpus into a concise set of candidate topics along with brief definitions (page 2). This step leverages the LLM’s strong contextual reasoning abilities and enables flexible incorporation of domain knowledge or user preferences directly into the prompt (page 2). The complete prompt templates and domain-specific examples are provided in Appendix A (page 11).

Classification: Topic Assignment

The third stage is topic assignment where each record in the original dataset is then independently assigned to one of the candidate topics through a separate LLM call (page 2). The model is constrained to choose from the topic list produced in the Topic Extraction step; if it outputs an invalid label, the call will be rerun (page 2). The full prompt template for this classification step is also detailed in Appendix A (page 12).

Experimental Results and Analysis

The experiments evaluate CLUSTERFUSION on three public benchmarks—Bank77, CLINC, and Tweet—and two new domain-specific datasets: OpenAI Codex and Adobe Lightroom (page 4). Quantitative comparison shows that CLUSTERFUSION consistently achieves the highest accuracy among state-of-the-art methods across all five datasets (page 5). For example, on OpenAI Codex, accuracy improves from 44.5 to 66.0, a relative increase of 48% (page 5). Qualitative analysis demonstrates that domain expertise can be injected into both summarization and assignment prompts to improve terminology understanding, and user preferences about which clusters should or should not be formed can be specified during the summarization stage (page 5). Furthermore, ordering matters to summarization quality, as performance improves consistently when records are pre-organized—whether by cluster assignments or by embedding similarity—with accuracy rising by 22% on the Tweet dataset solely due to improved ordering (page 6). The framework is training-free and cost-efficient compared to representation-learning approaches like SCCL or DSE, delivering substantially higher accuracy at comparable costs (page 8).

Limitations and Ethical Considerations

Limitations include that for very large datasets, context length constraints may still require a divide-and-conquer strategy, though the sampling and ordering design could be naturally extended to batch settings (page 9). The framework also assumes prior knowledge of the number of clusters K, which may not always be realistic, suggesting future work could explore strategies for automatically estimating or adapting the number of clusters (page 9). Ethical considerations address data privacy by using public benchmarks and newly constructed datasets that are free of personally identifiable information (PII) and comply with their respective terms of use (page 9). Additionally, the paper acknowledges potential societal or linguistic bias from the LLM component, suggesting future work could explore fairness-aware prompting or bias auditing (page 9). The authors document each step of the pipeline in detail to support reproducibility (page 9).

Acknowledgments

This work was conducted under the support of the Adobe Experience Intelligence team, and authors thank Lei Liu, Austin Xu, Pradeep Arockiasamy, William Yan and Roger Brooks for their valuable feedback and support throughout the development of this work (page 10). The paper concludes by summarizing that CLUSTERFUSION provides a training-free, cost-efficient, simple-designed, and easily adaptable solution for clustering across diverse domains (page 10).

The gist: CLUSTERFUSION is a hybrid framework that treats Large Language Models as the clustering core, guided by lightweight embedding methods to achieve state-of-the-art performance on standard and domain-specific text clustering tasks. The paper presents CLUSTERFUSION, a hybrid framework that rethinks the relationship between traditional clustering and LLMs by treating the LLM as the clustering core through two main tasks: topic summarization and topic assignment, which is described in section 2 CLUSTERFUSION Framework. This design enables direct incorporation of domain knowledge and user preferences by using prompts to specify what kind of clusters should be formed (e.g., whether ambiguous utterances such as “yes,” “no,” and “unsure” should be separated or grouped) and how individual records with domain terminology should be interpreted (page 1). The framework proceeds in three stages: embedding-guided subset partition, LLM-driven topic summarization, and LLM-based topic assignment (page 2).

Classification: Topic Assignment

The third stage is topic assignment where each record in the original dataset is then independently assigned to one of the candidate topics through a separate LLM call (page 2).

Improvements for AI systems

Improve domain-specific text clustering by treating Large Language Models as the core clustering mechanism instead of auxiliary modules, enabling direct incorporation of domain knowledge and user preferences into cluster formation. This allows the system to fully leverage the contextual adaptability of LLMs.
Implement a three-stage pipeline: embedding guided subset partition, LLM driven topic summarization, and LLM based topic assignment. This structure mitigates the context window bottleneck by first using embedding-based methods to partition the corpus and select representative, ordered exemplars before feeding them to the LLM for summarization.
Enhance topic extraction quality by using domain knowledge in prompts, as shown in Appendix A.1.1 for OpenAI Codex, where taskspecific context and preferences are provided to guide topic formation, leading to better interpretation of domain terminology like competitor mentions versus general criticism.
Improve cluster coherence and LLM output stability by incorporating exemplar ordering into the subset partition stage using strategies like Cluster-based ordering or Similarity-based ordering. This reduces fragmentation because pre-organization improves readability and reduces fragmentation, enabling the LLM to generate more consistent and higher-quality topic summaries.
Increase performance on specialized domains by explicitly prompting the LLM to enforce cluster constraints, such as in Table 4 examples where instructions are given on how to handle ambiguous utterances like yes, no, and unsure.