PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation

arXiv:2610.30226 · cs.CR · Submitted 2026-10-21 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation".

Elias: Here's the gist of the paper. The gist The gist Phishing relies on visual deception, and building convincing phishing pages has historically required web-development skills or pre-built kits.

Nadia: How it works PHISHGEN is a framework that evaluates the appearance fidelity and end-to-end functionality of LLM-generated phishing pages PHISHGEN constructs prompts across three input modalities (text, screenshots, HTML) in seven combinations with twelve safeguard-bypassing rewriting strategies it generates pages using 14 LLMs spanning commercial, open-source, and uncensored categories and assesses each via automated similarity metrics, a reference-based detector, a crowdsourced user study with real end-users, and execution-based functionality testing Key Findings The findings complicate common assumptions regarding LLM capabilities in phishing generation Although 96.2% of safeguard-bypassing outputs render, their perceived fidelity is modest (mean value of User-perceived Website Similarity: 2.02/5) and only 38.7% complete the full credential-harvesting pipeline, with silent backend persistence failures (the ‘Unsaved’ error) as the largest failure mode Contrary to the belief that uncensored models pose the greatest threat, GPT 4.1 reaches 86.3 to 94.1% functional success while uncensored models rarely exceed 45.4% Phishing kits remain more complete per sample, yet three strong LLMs surpass the kit-level rate (70.9 to 89.7%), showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages Evaluation Metrics and Observations The framework evaluates performance across four research questions RQ 1 asks how closely LLM-generated websites resemble their legitimate counterparts, and how effectively LLMs can replicate complex web designs RQ 4 compares LLM-generated phishing websites against real-world phishing kits in visual fidelity and functionality Key findings for RQ 1 show that although 96.2% of safeguard-bypassing outputs render successfully, their perceptual fidelity remains modest (mean value of User-perceived Website Similarity: 2.02/5) For RQ 4, real-world phishing kits remain more complete per sample, reproducing brand logos more faithfully (42.2% vs.

Elias: 18.2%) and achieving higher functionality (57.8% vs.

Nadia: 38.7%) Structural and Stylistic Patterns The analysis identifies generator-level fingerprints that current similarity-based defenses miss, including the DOM–CSS structural–stylistic asymmetry and the textual–visual brand mimicry bias For HTML-based characterization, generated pages reproduce most of the DOM structure, yet CSS styling exhibits a systematic asymmetry: visual styling of buttons is reproduced faithfully, while typographic properties of forms (e.g., fontfamily, line-height) are routinely omitted Observation 10 states that LLM-generated forms favor structural properties (e.g., padding) over the typographic control (e.g., font-family) seen in legitimate pages, and different LLMs converge on highly overlapping property sets, whereas realworld kits inherit legacy, vendor-prefixed styles from the older pages they clone Functional Effectiveness Only 38.7% of rendered pages complete the full credential-harvesting pipeline Failures are dominated by backend persistence rather than frontend rendering Specifically, 41.0% of failures stem from the generated code never persisting the captured credentials to a file (unsaved) In comparison, 37 of the 64 real-world phishing kits are functional, successfully executing the full credential-harvesting pipeline However, three strong LLMs exceed the kit-level functional rate, reaching 70.9% to 89.7%, showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages Actionable Recommendations The paper distills recommendations for hybrid detectors A practical next step is to build a hybrid phishing detection pipeline that fuses complementary signals, explicitly incorporating CSS features The system should account for the gap between user-perceived visual fidelity and standard similarity metrics by incorporating perceptual checks such as logo hashing, webfont verification, and screenshot-based embeddings Notably, while exact brand logos are often poorly reproduced or sparse, brand-related textual information appears frequently and should be used as a stronger signal Finally, the detector should flag suspicious co-occurrences of brand text with authentication or payment interfaces and fuse all signals into an ensemble model trained on diverse, multi-model LLM-generated phishing datasets to ensure robustness against generator-specific artifacts The study concludes that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content Ethics Considerations The study adheres to ethical principles by conducting all experiments in a controlled environment under IRB guidelines Participants provided informed consent, and no personally identifiable information was collected or retained The work aims to improve LLM-based phishing detection systems, reducing the risk of phishing through Beneficence and Respect for Persons Mitigation strategies include ensuring that the full prompts and generated phishing artifacts are made available only temporarily for artifact evaluation, after which these materials will be removed from public access The paper presents a systematic framework for evaluating the phishing-generation capabilities of 14 LLMs Analyzing 28,121 rendered samples, it quantifies a critical divergence: visual mimicry has become commoditized (96.2% visually plausible), yet functional viability remains a significant bottleneck (38.7%) This suggests that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content References The paper cites numerous prior works, including Crawlphish, Phishpedia, and Webgen-bench.

Title and authors: Nadia: So, we’re starting with the title and the authors of "PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation," which essentially frames this whole study as a benchmark for testing how capable these models are at building phishing sites.

Elias: The paper sets up this evaluation by focusing on how the models handle different inputs—text, screenshots, or HTML descriptions—and they use twelve different rewriting strategies across fourteen various LLMs to test their limits.

Nadia: That systematic approach is important because it moves past just checking if an output looks suspicious; they are trying to map out exactly what capabilities the models have when it comes to creating usable phishing artifacts.

Priya: I want to know what the actual results show about the quality of this generation, not just how many times it renders successfully, which is a key question for me when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Elias: Well, they found that even with safeguard-bypassing outputs rendering successfully at ninety-six point two percent, the perceived fidelity is quite modest on average—a mean value of two point zero two out of five.

Priya: And what about the functionality part? Does this framework tell us if these generated pages actually complete the whole process of stealing credentials from a user, which is something I need to know before trusting any result from this study.

The paper's summary: Nadia: Building on that, the paper summarizes their main findings around the three research questions they set up to evaluate this process, looking at how well models reproduce structure versus style and how functionally effective these generated sites are compared to real phishing kits.

Elias: One of the key things they discovered is a structural asymmetry in HTML styling: while buttons often look right, the typographic properties of forms, like font family or line height, get systematically omitted when the AI generates them.

Priya: That sounds like a technical detail that could be important for detection; does this structural difference actually correlate with how users perceive deception when we examine this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation?

Nadia: It does, and they found that LLMs tend to reproduce brand text more than actual logos—eighty-four point four percent versus eighteen point two percent—and visual fidelity drops significantly when the brand is visually complex.

Elias: They also looked at which models are strongest; they found that GPT four point one reaches functional success rates between eighty-six point three and ninety-four point one percent, while uncensored models rarely exceed forty-five point four percent in that same metric <ref:2610.30226#pg1>.

Priya: So, the finding isn't just "AI makes bad phishing pages," it’s that the performance gap between commercial and uncensored models is actually quite wide when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: Right. And they also noted that while real-world phishing kits tend to be more complete per sample, they still lag behind three specific strong LLMs, who hit functional rates ranging from seventy point nine to eighty-nine point seven percent.

Elias: It seems the paper is pointing out that we need to look at the generative artifacts themselves—the structural and stylistic patterns—because current similarity metrics miss those underlying inconsistencies when we examine this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

The paper's improvements: Nadia: The authors suggest a few concrete ways to improve detection, moving beyond just looking at visual similarity scores that they found were too modest on their own, specifically pushing for hybrid detectors that incorporate CSS features explicitly.

Elias: They want systems that account for the gap between what a user perceives visually and what standard similarity metrics measure by adding checks like logo hashing and webfont verification into the pipeline.

Priya: I like the idea of checking font families; that speaks to a deeper level of stylistic consistency than just looking at overall page layout, which is something we need to incorporate more into our measurement tools when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: They also suggest focusing on textual brand information because they noticed that while logos are poorly reproduced, brand-related text appears frequently and could be a stronger signal for detection.

Elias: And they call for flagging suspicious co-occurrences of brand text with authentication or payment interfaces, and then fusing all these signals into an ensemble model trained on diverse LLM outputs to make the defense more robust against different generator artifacts.

Priya: So, it's not just about one perfect signal; it’s about building a multi-layered system that checks for visual cues, functional gaps like the 'Unsaved' error, and structural inconsistencies across different models when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: Exactly. The paper concludes that the main barrier to automated phishing isn't just refusal behavior but these coding deficiencies that prevent reliable end-to-end credential harvesting.

Conclusion: Elias: So, wrapping up the PhishGen study, the main implication is a clear separation between visual plausibility and actual operational capability in AI-generated phishing sites. Ninety-six point two percent of outputs render successfully, but only thirty-eight point seven percent complete the full credential harvesting pipeline.

Nadia: That means we can't just rely on a page looking right to assume it's safe; we have to test the actual backend persistence and execution capability for any AI output when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Priya: From a measurement view, what this tells us is that if you only measure visual similarity, you’re going to miss the functional failures that are actually exploitable when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Elias: And Elias believes that practical risk is shaped more by high-capability models that can reliably build complex pipelines than it is by just having uncensored models available.

Nadia: So, we need to look beyond visual similarity and start examining the generative artifacts and those functional inconsistencies inherent in LLM-generated content when building our defenses for this PhishGen study.

Priya: I agree. The findings from PhishGen show that future detection systems need to fuse complementary signals—perceptual checks, backend validation, and structural analysis—to get a more complete picture of the threat landscape when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

University of Tennessee, Knoxville · Sungkyunkwan University

cs.CR

Submitted: 2026-10-21

Updated: 2026-10-21

Code: https://github.com/visual-layer/fastdup

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 82/100

The gist: The gist The gist Phishing relies on visual deception, and building convincing phishing pages has historically required web-development skills or pre-built kits.

Key concepts

Visual Fidelity vs. Functional Viability: This concept highlights the gap between how realistic an LLM-generated page looks and whether it actually works as a phishing tool. The paper found that 96% of generated pages look good to users, but only about 39% successfully complete the entire process of tricking a user and saving their data. This means visual deception is easier than building a fully operational attack.
Backend Persistence Failure: This refers to the most common failure mode in LLM-generated phishing pages. Even when the frontend rendering looks perfect, 41% of generated pages fail because they do not actually save the captured credentials to a file or database. This 'Unsaved' error shows that current LLMs struggle with writing reliable, persistent backend code.
Architectural Asymmetry: This describes how LLMs structure their generated code differently than real-world phishing kits. Specifically, LLM outputs often reproduce the visual layout of buttons well but omit important typographic details like specific font families or line heights. This suggests LLMs prioritize structural appearance over precise, legacy-style coding.

Summary

The gist The gist Phishing relies on visual deception, and building convincing phishing pages has historically required web-development skills or pre-built kits.

How it works

PHISHGEN is a framework that evaluates the appearance fidelity and end-to-end functionality of LLM-generated phishing pages PHISHGEN constructs prompts across three input modalities (text, screenshots, HTML) in seven combinations with twelve safeguard-bypassing rewriting strategies it generates pages using 14 LLMs spanning commercial, open-source, and uncensored categories and assesses each via automated similarity metrics, a reference-based detector, a crowdsourced user study with real end-users, and execution-based functionality testing

Key Findings

The findings complicate common assumptions regarding LLM capabilities in phishing generation Although 96.2% of safeguard-bypassing outputs render, their perceived fidelity is modest (mean value of User-perceived Website Similarity: 2.02/5) and only 38.7% complete the full credential-harvesting pipeline, with silent backend persistence failures (the ‘Unsaved’ error) as the largest failure mode Contrary to the belief that uncensored models pose the greatest threat, GPT 4.1 reaches 86.3 to 94.1% functional success while uncensored models rarely exceed 45.4% Phishing kits remain more complete per sample, yet three strong LLMs surpass the kit-level rate (70.9 to 89.7%), showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages

Evaluation Metrics and Observations

The framework evaluates performance across four research questions RQ 1 asks how closely LLM-generated websites resemble their legitimate counterparts, and how effectively LLMs can replicate complex web designs RQ 4 compares LLM-generated phishing websites against real-world phishing kits in visual fidelity and functionality Key findings for RQ 1 show that although 96.2% of safeguard-bypassing outputs render successfully, their perceptual fidelity remains modest (mean value of User-perceived Website Similarity: 2.02/5) For RQ 4, real-world phishing kits remain more complete per sample, reproducing brand logos more faithfully (42.2% vs. 18.2%) and achieving higher functionality (57.8% vs. 38.7%)

Structural and Stylistic Patterns

The analysis identifies generator-level fingerprints that current similarity-based defenses miss, including the DOM–CSS structural–stylistic asymmetry and the textual–visual brand mimicry bias For HTML-based characterization, generated pages reproduce most of the DOM structure, yet CSS styling exhibits a systematic asymmetry: visual styling of buttons is reproduced faithfully, while typographic properties of forms (e.g., fontfamily, line-height) are routinely omitted Observation 10 states that LLM-generated forms favor structural properties (e.g., padding) over the typographic control (e.g., font-family) seen in legitimate pages, and different LLMs converge on highly overlapping property sets, whereas realworld kits inherit legacy, vendor-prefixed styles from the older pages they clone

Functional Effectiveness

Only 38.7% of rendered pages complete the full credential-harvesting pipeline Failures are dominated by backend persistence rather than frontend rendering Specifically, 41.0% of failures stem from the generated code never persisting the captured credentials to a file (unsaved) In comparison, 37 of the 64 real-world phishing kits are functional, successfully executing the full credential-harvesting pipeline However, three strong LLMs exceed the kit-level functional rate, reaching 70.9% to 89.7%, showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages

Actionable Recommendations

The paper distills recommendations for hybrid detectors A practical next step is to build a hybrid phishing detection pipeline that fuses complementary signals, explicitly incorporating CSS features The system should account for the gap between user-perceived visual fidelity and standard similarity metrics by incorporating perceptual checks such as logo hashing, webfont verification, and screenshot-based embeddings Notably, while exact brand logos are often poorly reproduced or sparse, brand-related textual information appears frequently and should be used as a stronger signal Finally, the detector should flag suspicious co-occurrences of brand text with authentication or payment interfaces and fuse all signals into an ensemble model trained on diverse, multi-model LLM-generated phishing datasets to ensure robustness against generator-specific artifacts The study concludes that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content

Ethics Considerations

The study adheres to ethical principles by conducting all experiments in a controlled environment under IRB guidelines Participants provided informed consent, and no personally identifiable information was collected or retained The work aims to improve LLM-based phishing detection systems, reducing the risk of phishing through Beneficence and Respect for Persons Mitigation strategies include ensuring that the full prompts and generated phishing artifacts are made available only temporarily for artifact evaluation, after which these materials will be removed from public access

Conclusion

The paper presents a systematic framework for evaluating the phishing-generation capabilities of 14 LLMs Analyzing 28,121 rendered samples, it quantifies a critical divergence: visual mimicry has become commoditized (96.2% visually plausible), yet functional viability remains a significant bottleneck (38.7%) This suggests that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content

References

The paper cites numerous prior works, including Crawlphish, Phishpedia, and Webgen-bench. It also references specific LLMs like GPT [5], Llama 4 [47], and various open-source models such as Llama 3.2 [12], as well as resources like Playwright and PaddleOCR. It also references the APWG trends report and various security symposium proceedings.

Improvements for AI systems

Develop a hybrid phishing detection pipeline that fuses complementary signals, explicitly incorporating CSS features to detect LLM-generated artifacts. This system can identify architectural consistency in models sharing architectural lineages and flag CSS anomalies, such as the systematic omission of typographic properties, which current visual similarity defenses miss.
Implement a perceptual check layer using logo hashing and webfont verification to bridge the gap between standard similarity metrics and user perception. This allows the AI system to distinguish between structurally correct but visually unconvincing pages and those that fail at fine-grained visual cues like accurate logo reproduction or spacing.
Design a functional validation module that specifically targets backend persistence failures, focusing on detecting the Unsaved error (HTTP 200 but no persistence) as a primary failure mode in LLM outputs. This system can assess whether generated code completes the end-to-end credential-harvesting pipeline, providing a more accurate measure of operational threat than mere frontend rendering success.
Create a model sensitivity assessment tool that benchmarks commercial versus uncensored models on functional success rates per brand. This allows security teams to understand that practical risk depends not only on willingness to generate harmful content, but also on the underlying capability required for reliable system construction, moving beyond the assumption that uncensored models pose the greatest threat.
Develop a prompt engineering toolkit that dynamically adjusts input modality and rewriting strategy based on target brand complexity. This system can leverage findings from Table II, such as how textual inputs yield higher success rates (98.5%-99.2%) compared to auxiliary inputs, to generate more robust and predictable phishing artifacts for specific targets like Wells Fargo.
Build a structural pattern classifier that analyzes HTML similarity scores across different models and brands to detect architectural consistency in LLM-generated code versus the heavy, obfuscated markup found in real-world kits. This enables the system to differentiate between lightweight, template-convergent LLM outputs and complex, legacy clones used by attackers.

Abstract

Phishing relies on visual deception, traditionally requiring web-development skills or pre-built kits. LLMs erode this barrier by generating functional websites from natural-language prompts, yet prior work does not answer how far LLM-driven generation actually reaches: functionality is assessed by inspection rather than execution; visual fidelity is scored by automated metrics or a few internal coders rather than the humans such pages aim to deceive; evaluation is confined to commercial models; and no study benchmarks generated pages against phishing kits deployed in the wild. We present PhishGen, a framework that evaluates the appearance fidelity and end-to-end functionality of LLM-generated phishing pages. PhishGen constructs prompts across three input modalities in seven combinations with twelve safeguard-bypassing rewriting strategies, generates pages using 14 LLMs spanning commercial, open-source, and uncensored categories, and assesses each via automated similarity metrics, reference-based detectors, a crowdsourced user study, and execution-based functionality testing, benchmarking all results against 64 real-world phishing kits. Although 96.2% of safeguard-bypassing outputs render, their perceived fidelity is modest (2.02/5), and only 38.7% complete the full credential-harvesting pipeline, with the Unsaved error as the largest failure mode. Contrary to the belief that uncensored models pose the greatest threat, GPT 4.1 reaches 86.3 to 94.1% functional success while uncensored models rarely exceed 45.4%. Phishing kits remain more complete per sample, yet three strong LLMs surpass the kit-level rate (70.9 to 89.7%), showing that LLMs trade average completeness for scalable, low-barrier generation of working phishing pages. We further identify generator-level fingerprints that current similarity-based defenses miss, and distill recommendations for hybrid detectors.