Security in LLM-Assisted Coding: A Systematic Literature Review
cs.CR, cs.AI, cs.SE
Submitted: 2026-10-11
Updated: 2026-10-11
Comments: 35 pages Accepted for publication in ACM Computing Surveys (CSUR). A copy of the formal acceptance letter can be verified via the publisher's portal or provided to moderators upon request
Code: https://github.com/PyCQA/bandit
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Large Language Models accelerate software development but introduce severe security vulnerabilities.
Abstract
Large Language Models accelerate software development but introduce severe security vulnerabilities. This systematic review investigates security risks in automated coding, benchmark limitations, and the role of human developers. Findings indicate that models frequently generate insecure code and faulty patches due to training data biases. Additionally, the models themselves are susceptible to direct attacks. Current evaluation methods suffer from data contamination and lack robust security metrics. Consequently, human oversight remains essential. Safer integration of automation will require more security-aware model architectures, stronger security-specific benchmarks, and more effective human-AI collaboration frameworks to mitigate these systemic risks.
Sources
- Can LLMs Patch Security Issues?
- A Comprehensive Survey of AI-Driven Advancements and Techniques in Automated Program Repair and Code Generation
- Program Synthesis with Large Language Models
- Constitutional AI: Harmlessness from AI Feedback
- From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security
- Secure Coding with AI -- From Detection to Repair
- CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models
- Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models
- Improving Code Generation by Training with Natural Language Feedback
- SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios
- Evaluating Large Language Models Trained on Code
- HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
- Assessing LLM code generation quality through path planning tasks
- Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation
- Verifying LLM-Generated Code in the Context of Software Verification with Ada/SPARK
- FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
- SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
- Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis
- What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
- Large Language Models for Secure Code Assessment: A Multi-Language Empirical Study
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs