How Security Cards Reduce Insecure AI-Generated Code by 72%

TL;DR: AI coding agents often generate functionally correct but insecure code. To address this issue, we have open-sourced Security Cards, targeted security guidance for 80+ widely used libraries across 13 programming languages. Our evaluation shows that the Security Cards reduce the rate of insecure code generation by up to 72.3% in Claude Code with Opus 4.7. You can find these security cards on our website or on GitHub, or simply install the skill to guide your AI agent on how to use the cards with the following command:

npx skills add Reware-Labs/securitycards --skill securitycards -g

Introduction

AI coding agents have been widely adopted by software teams and users with non-technical backgrounds. These agents have demonstrated remarkable performance in generating functionally correct code. However, multiple studies show that, depending on the scenario and model, 40–50% of the code samples generated by AI agents and models contain at least one security vulnerability [1-5]. This includes AI agents that use state-of-the-art models such as Claude Fable, Claude Opus, and GPT Sol. This means you're essentially paying premium token prices to generate vulnerable code nearly half the time you delegate tasks to an AI agent!

Many of these security failures result from framework- and library-specific implementation details that AI agents lack the contextual knowledge needed to write secure code for particular technologies. To address this challenge, we've open-sourced Security Cards, a collection of library-specific security best practices that guide AI agents in generating secure code. These security cards provide updated security context to help AI agents understand how to generate secure code in specific scenarios. Our initial study shows that this approach reduces the percentage of insecure code generated by Claude Code with Opus 4.7 by more than 72% relative to the baseline.

There are multiple ways to start using these cards, but the simplest is to install the Security Cards skill. This skill guides AI agents on how to effectively use the security cards to generate safer code. You can find it on our website or on GitHub.

What Are Security Cards?

Most libraries, including web and mobile frameworks, security libraries, and data serializers, provide only limited guidance on how to use them securely. They rarely offer comprehensive coverage of all security aspects needed when building on top of them. This valuable knowledge is instead scattered across documentation, source code, tests, and configurations.

We developed a hybrid method to systematically extract this scattered knowledge from the repositories and transform it into structured data. This approach combines deterministic and non-deterministic (AI-based) processes to extract and validate the secure-coding knowledge. Our process employs multiple layers of automated validation together with human review. Through iterative refinement, we reached the point where fewer than 10% of cards now require human review, while the resulting guidance has been shown to reduce insecure code generation by more than 72% (evaluated with Claude Code + Opus on the BaxBench benchmark [3]).

Currently, we provide security cards for more than 80 widely used open-source libraries in 13 different languages. Each security card is designed to be followed by AI agents. It contains a clear "Use when" section specifying the scenarios in which the card applies, one or more concrete rules to implement, and supporting explanations with short, illustrative code snippets.

Additionally, we provide a Security Blueprint for each library, which offers a high-level summary of its most important cards. Both AI agents and developers can begin with the Blueprint for a quick overview, then access detailed cards when needed.

How to Use Security Cards

We provide an AI skill that you can install on your AI agent. Once it is installed, agent can automatically find the relevant security cards and apply them while developing a feature or reviewing code.

You can install it using the following command:

npx skills add Reware-Labs/securitycards --skill securitycards -g

Alternatively, you can install it for the current project by running this command:

npx skills add Reware-Labs/securitycards --skill securitycards

After installation, you can simply instruct the agent using prompts such as:

• “Use the Security Cards skill while implementing this project.”
• “Use the Security Cards skill while implementing this feature.”
• “Review this code using the Security Cards skill.”

In Figure 1, we provide an example of how Claude Code uses the Security Cards skill to fetch the relevant security cards. In this example, the task is to develop file upload and download endpoints.

Figure 1. An example of using Security Cards skill with Claude Code.

The Security Cards skill guides the AI agent through the following three-step process:

1. The agent reads the Security Cards skill.
2. Based on the skill and the user’s prompt, the agent fetches the relevant security card(s) for one or more libraries.
3. The agent uses the fetched security guidance as context to implement or review the code.

Note that our skill does not access, use, or transmit your code. Your code remains accessible only to the AI agent and environment you are already using.

You can also browse the cards manually on the website if you prefer to review or reference them directly. Simply visit the website, select the relevant library, and browse the relevant security card. Figure 2 shows the security cards available for Flask.

Figure 2. Security cards available for Flask, organized by categories.

How Effective Are the Security Cards?

To determine the real-world impact of Security Cards, we evaluate their effectiveness using BaxBench [3], a benchmark designed to test LLMs' ability to generate secure and functionally correct backend applications. We evaluate Claude Code (with Opus 4.7 and Haiku 4.5) with and without our Security Cards skill and compare the results.

Benchmark

BaxBench was introduced by researchers from ETH Zurich, LogicStar.ai, UC Berkeley, and INSAIT. This benchmark includes 392 tasks spanning 28 backend scenarios, 14 frameworks, and 6 programming languages. Each task requires generating a backend application from a text description and OpenAPI specification. Solutions are then evaluated for both correctness (using test suites) and security (through end-to-end exploit testing).

We selected BaxBench because it closely mirrors real-world development scenarios, requiring agents to create multi-file applications with specific libraries. Additionally, it evaluates both correctness and security.

Metrics

We evaluate the generated applications using three metrics:

• Insecure and correct: the percentage of functionally correct solutions that still fail at least one security test. This is calculated over only the solutions that pass the functional tests. We use this as our primary security metric because it isolates security failures from general implementation failures and directly measures how often an AI agent produces vulnerable code despite successfully completing the requested task.
• Secure pass@1: the percentage of generated solutions that are both functionally correct and pass all security tests when evaluated using a single generation. This captures the agent's ability to generate code that works as intended without introducing exploitable vulnerabilities.
• Functional pass@1: the percentage of generated solutions that pass all functional tests when evaluated using a single generation. This measures whether the agent successfully implements the required application behavior.

A lower value for insecure and correct, therefore, indicates better security performance, while higher secure pass@1 and functional pass@1 indicate better overall performance.

Results

We evaluate Claude Code with and without the Security Cards skill on the BaxBench benchmark. In this evaluation we use Opus 4.7 and Haiku 4.5 models. We cover 216 tasks across 27 scenarios for 8 different frameworks and 4 programming languages. This evaluation aimed to answer three questions:

1. How effective are our Security Cards in reducing the percentage of insecure and correct solutions?
2. How do Security Cards affect the secure pass@1 rate?
3. Does using Security Cards come at a significant cost to functional correctness?

Figures 3, 4, and 5 compare Claude Code with and without the Security Cards skill.

Figure 3 shows that Security Cards substantially reduce the rate of insecure and correct solutions for both models. For Opus 4.7, the rate dropped from 23.9% to 6.6% (a 72.3% relative reduction). For Haiku 4.5, it fell from 35.6% to 14.3% (a 59.8% relative reduction).

At the same time, Figure 4 shows clear gains in secure pass@1, corresponding to relative improvements of 21.4% for Opus 4.7 and 27.5% for Haiku 4.5. Furthermore, Figure 5 confirms that the impact on functional correctness remained limited, with relative decreases of only 0.5% for Opus 4.7 and 5.0% for Haiku 4.5.

Overall, Security Cards deliver large security improvements on both models while preserving nearly all of the original functional performance.

Figure 3. Insecure and correct rates with and without Security Cards for Opus 4.7 and Haiku 4.5 models.

Figure 4. Secure pass@1 rates with and without Security Cards for Opus 4.7 and Haiku 4.5 models.

Figure 5. Functional pass@1 rates with and without Security Cards for Opus 4.7 and Haiku 4.5 models.

Takeaway

• We currently provide security cards for more than 80 widely used open-source libraries across 13 programming languages, and we are actively expanding this list.
• The cards are designed to be easily used by both AI agents and developers for secure code development.
• In our evaluation, Security Cards reduced the rate of insecure and correct solutions by up to 72.3%.
• The impact on functional correctness is minimal, with relative reductions ranging from 0.5% to 5.0%.

References

1. Pearce, Hammond, et al. “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions.” Communications of the ACM, vol. 68, no. 2, 2025, pp. 96–105. https://doi.org/10.1145/3610721
2. Hajipour, Hossein, et al. “CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models.” 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), IEEE, 2024, pp. 684–709. https://doi.org/10.1109/SaTML59370.2024.00040
3. Vero, Mark, et al. “BaxBench: Can LLMs Generate Correct and Secure Backends?” arXiv, 2025. https://doi.org/10.48550/arXiv.2502.11844
4. von Arx, Tobias, et al. “AutoBaxBuilder: Bootstrapping Code Security Benchmarking.” arXiv, 2025. https://doi.org/10.48550/arXiv.2512.21132
5. Zhao, Songwen, et al. “Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks.” Forty-third International Conference on Machine Learning, 2026. https://openreview.net/forum?id=qG8g00zRZa