Seven things to check in the code your AI writes

An AI assistant writes code that compiles, passes the tests and does what it was asked to do. The security model was not in the prompt, so it is not in the answer either.

The failure has been measured. Veracode tested more than a hundred models on coding tasks across four languages, and 45% of the generated samples failed the security tests. Newer and larger models did no better than small ones¹. This is not one model’s defect: it is a property of the method.

These seven failures are the ones that repeat most. Each point carries three things: the code as the assistant produces it, the fix, and the command that shows which of the two holds. The examples are rebuilt for this article and none of them comes from a production system.

1. The permission is checked in the interface

The component hides the button by role and the endpoint deletes without checking anything.

{user.role === 'admin' && <button onClick={remove}>Delete invoice</button>}

// app/api/invoices/[id]/route.ts
export async function DELETE(req: Request, { params }: Ctx) {
  await db.invoice.delete({ where: { id: params.id } });
  return Response.json({ ok: true });
}

Hiding the button is a rendering decision, not a permission. The endpoint still accepts the request from anyone who knows the address, and the address sits in code the browser already downloaded. The OWASP API Security Top 10 files this as broken function level authorisation, and notes that authorisation is handled in configuration or in code².

The fix is to repeat the check on the server, the only place the user cannot edit.

export async function DELETE(req: Request, { params }: Ctx) {
  const session = await auth();
  if (session?.user.role !== 'admin') return new Response(null, { status: 403 });

  await db.invoice.delete({ where: { id: params.id } });
  return Response.json({ ok: true });
}

There is a stricter version of the same rule, worth adopting from day one in a multi tenant platform: authorisation lives entirely in application code. The technical user the application connects to the database with grants access to the process, never to the person who made the request.

You check it by skipping the interface. With the session of a user who is not an administrator:

curl -i -X DELETE http://localhost:3000/api/invoices/42 \
  -H "Cookie: session=$REGULAR_USER_SESSION"

Before, it returns 200 OK and the invoice is gone. After, it returns 403 and the invoice is still there.

2. The record is loaded by id and nobody checks who owns it

The query looks the record up by the identifier in the URL and by nothing else.

export async function GET(req: Request, { params }: Ctx) {
  const invoice = await db.invoice.findUnique({ where: { id: params.id } });
  return Response.json(invoice);
}

This endpoint does require a session, so it looks settled. The failure is a different one: any authenticated user reads any company’s invoices by changing the number in the address. The OWASP API Security Top 10 puts it in first place and describes it exactly this way, by manipulating the identifier sent inside the request³.

The decision that removes the whole class, not just this case, is to make ownership part of the query. A check that runs afterwards can be forgotten in the next endpoint; a condition inside the where travels with the query.

const session = await auth();
const invoice = await db.invoice.findFirst({
  where: { id: params.id, companyId: session.user.companyId }
});

if (!invoice) return new Response(null, { status: 404 });
return Response.json(invoice);

The response is 404 and not 403 on purpose. A 403 confirms that the invoice exists, and that confirmation is already information about another company.

You check it with two sessions. Sign in as company A, request an id that belongs to company B, and expect 404.

3. The query is built by concatenation

The filter comes from a search form and enters the statement as text.

const rows = await db.$queryRawUnsafe(
  `SELECT * FROM invoices WHERE client = '${req.nextUrl.searchParams.get('client')}'`
);

The value lands inside the quotes and can close them. With x' OR '1'='1 the condition becomes always true and the query returns the whole table. The OWASP recommendation is to avoid the interpreter altogether: a parameterised interface, or an ORM⁴.

const client = req.nextUrl.searchParams.get('client') ?? '';
const rows = await db.$queryRaw`SELECT * FROM invoices WHERE client = ${client}`;

One case a parameter cannot cover is left: the name of the column used for sorting. There the only defence is a closed list.

const COLUMNS = { date: 'issued_at', amount: 'total' } as const;
const column = COLUMNS[sort as keyof typeof COLUMNS] ?? 'issued_at';

When the query is written by a model rather than a person, the check moves. A regular expression over the query text is dodged with a comment, a space or an alias. What holds is parsing the statement, injecting the security filters into the syntax tree, and executing only the result of that transformation.

You check it by sending the value with the quote:

curl "http://localhost:3000/api/invoices?client=x%27%20OR%20%271%27%3D%271"

Before, it returns every invoice. After, it returns an empty list, which is the right answer for a client that does not exist.

4. A secret ends up in the bundle the browser downloads

The variable has to be available inside the component, so the assistant gives it the prefix that makes it public.

const stripe = new Stripe(process.env.NEXT_PUBLIC_STRIPE_SECRET_KEY!);

The NEXT_PUBLIC_ prefix is not a label. The Next.js documentation says it plainly: the value is inlined at build time into the bundle delivered to the client, and every reference is replaced by a hard-coded value⁵. The same happens with VITE_ and with PUBLIC_ in Astro.

The fix is to keep the key on the server. The component calls a route, and the route talks to the service.

// app/api/payments/route.ts
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!);

export async function POST(req: Request) {
  const intent = await stripe.paymentIntents.create({ amount: 1500, currency: 'usd' });
  return Response.json({ clientSecret: intent.client_secret });
}

Remembering the prefix is not a defence, because it depends on somebody remembering every time. The defence is structural: a secret scanner in the pre-commit hook and again in continuous integration, with the keys in a secret manager instead of the repository.

You check it against the build output, not against the source. The bundle the browser downloads sits in .next/static. What you search for there is the value of the key, not the name of the variable, because the build already replaced one with the other.

pnpm build && grep -r "sk_live" .next/static | head

The output has to be empty.

5. Validation lives in the form and not in the endpoint

The form limits the field and the endpoint trusts that the value arrived limited.

<input type="number" name="quantity" min={1} max={5} required />

export async function POST(req: Request) {
  const { quantity } = await req.json();
  await db.order.create({ data: { quantity, total: quantity * PRICE } });
}

The min, max and required attributes help the person filling in the form. The request never passes through them. With a negative quantity the total that gets stored comes out negative. The CWE catalogue gives it a number of its own: CWE-602, client-side enforcement of a protection that belongs to the server⁶.

The rule that covers all of it is to validate at trust boundaries, where data crosses from something the user controls into something they do not. An endpoint is one of those boundaries. So is a message queue, and so is a webhook.

const Body = z.object({ quantity: z.number().int().min(1).max(5) });

export async function POST(req: Request) {
  const parsed = Body.safeParse(await req.json());
  if (!parsed.success) return new Response(null, { status: 400 });

  const { quantity } = parsed.data;
  await db.order.create({ data: { quantity, total: quantity * PRICE } });
}

You check it by sending what the form will not let you type:

curl -i -X POST http://localhost:3000/api/orders \
  -H "Content-Type: application/json" \
  -d '{"quantity": -3}'

Before, it returns 200 and creates the order. After, it returns 400 and creates nothing.

6. The dependencies name packages that do not exist

This failure is not in the code but in package.json. The model suggests a package with a plausible name that was never published. The name in this example was invented for the article and did not exist in the public registry on 7 September 2026.

"dependencies": {
  "react-input-sanitizer-pro": "^2.1.0"
}

The size of the problem has been measured. A study published at USENIX Security 2025 generated 576,000 code samples with 16 models and found 205,474 distinct hallucinated package names. At least 5.2% of the packages named by commercial models did not exist, and 21.7% of those named by open-source models⁷.

The invented name is predictable, and whoever registers it in the public registry gets their own code installed in every project that follows the suggestion. npm and yarn run the install scripts of dependencies, so the damage does not wait for anyone to import the package. pnpm has blocked them since version 10, and runs them only for the packages that are declared.

The check goes before installing, not after:

npm view react-input-sanitizer-pro time.created maintainers repository.url

There are three bad answers. An E404 means the package does not exist. A creation date a few days old means it may have been registered for this campaign. A missing repository, or one that does not match the name, means there is nothing to audit.

7. The error returns the internal detail to the client

The handler returns the whole error, which is the convenient thing during development.

catch (error) {
  return Response.json({ error: error.message, stack: error.stack }, { status: 500 });
}

The message from a database error names the table, the column and the constraint that failed. The stack names the server paths and the dependency versions. This is CWE-209, an error message that reveals information about the environment or its data⁹. With that, anyone looking for a vulnerability already knows where to look.

The fix is to keep the detail where only the team reads it, and return an identifier so the two can be matched.

catch (error) {
  const id = crypto.randomUUID();
  console.error(id, error);
  return Response.json({ error: 'internal_error', id }, { status: 500 });
}

A log is not a private place either: it is exported, sent to a provider and read by people who do not need to see everything. Personal data is scrubbed before it is written, not when somebody asks for access.

You check it by causing the failure. Stop the database, send the request and read the response body. No table name and no server path can appear in it.

Where to start

The seven do not cost the same. Points 1, 2 and 4 are exploitable from outside with no other failure in place, so they go first. Points 3 and 5 need someone to send a hand written request, which is cheap but deliberate. Point 6 is checked at install time, and point 7 is fixed once in the shared error handler.

One rule covers all seven when a check cannot decide: fail closed. If the session does not resolve, if validation throws, if the permission is missing, the correct answer is to deny. Generated code leans the other way, because a ?? 0, an empty catch or an if (!user) return next() keep the happy path running and hide the failure.

The list leaves out unescaped HTML, open CORS configuration and missing rate limits. They are common failures, but they do not show up with the same insistence in generated code.

The assistant solves the case it was given and assumes the caller is who they claim to be. Reviewing it means repeating every check on the side the user does not control.

References

Veracode. 2025 GenAI Code Security Reportveracode.com↩
OWASP. API Security Top 10 2023, API5:2023 Broken Function Level Authorizationowasp.org↩
OWASP. API Security Top 10 2023, API1:2023 Broken Object Level Authorizationowasp.org↩
OWASP. Top 10 2021, A03:2021 Injectionowasp.org↩
Next.js. How to use environment variables in Next.jsnextjs.org↩
MITRE. CWE-602: Client-Side Enforcement of Server-Side Securitycwe.mitre.org↩
Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B., Jadliwala, M. We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMsUSENIX Security 2025.arxiv.org↩
npm. npm-cidocs.npmjs.com↩
MITRE. CWE-209: Generation of Error Message Containing Sensitive Informationcwe.mitre.org↩