How to Evaluate AI-Generated Application Security
A practical review you can run on an AI-built app before real users and real data arrive.
Your app needs a review, whoever wrote it
If your app was written by an AI, it needs the same security review as code from any other source. AI-generated code security is not a separate discipline: an attacker sees endpoints, forms, keys and data rules, and does not care how the code was produced. What changes is your starting position. An app gets built from your description, and any decision you left out, such as who may read a table or where a key lives, is filled by a default rather than by you.
That is the reason to review. Not because generated code is uniquely bad, but because the decisions you never made explicitly are the ones worth checking now. The OWASP Top 10 is the reference standard for the most critical web application security risks, and it is a sensible list to keep beside you.
A short set of questions covers most of the exposure in a small app: who can reach what, where keys sit, what arrives through inputs, and what your data layer hands back to anyone who asks.
Access control: test as a stranger, then as the wrong user
Sign out and walk through your app as a visitor. Every page that loads, every record that appears and every action that succeeds is something an anonymous person can do. Then create a second account and repeat the pass. You should see your own records and nothing belonging to anyone else.
A practical checklist:
- Signed out: load each page and note anything that shows real records or personal fields.
- As user A: try to open a record owned by user B by editing the URL or the identifier in the request.
- As user A: confirm that editing and deleting someone else's record fails, rather than that the buttons are hidden.
- As admin: confirm that admin views and actions work only with that role.
Hidden buttons are not access control. The check that matters is whether the server refuses the request.
In Draftship, each table has create, read, update and delete permissions, and you can describe the intent in the chat and have the rules written for you. "Users only see their own items" limits everything to the creator. "Anyone can submit, only I can read submissions" opens create to everyone and restricts reading to admins. The defaults are the part people miss: a table without its own rules allows anyone to create and read records, and only the creator to update or delete them, while anonymous records can only be changed by admins. A contact form whose inbox table has no rules is readable by anyone who queries it.
If a record should be public but one field should not, such as the email on a guestbook note, a field-level rule hides that field from everyone who fails it. Hidden fields are removed before filters run, so they cannot be inferred by querying. The data permissions guide shows the wording to use in chat and how to confirm the result in the Data tab.
Secrets: on the server, never in the browser
A key that reaches the browser is public. There is no clever way to hide it in a client component, a bundled config file or a request the user can inspect.
In Draftship, secrets are environment variables for your app's server, used by backend functions. You add them under Dashboard, Secrets with a name in upper snake case, and the app restarts so the value takes effect. Values are encrypted at rest and never shown again after saving, only names are listed. Names beginning with DRAFTSHIP_, NEXT_ or NODE_ are reserved. When you deploy to Vercel from Draftship, secrets are passed to that deployment and are never committed to GitHub.
What to check:
- Every third-party key is stored as a secret and read inside a server function.
- No key appears in a file that runs in the browser.
- Keys you no longer use are removed rather than left dormant.
- Nothing sensitive ends up in a commit, a screenshot or a shared link.
One useful detail: when the AI needs a key it shows a form in the chat, and if you paste a recognised key into a message, Draftship saves it as a secret and replaces it before the AI reads the message. The secrets page lists which services this covers and how to add anything else by hand.
Inputs: forms, uploads and anything shown back to other users
For every input, ask two questions: what happens when someone sends a malformed or hostile value, and where does that value end up afterwards.
- Anything rendered back to another user is a candidate for cross-site scripting. Raw HTML rendering is a scan check in Draftship for exactly that reason.
- Validate on the server. Client-side checks help honest users, not security.
- Uploads need rules on type, size and who may read them. If the file list is publicly readable, the files are effectively public too.
- Text stored in a database and shown in an admin inbox should render as text, not be interpreted as markup.
- Fields that decide access, such as a role or a team, should be set by a backend function rather than by the user's own request. A rule can require a record field to match a field on the user, and people cannot set that field for themselves.
What your data layer hands back
Every table your app exposes answers requests through its rules. Read them as an attacker would: which table can be listed by a signed-out visitor, which can be updated by any signed-in user, and which returns a column you would not publish.
Three things bypass rules entirely: users with the admin role, backend functions using the service client, and you in the dashboard. That is by design, and it is also where mistakes hide. A function that takes an identifier from the client and returns a record without checking the caller is a hole regardless of how tight the table rules are. The scan checks for functions that change data without verifying who is calling.
A table can be safe on paper and still leak. If records are public but a phone or email column is not restricted, every read returns that column. Field-level rules close this: a field that fails its read rule is stripped from every response and ignored by filters, and a field that fails its write rule returns an error. Admins and the service key always pass, so test as a regular user, not as yourself.
Run the scan, then read its limits
Draftship reviews your app every time you open Dashboard, Security and gives it a score out of 100. It looks at data permissions, secrets, code, backend functions and AI agents, and each finding carries a severity, a location and a recommendation.
Fixing is deliberately split. Apply fix rewrites a table's rules and applies them immediately, with no rebuild. Fix with AI sends one precise message to the chat, which suits code findings or changes you want made around new rules. Scan again afterwards to confirm the finding is gone.
What a scan cannot do matters just as much. It is a static review: it catches common, high-impact mistakes, not every possible vulnerability. A clean score is not a guarantee that the app is safe, and treating it as one is a mistake. Keep testing the flows that matter yourself, signed out, as a regular user and as an admin.
When a professional review is worth paying for
A scan plus your own testing covers a lot of small apps. Consider outside help when the stakes change:
- You handle personal data that is regulated where you operate.
- The app touches payments, balances or money movement.
- You deal with health data, children's data or another sensitive category.
- Enterprise customers send security questionnaires or require an assessment.
- You expose a public API that other systems depend on.
When you do hire someone, make the scope concrete. Give test accounts for each role, describe the flows that matter and say what data is stored. Do the scan and your own passes first so paid time goes to business logic and authorisation design rather than to defaults you could have fixed yourself.
Doing this with Draftship
Describe the permissions you want in plain language, store every key as a secret, then work through the scan from the highest severity down. Apply fix where a rule is wrong, use Fix with AI where code needs adjusting, and retest as a stranger and as a second user after each round. Start with Draftship's security scan before you invite real users in.
Primary sources
- OWASP Top 10 | OWASP Foundation · owasp.org
This article was drafted with AI assistance from the primary sources listed below, then reviewed and approved by a Draftship editor.