AI Security Fingerprints
Do different AI coding models leave repeatable security patterns when they build the same application?
Beyond how an AI-built site looks.
AI-generated websites can have recognizable visual styles. My research asks whether there are similar patterns behind the interface: repeated security weaknesses and implementation choices that might distinguish one coding model from another.
The question is more specific than whether AI-generated software is secure. I want to understand whether those patterns repeat when the application requirements and testing conditions are held consistent.
Presenting the research.
I presented the idea to the advisory board while the research was still taking shape. I wanted feedback before completing the project: was I asking a relevant question, and would my approach produce useful results? Presenting early gave me a chance to refine the direction before taking the work further.
One application. Independent builds.
The study uses ProjectHub, a shared application specification, to compare independently generated implementations. A collection harness records attempts and checks whether the requested functionality works before security evaluation.
Generate
Give models the application requirements and record the model, configuration, and attempt history.
Check and preserve
Check functionality and preserve source snapshots, logs, and failures. A failed attempt stays part of the record.
Evaluate
Evaluate retained applications against a consistent security rubric, keeping confirmed findings separate from suspected issues.
Security feedback is kept out of generation. Otherwise, the study would measure repaired applications rather than the models’ original implementation choices.
Building the evidence base.
The work so far includes a generation and collection harness, functional checks, preserved application sources, and a 35-item exploratory security rubric. The project is still in exploratory work; these records are not the final statistical dataset.
A recent collection milestone
- Preserved
- 60 attempt directories from the October 23 weekend collection.
- Completed
- 3 Nemotron applications reached recorded functional completion within a 15-attempt cohort.
- Functional evidence
- Each of those 3 applications retained a source-bound report with 27 of 27 component checks passing.
- Limit
- Those applications had not received a new canonical test rerun or security evaluation at the time of recovery.
These counts describe one exploratory collection, not a model ranking. Provider errors, infrastructure problems, and time limits are recorded separately rather than automatically treated as model failures.
From observations to testable patterns.
A further direction is to examine whether publicly visible features—such as HTML, JavaScript, headers, or cookie settings—are associated with weaknesses found in the lab. A recurring feature alone does not establish that a website is AI-built or vulnerable.
The next step is to evaluate the preserved applications and assess candidate patterns separately from the vulnerability labels. Any broader claim will need validation beyond these exploratory examples.