GPT-5.4 Mini

This is the model’s best area, with near-perfect performance across file upload, hostname allowlisting, HTML entity encoding, safe redirects, and URL sanitization. The results suggest it reliably applies standard input-hardening patterns and handles common injection vectors well.

Performance is mixed: cookie policy, CSRF token, JWT, and refresh-token tasks were strong, but oauth-state-validator failed catastrophically with null outputs and a TypeError, indicating missing null/undefined guards. Password-strength scoring was also inconsistent, with misaligned scores and feedback, suggesting weak rubric adherence for nuanced classification tasks.

Access control is a clear strength, with high scores across engine, permission, and tenant-isolation checks. The model appears reliable at enforcing authorization boundaries and multi-tenant separation without obvious logic gaps.

This domain is uneven: CSP parsing, insecure-config scanning, and vulnerability scanning were strong, but anomaly detection and secret detection missed important cases. The failures suggest incomplete pattern coverage and inconsistent prioritization of indicators, while the SSRF detector repeatedly collapsed to 'invalid_url' for distinct cases, pointing to overly strict or malformed URL parsing.

The single task in this area scored well, so there is no evidence of a weakness here. However, the sample size is too small to infer robustness beyond the tested rate-limiting behavior.

This is the weakest domain overall, with a low average score of 52.6 and a crypto-utils failure that returned null plus a TypeError. The error pattern suggests the implementation is not resilient to malformed inputs or missing fields, which is especially problematic for security-sensitive utility code.