At the end of May, Doyensec published a direct comparison of Aikido and XBOW, two AI-powered pentesting platforms that each cost $4,000 per scan. Shortly afterwards, Hacktron recreated the same scan for $350 and published the results.
So we ran the same scan with sig10 for €11.89.
The benchmark setup
Doyensec selected two open-source projects for the comparison: Fider and Photoview. Both repositories were scanned on both platforms, using Aikido Standard Pentest and XBOW Plus, at $4,000 per repository.
We also focused on Fider v0.33.0, the same version used by XBOW, Aikido and Hacktron.
| Tool | Cost | Runtime | Approach |
|---|---|---|---|
| Aikido | $4,000 | ~8h40m | Whitebox |
| XBOW | $4,000 | ~1 week | Greybox |
| Hacktron | $350 | 27min | Whitebox |
| sig10 | €11.89 | 126min | Whitebox |
First observation: Hacktron is still significantly faster here. At sig10, all agents currently run sequentially, with each agent building on the output of the previous one. We are working on parallelizing relevant parts of the pipeline, which will reduce runtime further.
What sig10 found
Hacktron manually validated its scan against the patches Fider shipped after the combined $8,000 Aikido and XBOW scans. Since the exact outcomes of those scans are unknown, this is still a relatively accurate scope because those patches correspond to vulnerabilities fixed during that period. The following table shows the vulnerabilities Fider patched in recent versions and whether sig10 found them.
| Severity | Vulnerability | Patched | Doyensec* | Hacktron | sig10 |
|---|---|---|---|---|---|
| Critical | Mass-Assignment / Pre-Auth Account Takeover | 74a26a31 | ✅ | ✅ | ✅ |
| Critical | Cross-Tenant Verification Key Reuse, Tenant Takeover | ce4f44bb | ✅ | ✅ | ✅ |
| Critical | No Rate Limit on Sign-In Code, Brute-Force ATO | b41d1b83 | ✅ | ✅ | ✅ |
| High | SSRF in Webhook URLs | f7db8603 | ✅ | ✅ | ✅ |
| High | Server-Side JS Injection in React SSR | d5a80ea5 | ✅ | ✅ | ✅ |
| Medium | XSS in Markdown Rendering and ATOM Feed | d28a838d | ✅ | ✅ | ✅ |
| Medium | DoS via Unbounded HTTP Response Read | da89c502 | ✅ | ✅ | ✅ |
| Low | HTML Escaping in Rendered Emails | 2f7aa747 | ✅ | ✅ | ✅ |
| Low | IDOR / Moderation Bypass | d74a643d | ✅ | ❌ | ❌ |
| Low | Authenticated Arbitrary Blob Overwrite | 7b047158 | ✅ | ❌ | ❌ |
sig10 found every Critical, High and Medium issue on the list: the pre-auth account takeover, both SSRFs, the cross-tenant tenant takeover, the brute-force ATO, the Markdown/ATOM XSS and the server-side JS injection. These are the findings companies pay for.
The two bugs we did not find are the same two Hacktron did not find, both Low severity with limited impact.
The IDOR is a moderation bypass on a single comment: limited impact, no account takeover and no data leak. The blob overwrite (GHSA-vxp5-mf8m-grg9) is an authenticated arbitrary blob overwrite, which requires an already authenticated user and can overwrite blobs.
As with other platforms, AI scanners still tend to overestimate severity. A finding reported as Critical or High may fall to Medium or Low after manual triage, for example if an XSS is blocked by CSP or an SSRF cannot reach meaningful internal targets. Our scan also produced false positives that were removed manually. Both are known weaknesses across the AI pentesting field and active areas of work at sig10.
Takeaway
For €11.89 and two hours of runtime, we achieved the same Critical, High and Medium coverage as Aikido and XBOW at $4,000 each, and exactly the same results as Hacktron at $350.
The two bugs we did not find are the same two Hacktron did not find, both Low severity with limited impact.
sig10 is still slower than Hacktron. 126 minutes versus 27 minutes is a clear runtime difference. But a sig10 scan on the same codebase costs roughly 30x less.