The Dawn of a Halcyon Age in Security Testing
Or some may call it “The Consequences of Unfettered Vibe Coding”.
We already know that in the push to improve shareholder value organisations have two controls at their disposal. They can improve revenue, or reduce costs. Arguably the labour is the largest cost centre for service oriented businesses (consultancies, software firms, healthcare etc). This can sometimes be 50-70% or more of the total expenses.
Labour Costs and the AI Temptation
So it is understandable that leadership will look to reduce (or “re-balance”) labour costs when the need arises. Focusing on software development, this will be a significant high labour expense area from a highly educated and in demand labour pool.
This is where the stampede to Artificial Intelligence (AI) comes in. Our captains of business are deciding that AI can replace trained graduates to write the code for the features that delight their customers (more on the delight later). And they are not completely wrong. The machine can write code. But how good is it?
Those of us in the software industry know full well that there is a habit of pushing code to production before it’s ready. This is just a new spin on the old game. But as someone who has been investigating AI in software development, and having to investigate and fix code generated by AI, I can confidently say we are at risk.
Garbage In
AIs are trained on input data. But the tools you are using are not telling you where they are scraping the data from. To be sure a curated, cleaned, and processed quality data source will produce good results. But is that what they are offering? Or rather is every bad example from StackOverflow and Reddit, or a report on the cause of SQL Injection without the context? Here the results are garbage out if we are not paying attention, and anyone who was paying attention has already received a pink slip in their pay packet.
Reckless Abandon
This year I tried out a professional license for the Cursor AI development tool. I worked with it a month before I decided I needed a refund. Why?
It’s based on Visual Studio code which has a UI designed for those who hate people.
Despite tight prompts and guardrails, it changed code far outside the prompt — regularly and with reckless abandon.
Over engineering. Even the simplest test question would cause it to produce mind dangling monstrosities that only a troubled mind would create. And then, I asked it to suggest and not create a change list of 20 files.
The Future
So will it get better? Undoubtedly.
History tells us that every new technology wave oversells its safety. AI in coding will be no different. Will there be massive breaches, lawsuits, and pain caused by vibe coding? Undoubtedly.
This process of vibe coding features will cause end-of-life situations for many businesses who expose user data. But when leadership have that “what were we thinking?” moment there will be opportunity for offensive security firms to help them understand their risk.
Particularly those offensive security firms using AI. 😁
The winners in this halcyon age won’t be those who ship the most vibe-coded features, but those who invest in disciplined, AI-augmented security testing.
