Rendered at 19:36:21 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Rustam_Khasanov 12 seconds ago [-]
I write code with AI daily and it got much faster. If it brakes - it gets fixed then. We face new vulnerabilities, but we find them faster and have better instruments to investigate and fix. The current period is transitional, but for me it's obvious that it AI is a win for software development
doug_durham 57 minutes ago [-]
This narrative isn't consistent with reality. Cloud computing is much older than 10 years ago. We were doing production grade cloud computing over 16 years ago. The OSS revolution dates to the late 90's. The author seems to think that SWE was in a golden age 10 years ago. Sorry, but it's about the same. We have the same problems in SWE as we had 40 years ago. The scale and ambition of the problems we are solving have increased as compute capabilities have increased.
dreambigwrkhard 16 minutes ago [-]
Yes, there's too much to pick at with this piece.
The PHP2Shell bug existed more than 15 years ago, spanning releases as early as 5.x. There was no AI coding back then. So it's very hard to blame AI. There was a coding mistake, and AI found it in 2026. Probably a good thing before a malicious party does the same.
OSS definitely flourished way before 10-15 years ago. I'm not sure what the author suggests as "knowing the details". Should we all rebuild our own web scripting language to know "all the details": e.g. PHP? Probably not practical. A line has to be drawn on where your knowledge and expertise stop. THe line can be vague and consistently changing based on needs. But there will always be a line. We will always stand on shoulders of of our predecessors one way or another
I understand the author's trying to say the overall situation is unstable and trending negative but I am not sure the writing's logical flow is holding up very well.
A better way to put it is we've written a lot of code, and a lot of it had bugs hiding in plain sight. AI made the whole bill come due at once. And will this AI code explosion create another bill we won't know about until 20 years later? Who knows.
swatcoder 25 minutes ago [-]
The timeline is very confused, but the arc is mostly right.
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
cmiles74 47 minutes ago [-]
AWS is 20 years old, before that people did rent machines from places like Rackspace. Maven Central (Java) is 25 years old, RubyGems is 22 years old, CPAN (Perl) is 31.
I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.
burcsahinoglu 40 minutes ago [-]
I agree with you. I can see how we might be accelerating decoherence with AI, but maybe it won't ultimately matter. I just don't know.
paldepind2 26 minutes ago [-]
I get the impression that when the author says "open source" they mean "rampant use of dependencies". And while the terminology is confusing and the time frame maybe not quite right, there certainly has been a shift where many projects went from depending on none or a few big reputable open source project (MySQL, Apache, Linux, etc.) into depending on hundreds of language packages made by random people.
nosefrog 49 minutes ago [-]
Yep.
nullsanity 50 minutes ago [-]
[dead]
CharlieDigital 57 minutes ago [-]
It doesn't have to be this way.
AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.
The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).
It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.
asdfman123 28 minutes ago [-]
I'm actually quite good at building robust systems, cleaning up messes, documenting things well.
As a result I get:
1) The enmity of my lead dev who prefers to do things more fast and loose.
2) Not promoted, due to the first reason and due to the fact that I don't have "impactful" metrics.
I'm not saying I'm perfect, there are definitely things I could have done better. But the fact still remains that the industry grossly undervalues maintenance and optimization that doesn't lead to easily measurable numbers that impact the bottom line.
That in turn is because it's hard to tell if SWEs are actually paying down critical debt or just wasting time. But that's the core problem, not that we're not good at it.
The good news is I can get AI agents to do this kind of work so I can now chase after metrics.
cess11 12 minutes ago [-]
While you do that, look for opportunities in niches where "fast and loose" is unacceptable, because while they might not market themselves as loudly as most software development organisations they do actually exist.
Typically these places are associated with compliance, audits and regulation with teeth. It might be adjacent to lawyers, in public institutions or weird consultancies. At least that's where I've found such work. Insurance and banking might be decent bets, and they sometimes also come with oddball tech stacks, perhaps mainframes, perhaps Smalltalk.
cmiles74 50 minutes ago [-]
The urge to one-shot everything isn't helping. Getting people to push back on assumptions the model is making, and to have an actual conversation with it about the solution, would lead to more maintainable output.
Maybe part of it is laziness but for sure part of is low morale and hopelessness.
Fr0styMatt88 42 minutes ago [-]
Part of it is the hype cycle too. Everywhere you look online, all the easy examples are people one-shotting this and that. Strangely it seems like the 'standard' way for people on YT to 'test' a model is to just one-shot some random game idea. It's meaningless, but it creates this unconscious expectation (even amongst experienced devs who might not be as experienced in AI use) that the models _should_ be able to derive exactly what's needed from a one-shot.
That's fundamentally impossible though -- if the information isn't there, the model is just going to move toward the average and fill in the blanks.
On another note, I'm also not sure where I stand yet on the thing I see where software devs end up falling down the rabbit hole of building little mini software factories, with agents reviewing agents, coordinating agents and having agents cosplay human roles -- essentially 'orchestration porn'. I'm yet to dive into that rabbit hole (I likely will at some point, so I'm not judging), but from the outside it looks like you end up worsening your understanding of the system, while spending tokens unnecessarily and creating a closed-loop mini model collapse.
I've actually run into the situation at work where I've asked people whether the libraries they've made can do a particular thing and sometimes the answer has been "I don't know" (I'm talking just _features_, not even low-level architecture). At that point, something has gone wrong.
I'm starting to get really interested in "Okay, if human code review is fundamentally impossible given the volume of code, what can we do?". We have a tool that can essentially build tools for almost free, so I'm trying to approach it from that angle. Let's see what sticks.
AlienRobot 34 minutes ago [-]
I think the correct way to solve this problem is completely at odds with AI.
Write less code. Reuse things that have already been written and thus were already reviewed and tested by someone else. Do not reinvent the wheel. KISS.
With AI everyone is implemented a dozen libraries from scratch in their project instead of using something someone else made. Why use a library for 2D geometry when ChatGPT can write one in 10 minutes?
cmiles74 25 minutes ago [-]
I have to think part of this is model training, encouraging the model to code-up something new instead of looking for library code that can fit in the project. One of the things I ask Claude repeatedly is something like "can Spring do this already, or is someone working on this already?"
More tokens, more money.
jltsiren 33 minutes ago [-]
That just follows from the fact that people can learn. Human organizations always have to balance between tacit knowledge and explicit documentation. Document too little, and onboarding new people will be more difficult, and you will lose information when people quit. Document too much, and you are just wasting time without doing anything productive, because people would have figured out that extra information in less time than it took to write it down.
The fundamental problem is that current AI tools don't learn. They make the same mistakes over and over again, unless you give them explicit instructions to avoid those mistakes. Maybe we need to accept that all work is now done by new hires, who have read all the documentation but have no experience in anything. Or maybe AI tools will get better, and all that tedious documentation effort was just a waste of time.
CharlieDigital 26 minutes ago [-]
> The fundamental problem is that current AI tools don't learn
They don't learn, but they are very, very good at searching. That is why writing things down is important, but so is getting that into some sort of agent accessible index.
ajb 23 minutes ago [-]
The problem is not the input (documentation) it's the evaluation of the output. Managing an AI is like managing a team of specialists, whose subjects you are not a master of. Most programmers did not have the problem of managing experts - we were the experts. There are those that do, but several of the levers you have over a team of humans (incentives, responsibility, even hiring and firing) are not available to control an AI, and the new levers ( fine tuning, training, reinforcement learning) are not in the hands of those relying on it. This is not solved by any amount of documentation.
logarhythm 46 minutes ago [-]
I completely agree, and I’m building a tool to solve exactly that challenge.
Pathfinder lets you understand your product’s building blocks. So you and your team pilot your agents at the product level, not the code level.
It does have to be this way, unfortunately, because AI bros will press to have as much software produced at the absolute minimal cost possible.
I've seen it first hand. 70 people fired, 20 people left. The 20 that are left take on all the work that was done before, plus some extra, because every company now has a new AI product, obviously. AI agents make us so productive. Every review is a 30 second LGTM on 500 new lines of code. Confluence filling up with AI slop.
A few weeks back I saw, first hand, a DevOps lead getting ready to ship a big update to performance-critical product where Opus 5.5 suggested they completely turn off JIT everywhere because they were running into a HotSpot C2 optimizer bug that would make the first request after a restart fail with a 500. Actual fix was a one-line adjustment somewhere in a library.
People are scared of being replaced, so they need to appear as productive as possible.
CharlieDigital 30 minutes ago [-]
You can be productive and guide AI to operate in a more compliant way: by writing down basically everything you know, every heuristic you have, documenting the correct patterns, documenting the framework level approach, explicitly spelling out what's good code, writing down detailed specs for an agent to work on, writing the test plan, etc.
A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
A concrete example: I wrote a small library[0] (MIT, OSS) that improves how agents write unit and integration tests. This isn't product code, but this makes the agent better at writing the tests it uses to verify its work. (Tests become denser, more semantic, less context, type-checked setup, etc.). You can't expect an agent to do this by itself; the human needs to produce the scaffolding.
I suspect the majority of reviews had been a rubber stamped LGTM. Often the minority with substantive feedback were heeded and changes made; I'm not sure that is happening anymore.
asdfman123 25 minutes ago [-]
> AI bros
You have to stop thinking this way. It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
There are legitimate criticisms about AI use and how AI has changed the profession, but you have to learn which battles are worth fighting.
dotdi 26 seconds ago [-]
Just like crypto bros a few years back, where we "do what our management wants us to do", built stupid-ass blockchain into every every which app that didn't need it, and when it doesn't sell get blamed for it.
Personally, I was asking "but what's the use case" in every meeting. "Why would somebody pay for this". "Who's the customer". Right now the vibes are quite similar to back then.
oulipo 49 minutes ago [-]
"AI is great at following patterns", sure but if some attackers can "inject attack patterns" at a high rate in your training set (check "dataset poisoning") then it's going to be an issue...
Forgeties79 51 minutes ago [-]
SOP and documentation is everything. You don’t want it going out of bounds and coming up with its own solutions, which are often just inefficient facsimiles of random stuff it finds that seems to makes sense. You also run the risk of not understanding the solution it puts in front of you. With an ironclad SOP and quality documentation, you know when it is straying from the best paths, because they are laid out for both of you.
Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.
I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story
cmiles74 45 minutes ago [-]
IMHO, most places are pretty terrible and documentation. I work with people today that seem to think someone else is going to do it for them.
rglover 3 minutes ago [-]
Imo: the way we make software is breaking down at a systemic level.
The harsh reality is that some people/systems are adapting just fine to the new way of doing things, others are making a mess of things by rushing via vibe coding and not checking the work (either by choice or by circumstance at work). The results won't be (and shouldn't be expected to be) linear.
The 2010s brought on a very groupthinky dynamic to software dev. This led to a culture where the majority of developers just ask "what is everyone else doing." Now that we're past the blog era of development, that shared reality appears to be breaking down. Now the answer really is "whatever makes sense for/to you," but I think (anecdotally) that's freaking a lot of people and teams out.
For the first time in a long time, a "best practice" doesn't really exist.
LPisGood 1 hours ago [-]
Software has always been broken at a systemic level. It’s a complicated continuously changing web of abstractions and abstraction breaking hacks attempting to survive prolonged contact with reality.
kerblang 53 minutes ago [-]
It was less broken before open source allowed an architectural explosion, because the dependencies were too expensive. It took a lot of arguing to get management to splurge on a single library or other tool. A lot of astronaut-architect wannabes were constantly grumpy about mgmt that wouldn't pay for the latest whiz-bang thingie.
The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.
Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.
cmiles74 41 minutes ago [-]
Is the problem open source? I haven't see these kinds of widespread problems coming from Java dependencies (25 years old) or Perl (maybe 30 years old)? Okay, maybe people aren't using Perl like they used to but Java is still a thing.
Part of it is the inclination of the developer to make the thing more robust. Part of it is lowering the barrier of entry, maybe to the point it's _too_ easy to submit a package. Part of it is bad decisions made by the package manager, like running arbitrary scripts.
psychoslave 54 minutes ago [-]
Well, yes, but the proportion of systemic level broken stuffs might have change a bit. Just like, ok, there was always rule34 lurking to make dirty everything innocent that remained of the untainted child memory one had.
Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.
SV_BubbleTime 1 hours ago [-]
I do embedded. I’ve always marveled at how ludicrous that “real software” is. I have truths. I have registers that will never be or act differently, I can make pretty damn good assumptions that no one will put this code somewhere else’s etc.
In the past few years though, it’s been creeping into my domain too.
bumby 60 minutes ago [-]
>I have registers that will never be or act differently
Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.
LPisGood 54 minutes ago [-]
At high altitude or high deployment numbers. Hyper scaler data centers measure these “one in a million” anomalies in thousands of times per houe.
leftnode 53 minutes ago [-]
Yeah, don't we have error correcting RAM for this reason, here on Earth?
Recurecur 6 minutes ago [-]
Sure, but RAM isn’t the only place a bit can flip. Registers are a thing. IBM had error correction in Power architecture CPU registers, not sure where AMD, Intel, or Apple (lol) are on that issue. IIRC register bit flips first came up with Sun SPARC hardware, which Sun mitigated, but of course SPARC isn’t used too much these days…
A quick AI-assisted search informs me that neither AMD nor Intel document any on-CPU error correction, though perhaps there is some. Apple currently has zero systems that even use ECC, which seems insane to me given they’re shipping systems with up to 512 GB of RAM.
This seems to be an issue that’s falling through the cracks again. I guess nothing will be done until there’s a critical failure due to a bit flip, and verifying that was the cause for a given problem is difficult.
On ECC, Apple needs to Think Different (tm). Maybe it should introduce actual “Pro” machines…
marcosdumay 51 minutes ago [-]
Besides, chip bugs are famous since the Pentium, and unreliable circuits are the normal way to describe circuits, you have to add notice when you get a reliable one.
skydhash 47 minutes ago [-]
That's why we do requirement gathering and analysis. Engineering is always about tradeoffs and things that matters in certain context don't in others. The issue is that in a lot of orgs, while there's a record of decisions, we don't usually track assumptions and the overall system is full of contradictions. Also there's no overall keeper of knowledge (either one person or a team) that periodically ensure that contradictions are kept at minimum.
add-sub-mul-div 54 minutes ago [-]
I don't get this thing where people are allergic to the idea that change can happen.
No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.
sega_sai 8 minutes ago [-]
I think the article has a point that the if you have a very complicated legacy system, then if you just let an LLM run on this, there is a significant chance that some parts of the system will be become inconsistent with each other, and the whole thing will be more of a mess and more likely to break.
In my view, however, what is more likely to happen soon is usage of LLM to transform various parts of code into the form with much more clearly defined interfaces/rules/boundaries that can be essentially proven to work correctly. That in turn (in my view) will enable much more robust systems.
dividedcomet 1 hours ago [-]
This feels a little hyperbolic, the grounding was a result of a cable swap and the ssh cert incident they cite was so human caused it’s not even funny. I think things are a little more shaky, but not massively so. I had the town I was living in during college lose its internet and phone lines due to a cable being cut on a mountain pass and it was fine. If your business is sensitive to an email outage, prioritize high reliability over integration advantages that outlook offers. The php2shell vuln is bad, but we also had log4j years ago.
sharkbot 46 minutes ago [-]
I’d like to analogize the situation with the Kessler syndrome: the increasing density of satellites in low Earth orbit increases the probability that a cascading failure occurs, which would deny us access to LEO for centuries. The inciting incident could be intentional or could be accidental, but the result is the same.
The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.
Intentionality isn’t necessary: this is a systemic problem.
ljm 51 minutes ago [-]
Software wasn't exactly hard 12 years ago either, the barriers to entry were being lowered at a rapid pace. 2014 was the dawn of the software bootcamp where JS became more and more pronounced in web architecture.
I feel this piece is either more alarmist than it deserves to be, or it is late to the party.
It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.
My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible. A similar incident in the 90s would involve a product recall or a way to get updated media from a retailer with the fix. Very expensive.
Standards have dropped and continue to do so, because we can also fix fast. It's not suddenly breaking down. If anything, it's just too centralised.
kaboomshebang 14 minutes ago [-]
Thanks to the author. It's a good read and interesting HN comments. Nothing wrong with having concerns, and I'm sure many people share those concerns. It's scary sometimes. But, the view of the author is somewhat on the negative side. I feel that AI development is still in early phases and people or companies are discovering what works and what doesn't. AI empowers us in new ways with new problems, and new trade-offs. The old way of doing things had different hurdles. Interesting times. It's not going anyway any time soon I expect ;)
anonreplier 1 hours ago [-]
There are some decent points, but this is collating a lot of separate incidents with wildly different causes that just happened to occur in a close timeframe.
xg15 59 minutes ago [-]
Let's see how many more incidents will just happen to occur in proximity to those events.
llm_nerd 1 hours ago [-]
I don't even know if there are decent points. This flails around in a way that's just...not super useful to spend the time trying to digest.
Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.
These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.
We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.
None of this is new.
drooby 1 hours ago [-]
Most folks here didn't even know, but a Bitrise security outage hobbled our entire CI this weekend. Friday to Sunday night.
The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..
spiralpolitik 60 minutes ago [-]
There have been parts of the software industry that have been mission critical from day one and have built processes, languages, and tools to make sure things are 100% reliable. Building software that way is slower and more expensive and in the past has been an overkill for most systems.
The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.
Hence why we are at the breaking point.
bob1029 51 minutes ago [-]
It feels like we are currently experiencing a soft version of the sewing machine riot.
Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.
skeledrew 51 minutes ago [-]
Things are breaking down because they were weak to begin with, and demolition is needed in order to rebuild better, stronger. This is just a period of trial by fire: iron has to be heated and beaten numerous times (with carbon) before it becomes steel. All that's happening is AI is accelerating the evolution of industries into something which all eventually have the strength of steel.
jstimpfle 1 hours ago [-]
My impression is it's always been a mixed bag. The level of bugginess has been relatively constant. In the meantime, scope and capabilities have gone way up. I hate Teams and Outlook and the virus scanner on my company's laptop though. Those are keeping me from doing my work. Windows + anti-virus makes file access and process spawning 10x to 100x slower compared to a Linux machine.
keithnz 57 minutes ago [-]
To talk abstractly about all software, you'd also need to analyze all the software that stays working. This article is just cherry picking a few recent incidents and try's to make some conclusions without really looking at causes and whether that can be generalized to "Software".
agosz 39 minutes ago [-]
The author seems to have glossed over the viruses of the early 2000s,or the Y2K panic, and other incidents from before. Building software has always been hard and it's always been abstractions on more abstractions.
The one compelling part is that now we might be forced to pay for some of the rickety towers that we've erected because AI has made exploits pretty cheap to try.
uludag 8 minutes ago [-]
I think the author is pointing to something fundamentally different than software being hard.
Like before, individuals in an organization had various interests, motivations, connections, knowledge, context, that allowed our organizational brain to function properly. Now, everyone is (or will be in the future) a meat proxy, isolated, and disengaged which produces an organization with neurons but no neural connections: apparently healthy but, as the author put it, decoherent.
tflinton 52 minutes ago [-]
I’ve had AI running to auto upgrade our infrastructure and it’s successful twice as much as humans. Failure rate of a human managed upgrade is 1/10, failure rate of an AI agent is 1/20 in my small set data.
Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.
I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.
herval 1 hours ago [-]
This reads like nostalgia way more than reality
wvenable 57 minutes ago [-]
I agree. In my life time there hasn't been a minute of software reliability. There's nothing special about this year or even this month.
I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.
If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.
SoftTalker 1 hours ago [-]
> You can find programmers who know how to manage the complexity, but they want to be paid lots of money.
This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.
travisgriggs 60 minutes ago [-]
> About ten years ago, two small revolutions happened in our industry ... And then, we had the Open Source boom.
This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.
That said...
> But AI is both a cause and a symptom
This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.
tcoff91 1 hours ago [-]
There were package managers long before npm. Maven, rubygems, etc...
noosphr 1 hours ago [-]
Apt and yum.
It was a very long time ago indeed that you needed to manually edit popular programs to build in your environment.
asdfman123 20 minutes ago [-]
Software isn't breaking down, people just haven't figured out how to use AI yet.
It reminds me when I first started work on a codebase with proper unit tests: I thought I could ship whatever and the tests would catch it. That was incorrect; while unit tests are a guardrail, I still had to properly test my code.
Similarly, people need to learn how to use AI and make sure they don't introduce bugs or slop into the codebase.
But this brings up a larger point for me: most people are grieving the loss of what their jobs used to be, and denial is a core phase of grief.
I went through it too. I tried to find ways to say AI was silly or wouldn't work. But now it's inescapably here and we need to work in the new paradigm. There are plenty of valid criticisms, but it's already very good, it's getting better, and you have to learn how to use it as best you can.
rurban 1 hours ago [-]
He obviously is preparing another doomsday conference talk. Nothing interesting to see here.
On the other hand software is improving dramatically at a systemic level, even without safe system languages arriving on the horizon.
tippytippytango 49 minutes ago [-]
Is defect rate increasing or is the amount of, and use of, software increasing? What is the defect rate relative to that?
cadamsdotcom 25 minutes ago [-]
All software has bugs, and some software is critical
Which means all critical software has bugs!
There's also more of it! Even more once you finish reading this sentence. If security incidents scale sublinearly vs the growth of critical software, we should be proud.
Crappy work is not universal either. There are engineers out there, guiding agents diligently, doing a good job. Never letting an agent go wild unattended, writing the same code they'd have written themselves, but with more tests and more better assurances and more docs and better infra and and and.
The problem that leads to blog posts like this is that good, diligent work, like critical software that does its job, are invisible.
Because no one notices a building that's not on fire. They just walk right by.
gkcnlr 38 minutes ago [-]
> Most SWEs now are unable to tackle engineering problems without some form of AI assistance. That’s not because they’re incompetent - they just can’t keep on top of the complexity. Even if they could once, they now face a tsunami of AI generated code.
That's called outsourcing, which inevitably enables "vibe" effect of the coding: Your AI goes out of its constraints and does totally irrelevant shit you didn't ask for or does something barely passable you could approve, without knowing further consequences because you outsourced your thinking and control over the project.
To try to find a remedy for this shortcoming, they invented "skills", which totally negates the purpose of an AI (LLMs, genAI, whatever you call it to describe the current tech) IMHO.
It's also funny reading these comments. Most say "well, it was already broken" and they continue trusting AI more. Well, AI is not exempt from this "brokenness".
_pdp_ 49 minutes ago [-]
On the flip side, once you hit rock bottom the only way is up.
titzer 55 minutes ago [-]
Not everyone is using AI to make shittier software. Most big important software projects I know of are frantically fixing an avalanche of AI-found bugs. Important software is getting more robust and resistant to attacks, while still adding functionality.
There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.
Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.
Things can have rocket boosters, you just gotta know where to place them.
acedTrex 20 minutes ago [-]
Who could have forseen this except literally anyone with a braincell
tracerbulletx 1 hours ago [-]
When I first started my career we used to ship like once a month and that was considered good, everything was extremely convoluted and worse than it is today, whole segments of the industry didn't even use Version Control, half the government websites you used stored passwords in the clear and emailed them to you for resets, e-commerce was a cgi script that sent an email for fulfilment and it took weeks to get your shipment, sites went down at night and weekends for maintenance regularly, servers fell over with traffic that would run on a raspberry pi now, but for some reason everyone keeps saying how bad things are now. They're delusional or don't actually remember the past.
AndrewKemendo 1 hours ago [-]
Software was mostly always broken but it was hidden and invisible, now it’s visible because the cost of making it visible is so low
It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated
jmclnx 1 hours ago [-]
Well there was this saying "Do One Thing and Do It Well", seems once again we are seeing the affects of ignoring that.
chalmovsky 50 minutes ago [-]
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
LMAO. They can't lose something they never had in the first place.
This article has other pearls like this:
> Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.
Like, with all due respect, what the fuck do they know?
Also this:
>It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.
Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D
This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.
adamddev1 1 hours ago [-]
People keep believing that the big LLM players with somehow disprove Rice's Theorem. "Somehow we'll be able to get AI to check if a program does X or Y." "We'll be able to have the machines verify things automatically." But this is is of course as hopeless as solving the Halting Problem. Theoretically, we know this is impossible.
Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.
People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.
marcosdumay 24 minutes ago [-]
As the sibling says, the problem is not really Rice's Theorem. We were doing some great progress into software reliability before the LLM-crazy destroyed the industry.
The problem for LLMs is less formal and way more wicked. We don't have any good theory of software reliability. At all. All the reliability we have is artisanal and designed ad-hock.
habitue 1 hours ago [-]
Rice's theorem is across arbitrary turing machines. We intentionally limit the kinds of programs we emit with type systems, knowing they will prevent us from writing some programs, but we'll be able to analyze the ones we can write much better.
We prove properties of programs all the time, that's what a type system is for.
Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.
adamddev1 58 minutes ago [-]
Are vibe-coded apps being being written in Rocq programs that perfectly match dependent types to every possible behavior? Is anyone anywhere writing software with type theory at the level where it perfectly limits things and saves our programs from semantic mistakes?
vouwfietsman 42 minutes ago [-]
> every possible behavior
> semantic mistakes
Of course the point is not whether we can now do this with AI, but whether AI can help us get the same quality we used to get, faster.
Also, consider that it should rarely be the goal to enumerate and judge every possible behavior of a program, with exceptions for life-saving or otherwise critical software.
d0mine 38 minutes ago [-]
Many of the recent maths solutions are just a lot of Lean code (types + guarantees).
ginthian 47 minutes ago [-]
[flagged]
0xbadcafebee 34 minutes ago [-]
There is one root cause for all major software system issues: business doesn't give a shit.
There isn't a software system in the world for which we can't predict the failure modes and make solutions for. You simply ask two questions: 1) "How can I tell for sure this thing is working?", 2) "If it doesn't work, how do I fix it so it does work?". You might not know the answers at first, and that's fine. The business's job is to ask the questions, do all the work to figure out the answers, and then test & implement the answers. If the business does this, they prevent most of the failures, and have a quick deterministic solution for when eventual failure happens. If the business doesn't do these things, they are choosing to let it fail in an unpredictable and likely very expensive way (they don't give a shit).
This process has a bunch of names. Failure Mode and Effects Analysis, Business Continuity Planning, Risk Management, etc. No matter how you slice it, it's the same basic work: find out what could go wrong (at a high level) and have a plan to fix it when it does. It's not always easy, but you can always figure out some kind of plan that works.
For all the security and reliability issues we've seen lately, there are known methods to either prevent those problems or fix them quickly. The incidents aren't happening because something new is going on. They're happening because businesses stopped giving a shit a long time ago, and the chickens are coming home to roost.
TacticalCoder 58 minutes ago [-]
It's because amateurism has been hailed as the second coming of Christ.
Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.
"People can't write a proper leftpad".
Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.
Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).
We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.
And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.
They're just mediocre.
That's what happened: the veneration of mediocrity.
Apocryphon 38 minutes ago [-]
You're repeating points that were brought up in the discussions about leftpad a decade ago, but the root cause wasn't venerating amateurism (even though the everyone can code / software bootcamp trends of the time was a by-product of it); it was the veneration of velocity. Which, I suspect, is inevitable in software because it's so powerful, low-resource intensive, and lucrative compared to other engineering disciplines. This technology was always going to create an industry focused on speed above everything else.
vouwfietsman 32 minutes ago [-]
I think the exact opposite has happened: needless gatekeeping and NIH-syndrome has created this idea that software development is a skill that measures intelligence.
Because of this, battle-scars from the rough edges of software are seen as part of software-competence, not as a failure of industry. Novices get met with 'you're holding it wrong', but at the same time veterans lose the plot and spend most their time doing technical onanism, or worse, fighting meaningless complexity.
Until the stack exceeds any individuals understanding, and the cracks start to show. Leaky abstraction upon leaky abstraction upon layers and layers of ducttape, under the guise of HA, distributed system mantras, robustness, you name it.
Every part needing babysitting because every part is half the solution to half of your problem, but admitting that is seen as a skill issue.
Yes my DB dies because it caches stuff and (under obscure circumstances) never frees it: 'thats just MySql, deal with it', yes of course my frontend framework needs to be kept up to date every week because everyone is evalling strings all the time, 'don't you sanitize your input?', multiplied by every layer of the stack.
Don't you let your static analysis automatically merge security PRs so you can stay on top of this giant heap of string evals? No? That's best practice man!
The PHP2Shell bug existed more than 15 years ago, spanning releases as early as 5.x. There was no AI coding back then. So it's very hard to blame AI. There was a coding mistake, and AI found it in 2026. Probably a good thing before a malicious party does the same.
OSS definitely flourished way before 10-15 years ago. I'm not sure what the author suggests as "knowing the details". Should we all rebuild our own web scripting language to know "all the details": e.g. PHP? Probably not practical. A line has to be drawn on where your knowledge and expertise stop. THe line can be vague and consistently changing based on needs. But there will always be a line. We will always stand on shoulders of of our predecessors one way or another
I understand the author's trying to say the overall situation is unstable and trending negative but I am not sure the writing's logical flow is holding up very well.
A better way to put it is we've written a lot of code, and a lot of it had bugs hiding in plain sight. AI made the whole bill come due at once. And will this AI code explosion create another bill we won't know about until 20 years later? Who knows.
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.
AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.
The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).
It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.
As a result I get:
1) The enmity of my lead dev who prefers to do things more fast and loose.
2) Not promoted, due to the first reason and due to the fact that I don't have "impactful" metrics.
I'm not saying I'm perfect, there are definitely things I could have done better. But the fact still remains that the industry grossly undervalues maintenance and optimization that doesn't lead to easily measurable numbers that impact the bottom line.
That in turn is because it's hard to tell if SWEs are actually paying down critical debt or just wasting time. But that's the core problem, not that we're not good at it.
The good news is I can get AI agents to do this kind of work so I can now chase after metrics.
Typically these places are associated with compliance, audits and regulation with teeth. It might be adjacent to lawyers, in public institutions or weird consultancies. At least that's where I've found such work. Insurance and banking might be decent bets, and they sometimes also come with oddball tech stacks, perhaps mainframes, perhaps Smalltalk.
Maybe part of it is laziness but for sure part of is low morale and hopelessness.
That's fundamentally impossible though -- if the information isn't there, the model is just going to move toward the average and fill in the blanks.
On another note, I'm also not sure where I stand yet on the thing I see where software devs end up falling down the rabbit hole of building little mini software factories, with agents reviewing agents, coordinating agents and having agents cosplay human roles -- essentially 'orchestration porn'. I'm yet to dive into that rabbit hole (I likely will at some point, so I'm not judging), but from the outside it looks like you end up worsening your understanding of the system, while spending tokens unnecessarily and creating a closed-loop mini model collapse.
I've actually run into the situation at work where I've asked people whether the libraries they've made can do a particular thing and sometimes the answer has been "I don't know" (I'm talking just _features_, not even low-level architecture). At that point, something has gone wrong.
I'm starting to get really interested in "Okay, if human code review is fundamentally impossible given the volume of code, what can we do?". We have a tool that can essentially build tools for almost free, so I'm trying to approach it from that angle. Let's see what sticks.
Write less code. Reuse things that have already been written and thus were already reviewed and tested by someone else. Do not reinvent the wheel. KISS.
With AI everyone is implemented a dozen libraries from scratch in their project instead of using something someone else made. Why use a library for 2D geometry when ChatGPT can write one in 10 minutes?
More tokens, more money.
The fundamental problem is that current AI tools don't learn. They make the same mistakes over and over again, unless you give them explicit instructions to avoid those mistakes. Maybe we need to accept that all work is now done by new hires, who have read all the documentation but have no experience in anything. Or maybe AI tools will get better, and all that tedious documentation effort was just a waste of time.
Pathfinder lets you understand your product’s building blocks. So you and your team pilot your agents at the product level, not the code level.
It’s augmented product engineering: https://pathfinder.com.ai
I've seen it first hand. 70 people fired, 20 people left. The 20 that are left take on all the work that was done before, plus some extra, because every company now has a new AI product, obviously. AI agents make us so productive. Every review is a 30 second LGTM on 500 new lines of code. Confluence filling up with AI slop.
A few weeks back I saw, first hand, a DevOps lead getting ready to ship a big update to performance-critical product where Opus 5.5 suggested they completely turn off JIT everywhere because they were running into a HotSpot C2 optimizer bug that would make the first request after a restart fail with a 500. Actual fix was a one-line adjustment somewhere in a library.
People are scared of being replaced, so they need to appear as productive as possible.
A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
A concrete example: I wrote a small library[0] (MIT, OSS) that improves how agents write unit and integration tests. This isn't product code, but this makes the agent better at writing the tests it uses to verify its work. (Tests become denser, more semantic, less context, type-checked setup, etc.). You can't expect an agent to do this by itself; the human needs to produce the scaffolding.
[0] https://github.com/zeeq-ai/zeeq-biome
You have to stop thinking this way. It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
There are legitimate criticisms about AI use and how AI has changed the profession, but you have to learn which battles are worth fighting.
Personally, I was asking "but what's the use case" in every meeting. "Why would somebody pay for this". "Who's the customer". Right now the vibes are quite similar to back then.
Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.
I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story
The harsh reality is that some people/systems are adapting just fine to the new way of doing things, others are making a mess of things by rushing via vibe coding and not checking the work (either by choice or by circumstance at work). The results won't be (and shouldn't be expected to be) linear.
The 2010s brought on a very groupthinky dynamic to software dev. This led to a culture where the majority of developers just ask "what is everyone else doing." Now that we're past the blog era of development, that shared reality appears to be breaking down. Now the answer really is "whatever makes sense for/to you," but I think (anecdotally) that's freaking a lot of people and teams out.
For the first time in a long time, a "best practice" doesn't really exist.
The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.
Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.
Part of it is the inclination of the developer to make the thing more robust. Part of it is lowering the barrier of entry, maybe to the point it's _too_ easy to submit a package. Part of it is bad decisions made by the package manager, like running arbitrary scripts.
Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.
In the past few years though, it’s been creeping into my domain too.
Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.
A quick AI-assisted search informs me that neither AMD nor Intel document any on-CPU error correction, though perhaps there is some. Apple currently has zero systems that even use ECC, which seems insane to me given they’re shipping systems with up to 512 GB of RAM.
This seems to be an issue that’s falling through the cracks again. I guess nothing will be done until there’s a critical failure due to a bit flip, and verifying that was the cause for a given problem is difficult.
On ECC, Apple needs to Think Different (tm). Maybe it should introduce actual “Pro” machines…
No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.
In my view, however, what is more likely to happen soon is usage of LLM to transform various parts of code into the form with much more clearly defined interfaces/rules/boundaries that can be essentially proven to work correctly. That in turn (in my view) will enable much more robust systems.
The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.
Intentionality isn’t necessary: this is a systemic problem.
I feel this piece is either more alarmist than it deserves to be, or it is late to the party.
It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.
My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible. A similar incident in the 90s would involve a product recall or a way to get updated media from a retailer with the fix. Very expensive.
Standards have dropped and continue to do so, because we can also fix fast. It's not suddenly breaking down. If anything, it's just too centralised.
Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.
These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.
We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.
None of this is new.
The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..
The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.
Hence why we are at the breaking point.
Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.
The one compelling part is that now we might be forced to pay for some of the rickety towers that we've erected because AI has made exploits pretty cheap to try.
Like before, individuals in an organization had various interests, motivations, connections, knowledge, context, that allowed our organizational brain to function properly. Now, everyone is (or will be in the future) a meat proxy, isolated, and disengaged which produces an organization with neurons but no neural connections: apparently healthy but, as the author put it, decoherent.
Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.
I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.
I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.
If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.
This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.
This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.
That said...
> But AI is both a cause and a symptom
This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.
It was a very long time ago indeed that you needed to manually edit popular programs to build in your environment.
It reminds me when I first started work on a codebase with proper unit tests: I thought I could ship whatever and the tests would catch it. That was incorrect; while unit tests are a guardrail, I still had to properly test my code.
Similarly, people need to learn how to use AI and make sure they don't introduce bugs or slop into the codebase.
But this brings up a larger point for me: most people are grieving the loss of what their jobs used to be, and denial is a core phase of grief.
I went through it too. I tried to find ways to say AI was silly or wouldn't work. But now it's inescapably here and we need to work in the new paradigm. There are plenty of valid criticisms, but it's already very good, it's getting better, and you have to learn how to use it as best you can.
On the other hand software is improving dramatically at a systemic level, even without safe system languages arriving on the horizon.
Which means all critical software has bugs!
There's also more of it! Even more once you finish reading this sentence. If security incidents scale sublinearly vs the growth of critical software, we should be proud.
Crappy work is not universal either. There are engineers out there, guiding agents diligently, doing a good job. Never letting an agent go wild unattended, writing the same code they'd have written themselves, but with more tests and more better assurances and more docs and better infra and and and.
The problem that leads to blog posts like this is that good, diligent work, like critical software that does its job, are invisible.
Because no one notices a building that's not on fire. They just walk right by.
That's called outsourcing, which inevitably enables "vibe" effect of the coding: Your AI goes out of its constraints and does totally irrelevant shit you didn't ask for or does something barely passable you could approve, without knowing further consequences because you outsourced your thinking and control over the project.
To try to find a remedy for this shortcoming, they invented "skills", which totally negates the purpose of an AI (LLMs, genAI, whatever you call it to describe the current tech) IMHO.
It's also funny reading these comments. Most say "well, it was already broken" and they continue trusting AI more. Well, AI is not exempt from this "brokenness".
There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.
Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.
Things can have rocket boosters, you just gotta know where to place them.
This is classic surveillance/detection bias: https://healthjournalism.org/glossary-terms/surveillance-bia...
It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated
LMAO. They can't lose something they never had in the first place.
This article has other pearls like this:
> Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.
Like, with all due respect, what the fuck do they know?
Also this:
>It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.
Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D
This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.
Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.
People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.
The problem for LLMs is less formal and way more wicked. We don't have any good theory of software reliability. At all. All the reliability we have is artisanal and designed ad-hock.
We prove properties of programs all the time, that's what a type system is for.
Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.
Of course the point is not whether we can now do this with AI, but whether AI can help us get the same quality we used to get, faster.
Also, consider that it should rarely be the goal to enumerate and judge every possible behavior of a program, with exceptions for life-saving or otherwise critical software.
There isn't a software system in the world for which we can't predict the failure modes and make solutions for. You simply ask two questions: 1) "How can I tell for sure this thing is working?", 2) "If it doesn't work, how do I fix it so it does work?". You might not know the answers at first, and that's fine. The business's job is to ask the questions, do all the work to figure out the answers, and then test & implement the answers. If the business does this, they prevent most of the failures, and have a quick deterministic solution for when eventual failure happens. If the business doesn't do these things, they are choosing to let it fail in an unpredictable and likely very expensive way (they don't give a shit).
This process has a bunch of names. Failure Mode and Effects Analysis, Business Continuity Planning, Risk Management, etc. No matter how you slice it, it's the same basic work: find out what could go wrong (at a high level) and have a plan to fix it when it does. It's not always easy, but you can always figure out some kind of plan that works.
For all the security and reliability issues we've seen lately, there are known methods to either prevent those problems or fix them quickly. The incidents aren't happening because something new is going on. They're happening because businesses stopped giving a shit a long time ago, and the chickens are coming home to roost.
Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.
"People can't write a proper leftpad".
Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.
Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).
We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.
And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.
They're just mediocre.
That's what happened: the veneration of mediocrity.
Because of this, battle-scars from the rough edges of software are seen as part of software-competence, not as a failure of industry. Novices get met with 'you're holding it wrong', but at the same time veterans lose the plot and spend most their time doing technical onanism, or worse, fighting meaningless complexity.
Until the stack exceeds any individuals understanding, and the cracks start to show. Leaky abstraction upon leaky abstraction upon layers and layers of ducttape, under the guise of HA, distributed system mantras, robustness, you name it.
Every part needing babysitting because every part is half the solution to half of your problem, but admitting that is seen as a skill issue.
Yes my DB dies because it caches stuff and (under obscure circumstances) never frees it: 'thats just MySql, deal with it', yes of course my frontend framework needs to be kept up to date every week because everyone is evalling strings all the time, 'don't you sanitize your input?', multiplied by every layer of the stack.
Don't you let your static analysis automatically merge security PRs so you can stay on top of this giant heap of string evals? No? That's best practice man!