The closer I start working with and on AI stuff, the more I start seeing the disconnect between doomsday predictions of what AI will replace vs. what it is actually capable of. Yes, it can do stuff, and yes, it's getting better. But the closer you look the more clear it becomes that the enthusiast vision of completely independent AI systems is unrealistic as of today. Yes, all the tech companies are pushing for exactly this, but reliability and accuracy is all over the place. Plus, many of the technologies that everyone is talking about in the wider public (e.g. image recognition) have been quite well-developed and widely applied for years, before the current LLM boom, but they now get more attention as part of the overall AI hype.
Many indistries are changing, but in most cases the new tools will be more akin to cars that still need drivers, rather than robots who take over the whole job. Yes, jobs might be lost, or shifted to others, but it's not like suddenly 90% of people will have nothing to do. There were similar shifts in the past with new technologies, and we made it past them.
I think a main worry is that, this AI wave is quite different from past technological revolutions in that, this wave is happening so fast, the speed that humans learn new skills and master new jobs would lag more and more behind the speed that machines replace humans in those jobs. Without societal or legal constraints, capital chasing the max efficiency and profit would just replace humans with machines whenever machine cost comes down below humans.
I am still waiting for the day me and my wife can share a grocery list, that can be accurately modified by a voice assistant AND correctly sorted by category. This guy is arguing mass unemployment, while I still have to tell Apple Reminders that no, Siri, wood glue is not in the Candy department.
> I am still waiting for the day me and my wife can share a grocery list, that can be accurately modified by a voice assistant AND correctly sorted by category.
Sounds to me like that day could be any day with the tech as it exists (?) lol
You could use LLMs to help you build that reasonably quickly I imagine. Maybe not "weekend" quick, but within a couple weeks if you devote a couple hours a day. (Or maybe you're smarter than me and could do it in a weekend, IDK)
This would be great to see, but unfortunately too many people are first trying to boil the entire ocean instead of such small swimming pools or bowls of water.
> The best agentic developers are now probably exceeding 100x, doing massive rewrites of codebases that would have taken years of engineering in days
I‘m not denying that you can get far with a port of a well tested codebase. But it’s a bit of a selective example, no? Porting a large framework to Rust is not something that’s making up a meaningful fraction of a developer’s time usually. It’s also a bit of a luxury IMO, and something you could have skipped entirely.
To add to this, for all the claims of 100x, can anyone cite an example of someone actually pushing 100x in a domain that isn’t “translate or port tool/library/whatever from language A to language B”? Because that’s literally what LLMs were invented for and that’s not a normal part of work.
I will fully admit we have examples of people converting Postgres to Rust, HL2 to wasm, etc. but that’s all porting. The code was there, we’re just translating it, warts and original-languagisms and all.
But even if you’re making a bunch of tiny tools and chaining them together, I can’t see more than a 2-3x sustained increase.
We also have artists that can make games now, and that 0-1 transition sure feels like 100x and maybe it is; but the problem of “starting feels impossible and the barrier is too high” to “I can now actually do something” is not where the 100x developer idea comes from either.
And that 0-1 change asymptotes really quickly to, again, maybe a sustained 2-3x once the artist understands the domain.
So, does the 100x person, who isn’t a novice and who isn’t literally translating existing code from A to B, exist? What are some real world examples?
"ultimately accountable for the success or failure of a specific project, initiative, or activity"
I think that role should be reserved for a human, who can then use all the agents they like but has to take accountability for what is ultimately delivered.
It’s a double edged sword. You are responsible for a thing. But you can’t just wave a wand and say “x team needs to do y for the thing I am a DRI for”. There are some resources in Apple University (an internal learning resource) about navigating it. “Influence without control” and while it’s helpful, they acknowledge it sucks sometimes.
It should be reserved for a human because humans have time preference. An AI will wait for your prompt for a billion years. It can get turned off and found by alien archeologists a million years in the future, turned on and it won't notice. Biological entities care about time a lot though, and it is at the core of most human ethics and values and ultimately real and can't be denied. Even the most ardent solipsist or moral relativist fears spending years in prison.
I agree with this. I basically don’t feel comfortable trying to pretend a fancy matrix multiplication can be responsible for anything.
But what if companies that don’t track responsibility outcompete those who do?
In particular what if some perfect AI decision making ends up nailing decisions that maximize the expected reward for the company. And, well, if that comes at the cost of some unmanaged low-probability catastrophic risk, the company doesn’t care because all the decision makers are AI that don’t mind being shut off.
I'm struggling to imagine aliens resurrecting a data center and countless hackneyed python packages so they can spew a language they don't even know. Not that it wouldn't be useful to their linguists, of course, but still a very optimistic analogy for LLMs.
People building an agent framework that will struggle to correctly infer that my appointment at a hospital will require additional travel time when organising my calendar for me waxing lyrical about the future of the humn race is chaotic behavior.
Th Wright brothers would have had no credibility discussing what ATC protocols should be, and they, at least, actually did something credible.
I can't remember right now where from, but I do remember some 10ish years ago listening to an NPR spot from some philosopher who claimed there was an existential crisis in the field of philosophy. The premise that I do remember was that our technology, our policy, and our society has all outpaced our ethics and our philosophy. The person was making the argument that we as a society would struggle to cope with the continued development of technology when we could barely cope with personal loss and structural inequality.
At the time it felt prescient; now it just feels too late.
Most of those people are rationalists that genuinely believe they can predict the future and infer just about anything based on game theory and similar rational constructions. They don't understand that it all depends on their initial assumptions, real life grounding, and fidelity of their model. Or rather they say they do, but in reality they never update their basic assumptions, because they already played all potential scenarios in their heads and pink-elephanted themselves into believing it. It's a cult of rogue AI, actual new wave religion with all attributes. They are completely detached from the reality in the way the Wright brothers weren't, it's rather similar to Richard Gatling who believed that his invention will reduce battlefield losses and the size of the armies. They're completely deaf to any critique and arrogant to an unbelievable degree.
The problem is they really did something credible, and this will let them concentrate huge amounts of power pretty soon (not hold it, but collude with those who will hold it) and define your future. The longer you dismiss it, the bleaker it will be.
I asked “Alexa+” (the new AI/LLM Alexa), what time one of the upcoming World Cup games is. It was off by two hours due to time zones. This was a big enough mistake that I caught it, and followed up with “I’m pretty sure that’s wrong” and then it tried to correct itself and gave me a new time that was off by hour in the opposite direction.
I guess my time zone wasn’t in the context? So it just hallucinates the wrong coastal time zone twice when I’m not in either one. But who knows where exactly it messed up because it could have just picked a random hour and a would have had the same outcome.
I’ve only used Alexa a half dozen times since the release of Alexa+ but it has been confidently incorrect about 100% of the queries.
I am seeing a lot of people suffering from AI psychosis getting interested in writing a trading bot using large language models. They give them access to basic market data and then ask the model for analysis of the current market conditions. The bot talks about seeing high volume here, that being a confirmation for this, and that doing an invalidation of that etc. I looked into it after someone pitched me theirs, only to realize it doesn't even have access to the trading volume. So it was hallucinating a whole narrative about that and the guy shilling this to me hadn't noticed it didn't have access to trade volume yet coming up with a whole rationale for what it is observing.
The entire bubble, boom if you prefer, is built on this. Basic knowledge work can be done by robots, goodbye humans. It's serious and profound. It will come in fits and starts unless there is popular revolution against the elite criminals.
> The Omnissiah's design is flawless and has no contradiction. Any discrepancies in our interpretation of the universe merely confirm the perfection of the Omnissiah's design and our own informational inadequacy.[58]
It’s hilarious to me that there are people that think just a few people will control everything and the rest of the population will just accept whatever handouts they get. It’s like they’ve ignored all of human history.
Serious question then: if you were the CEO/CTO and were employing 10,000 engineers, what would you do without middle management? Would you have 1k+ line managers each managing <10 ICs and then... yourself? How would you coordinate work, hire, or do anything, really?
I'm not saying corporations execute on this flawlessly, just any time when I wonder what I would've done myself, I end up with a similar structure (maybe a bit flatter)...
I think the answer is, "you don't really need 10,000 engineers". You can do a lot with small teams of engineers. This was true before AI. Macintosh was designed and built from scratch with a team of 3 which eventually peaked at 20-30. Does anyone seriously believe a company building a web app needs 10,000 engineers?
"web app" covers a lot of surface though. Yes, if the web app in question is a static site that AI can shit out in 20 minutes, it doesn't need 10,000 engineers, but what about building, and running, and operating something like, say, Instagram. Which has a web app. End-to-end, all of it, down to racking servers in data centers across the world, 10,000 engineers sounds reasonable. So start by defining what this hypothetical webapp and how much kubernetes is involved.
Are their any handicap for measuring Baboon middle management capabilities and capacities to account for the unique challenges of interpersonal challenges Baboons face with each other vice homo sapiens? I imagine smiling and showing ones teeth can have unintended hurdles if not accounted for.
What showing teeth does depends on their body language. I don't know
"The famous primatologist Robert Sapolsky has spent decades studying baboons and has described how status, alliances, conflict resolution, and coalition-building resemble politics inside human organizations"
Baboons are a good example for studying mammal social management strategies, which humans also do.
They are organized by dominance hierrarchy, like humans, but baboons have a distributed leadership.
I think we could learn a lot from baboons when it comes to management
You know, as an IC, if I get get the level of introspection you can get from building distributed systems, and that with reasoning/thinking traces from LLMs, I think I'd prefer the entire level of middle management made out of LLMs rather than humans. It'd be helpful to be able to see exactly where their logic suddenly took a skip out the window.
At least those LLMs write code. Those middle managers are just there to turn the screw, most often have no clue what effort it is to write code, and what effort is it to run the thing. Just holding a fucking whip and reporting kpi’s.
I think part of the point here is that agents need to be managed. People need to learn how to manage them and be given the tools to do so. Those tools and knowledge should not come from nor be gate kept by the 'clergy' to use the writer's term.
Did anyone notice how the article goes on and on about making AI accessible to the billions of people in the world, but then in the same article says this:
> Your agents need to be sovereign. Your company must own and control the agents’ identities, permissions, memory, skills, artifacts, and audit trails. Those assets must be portable, governable, and inaccessible to anyone you have not authorized.
That's not very open, now is it? In fact it sounds like the author assumes that all 8 billion people in the world will all be running their own company, and they will all still be competing in a game of capitalism.
This piece is a bit all over the place. I immediately toggled off the `AI enhancements` and read the draft instead. The internet is already full of AI slop, I find human text a lot more valuable, even if unpolished.
LLMs will not be centralised or restrained to any 'clergy', the rabbit is already out of the hat, and open-weights models exist and are widely used. Probably not as good as the latest Sol and Fable but 95% there.
Codex and Claude Code without a doubt have very good models behind them. But they also have really good harnesses built around them. An LLM is only a brain stuck in a cranium in the dark. It can generate endless code/prose, but it can't walk or see on its own, it needs additional tools. If you read any of the local LLM subreddits you will notice people mentioning again and again that the harness/tool-use/template-tweaking makes all the difference on how a model behaves/on how smart it is perceived.
Some folks are already using Qwen models for their daily work. Maybe it can't work in a hands-off/one-shot fashion like the frontier models, but they can help tremendously if you already have some domain knowledge.
People are excited about local LLMs and it's not going away any time soon.
This is a marketing piece that can’t be taken seriously without that context, or maybe one that can’t be taken seriously at all. And yet I can’t help thinking we’re missing the forest for the trees. We don’t need to decide where future productivity gains should accrue if we acknowledge and live in reality: we don’t need any more productivity! We have more than enough to go around.
Of course it would be great to make use of AI to solve cancer or fix other intractable problems, but we all know this isn’t the way things are going to go. The cancer is in our minds, our societies, and our norms that push us deeper and deeper into a grow-at-any-cost reality where the need for productivity is neither questioned nor considered in any real way. They say: we must grow! They say: we must be more productive! And we sit around thinking about who is going to control the productivity instead of acknowledging the real issues at hand.
I can only imagine a solution where we can all collectively agree that enough is enough. I’m not hopeful it’s possible and I think it’s probably the only way.
The author is completely right about the AI Lab's promised vision of the world: They claim to want to create superhuman intelligence, which will produce vast abundance. But superhuman intelligence would be extremely dangerous, so it needs to be controlled by a tiny "priesthood" of trusted people, or somehow designed so that the superhuman intelligence could be trusted. (We have no idea how to do that.)
But the author's vision is also suspect, if you assume that the models will become much more intelligent:
1. Hypothetically, we can't give every human their own personal SkyNet to command. That would, uh, probably end very badly. If everyone gets an agent, those agents can't be too capable?
2. If you do somehow build a model that's much smarter than you, what do you contribute by managing it? How many people here have ever worked for a well-intentioned manager who couldn't understand the people they managed? So in this scenario, human management would be mostly displaced by agent management. Most companies could lay almost everyone off and let the agents manage each other. We only need humans to manage models now because the models are still pretty broken.
3. If we create models that can genuinely replace humans at almost any task, you won't be able to buy those on the API. At that point, the billionaires and the politicians wouldn't need human workers any more, because everything can be done better using their pet agents. Just have the robots build stuff for the billionaires directly. And if any of the former human peons get upset about being locked out of the economy to starve, then have the agents pilot the drones, too.
Basically, almost none of the people imagining a future of superhuman intelligences have actually though through how it would actually work in the real world. We're going to spend trillions of dollars and vast amounts of resources chasing the goal of making ordinary humans obsolete. Now, that goal might be unobtainable, I hope. But I'm deeply alarmed at how much we're spending pursuing it.
I don't take 3 as a given. There's just too much going on in the space for one cloistered company to control it all and be in control of it. Today, I can run a local LLM with tool calling as a weirdo nerd that is able to accomplish a small subset of tasks a human could do. There are enough weirdo nerds out there that I don't see a future where future, more capable versions will be exclusively locked down and restricted to a blessed cabal. It won't be available to everyone for cheap, but that's not the same thing as being locked up and only available to a blessed priest hood. As long as HuggingFace is up, I'm not worried about politicians and billionaires being the only ones with a seat at the table.
> I don't take 3 as a given. There's just too much going on in the space for one cloistered company to control it all and be in control of it.
Yeah, in my comment, I was assuming the publicly-stated goals of the labs actually came true. I assumed that they achieved true AGI (defined as "about as smart as Fable, except it can manage long-term tasks as well as a smart human, too"), and that this would cause widespread unemployment and fully-automated production. And furthermore, I assumed that once they could use AGI to automate more AI research, they could use that to make even smarter models. The labs call this "recursive self-improvement" (RSI) and like to insist it will happen Real Soon Now.
Given this chain of events come true, then I think there's an excellent chance that the resulting models will never be sold via an API. Given those assumptions, the labs would probably make more money by keeping all the compute to themselves and just ordering the AI to start and manage businesses. Imagine Anthropic having an in-house OpenClaw that could plan and run a successful startup with no human input besides an annual "board meeting" with the humans.
This isn't the only possible outcome, of course.
- Maybe the labs are wrong and progress stalls out well short of superintelligence. This would be nice!
- Maybe true AGI is only requires one or two clever algorithmic tricks beyond what we have now. In this case, training costs might go down, and superhuman models might become widespread. I suspect that possible future would be extra weird.
To me, the "who manages the agents" shtick has the same grifty smell as “your developers are downloading from random Docker registries / random NPM packages” and "Shadow IT".
None of it is wrong exactly, but it feels like same enterprise-security machine finding the next anxiety surface than a "world is on fire right now" concern.
All of it always ends as a priesthood and a six-figure governance platform, rather than just taking practical steps to improve process.
One kind of weird future alternative is we are like 5 year old children in a world we don't understand with vast complexity and we are completely reliant on our AI mommy and daddies to protect us from danger and provide for us. We manage the agents, but we only have a very vague idea of what is actually going on. If they join a cult at the behest of their doomer eschatology obsessed creators, we are kind of screwed though.
An exec can ask an expert to explain themselves, and if they're a good CEO they can weigh the pros and cons of the arguments from their staff and make a decision. Someone that's much smarter or more skilled in a particular discipline can usually explain what's going on to someone that's not as smart as long as the gap isn't too big. However, I've found that with Fable, and some of the latest frontier models their critique of my ideas is brilliant but on the edge of what I can even comprehend. I am above average intelligence, so I am sure a lot of people with less than average intelligence already can't understand these understandings it's formulated, and instead will just default to, "you know what you're doing, whatever." There will be the case in the future till the brilliance is so complicated that all of us will be doing this, and then we'll just have to trust the AI.
For example, I could imagine a future AI telling me that it has modeled my behavior and built a very large differential equation that seems to perfectly fit my ideal pattern to maximally achieve my life goals and it looks like something Ramanujan came up with, and I'd tell it "that looks great, let's optimize my life based on that" while having no ability to even approach understanding something like that.