That in my experience is never the bottleneck, at least not for professionals. Letting Claude code is the easy part of the job. Gathering the requirements, Drafting a good story for claude, guiding claude through the endless mistakes it usually commits, reviewing the output of claude, steering the ship, this is the bottleneck. Unfortunately, no AI can't steer the ship currently, not o3 pro, not gemini, not opus 4, not any of their fancy cli agent tools, no matter how clever the md instruction files and other gimmicks. And boy, i'd be the first one to cheer if AI could do this. But currently, it's useless without fulltime attention of a senior experienced human.
This over reliance on llms is crazy. People are going to forget how to code. Sometimes the llm makes up shit or uses the wrong version of the API. Sometimes it's easier to look up the documentation and write some code.
Jokes aside, while I'm almost sure that the ability to code can be lost and regained just like training a muscle what I'm more worried is the rug pull and squeeze that is bound to happen sometime in the next 5 to 10 years unless LLMs go the way of Free Software GNU style. If the latter happens then LLMs for coding will be like calculators and such more or less and personally I don't know how more harmful that would be compared to the boost in productivity.
That said if the former becomes reality (and I hope not!) then we're in for some huge existential crises when people realize they can barely materialize the labour part of their jobs after doing the thinky part and the meetings part.
I don't think the rug pull and squeeze are possible. Because I've had the same worry. But using an existing LLM to train or fine tune a new one seems to be standard practice, and to work quite well. So any LLM with an API will end up training all the others - even open source LLMs - and all will benefit. And every day that passes, Moore makes it less and less costly for amateurs to commit the compute necessary for fine tuning, and eventually training from scratch.
In time, even video and embodied training may be possible for amateurs, though that's difficult to contemplate today.
Pretty much me with some IDEs and their code inspections and refactoring capabilities and run profile configurations (especially in your average enterprise Java codebase). Oh well.
The future is going to be great for us that have been resisting going all in. Unfortunately I feel a lot of work will be detangling the mess these llms make in larger repos.
The devils is in the details, as they say. And software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business.
I mean, it’s just like having an army of interns that works for (near) free. It’s a huge positive for productivity, and I don’t think we will forget how. I’m more concerned with how we make new senior/staff engineers from now on, since the old “do grunt work for a couple years, then do simple well defined work for a few years” is 100% not a career path that exists even now.
This is my question as well. I am already wondering how prepared college grads will be. Getting help with programming assignments meant going to the dungeon and collaborating with fellow students while the TA made their rounds and overall just figuring it out. Today, an LLM knocks out the programming assignments in once shot, probably. And industry seems hellbent on hiring seniors mostly so where are the juniors to become seniors going to come from?
I think the talent pipeline has contracted and/or will and overcorrect. But maybe the industry’s carrying capacity of devs has shrunk.
Why is the API usage-based billing so much more expensive than the $20/month tier? Like you can literally burn through $20 of usage in a day with the same amount of usage that you get included with the $20/month plan
Since downgrading from max to pro.... i have been using sonnet 4 a TON and i havent even been limited yet. The usage allowance is awesome since gemini cli released.
Opus and Sonnet are pretty similar for mainstream stuff heavily represented in the corpus.
But when you get into dark corners, Opus remains useful or at the minimum not harmful, Sonnet (especially Claude Code) are really useful doing something commonly done in $MAINSTREAM_STACK, but will wreck your tree in some io_uring sorcery.
I love these work-arounds and generous tiers. A bit of a tangent, but with very cheap essentially unlimited code generation, are there any active projects that just run this for days straight with an ambitious goal like "Develop an operating system" with instructions to just make all the necessary decisions to continue work?
I would love to see what a system like Claude Code could cook up running continuously for weeks. But I imagine it would get stuck in some infinite recursive loop.
I think you might be aiming too low. Tasked with writing a "perfect and most useful program" this would surely yield something more than merely writing 42 to stdout.
Current llms get lost fairly quickly in larger projects. They still benefit from reduced scope when promoting. Context is the biggest bottleneck right now by far. You can only summarize so much before the information is too vague to make meaningful changes.
Seems convenient. I have a couple of different Claude accounts, so I switched between them when one gets exhausted. Sometimes they both get exhausted. If other people have a couple of accounts then that would be a nice feature to add to this: switching between accounts and then resuming when either of them becomes available again.
That in my experience is never the bottleneck, at least not for professionals. Letting Claude code is the easy part of the job. Gathering the requirements, Drafting a good story for claude, guiding claude through the endless mistakes it usually commits, reviewing the output of claude, steering the ship, this is the bottleneck. Unfortunately, no AI can't steer the ship currently, not o3 pro, not gemini, not opus 4, not any of their fancy cli agent tools, no matter how clever the md instruction files and other gimmicks. And boy, i'd be the first one to cheer if AI could do this. But currently, it's useless without fulltime attention of a senior experienced human.
> And boy, i'd be the first one to cheer if AI could do this.
Yeah, well it would be the next major step towards human irrelevance.
Or at least, for developers.
This over reliance on llms is crazy. People are going to forget how to code. Sometimes the llm makes up shit or uses the wrong version of the API. Sometimes it's easier to look up the documentation and write some code.
It's the calculator all over again!
Jokes aside, while I'm almost sure that the ability to code can be lost and regained just like training a muscle what I'm more worried is the rug pull and squeeze that is bound to happen sometime in the next 5 to 10 years unless LLMs go the way of Free Software GNU style. If the latter happens then LLMs for coding will be like calculators and such more or less and personally I don't know how more harmful that would be compared to the boost in productivity.
That said if the former becomes reality (and I hope not!) then we're in for some huge existential crises when people realize they can barely materialize the labour part of their jobs after doing the thinky part and the meetings part.
I don't think the rug pull and squeeze are possible. Because I've had the same worry. But using an existing LLM to train or fine tune a new one seems to be standard practice, and to work quite well. So any LLM with an API will end up training all the others - even open source LLMs - and all will benefit. And every day that passes, Moore makes it less and less costly for amateurs to commit the compute necessary for fine tuning, and eventually training from scratch.
In time, even video and embodied training may be possible for amateurs, though that's difficult to contemplate today.
Just like how people forgot how to patch phone lines and punch cards.
> People are going to forget how to code.
Pretty much me with some IDEs and their code inspections and refactoring capabilities and run profile configurations (especially in your average enterprise Java codebase). Oh well.
> People are going to forget how to code.
Which is a problem when exactly? When civilization collapses?
The future is going to be great for us that have been resisting going all in. Unfortunately I feel a lot of work will be detangling the mess these llms make in larger repos.
The devils is in the details, as they say. And software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business.
I mean, it’s just like having an army of interns that works for (near) free. It’s a huge positive for productivity, and I don’t think we will forget how. I’m more concerned with how we make new senior/staff engineers from now on, since the old “do grunt work for a couple years, then do simple well defined work for a few years” is 100% not a career path that exists even now.
This is my question as well. I am already wondering how prepared college grads will be. Getting help with programming assignments meant going to the dungeon and collaborating with fellow students while the TA made their rounds and overall just figuring it out. Today, an LLM knocks out the programming assignments in once shot, probably. And industry seems hellbent on hiring seniors mostly so where are the juniors to become seniors going to come from?
I think the talent pipeline has contracted and/or will and overcorrect. But maybe the industry’s carrying capacity of devs has shrunk.
It will probably be like coding in assembly after the advent of the compilers. There are some people who still code in assembly, but it's rare.
so software will get even worse because nobody understands anything about how computers work?
Why is the API usage-based billing so much more expensive than the $20/month tier? Like you can literally burn through $20 of usage in a day with the same amount of usage that you get included with the $20/month plan
Since downgrading from max to pro.... i have been using sonnet 4 a TON and i havent even been limited yet. The usage allowance is awesome since gemini cli released.
As someone who cpuld have never considered Claude Code due to the cost of Max/Opus - do you notice any differences in practice with just Sonnet?
Opus and Sonnet are pretty similar for mainstream stuff heavily represented in the corpus.
But when you get into dark corners, Opus remains useful or at the minimum not harmful, Sonnet (especially Claude Code) are really useful doing something commonly done in $MAINSTREAM_STACK, but will wreck your tree in some io_uring sorcery.
Sonnet is noticeably worse in my opinion. It’s worth it to spring for Max and only use Opus
I love these work-arounds and generous tiers. A bit of a tangent, but with very cheap essentially unlimited code generation, are there any active projects that just run this for days straight with an ambitious goal like "Develop an operating system" with instructions to just make all the necessary decisions to continue work?
I would love to see what a system like Claude Code could cook up running continuously for weeks. But I imagine it would get stuck in some infinite recursive loop.
I think you might be aiming too low. Tasked with writing a "perfect and most useful program" this would surely yield something more than merely writing 42 to stdout.
Current llms get lost fairly quickly in larger projects. They still benefit from reduced scope when promoting. Context is the biggest bottleneck right now by far. You can only summarize so much before the information is too vague to make meaningful changes.
It would probably look suspiciously like Linux.
pairing this with Task Master could allow you to draft all of your tasks and effectively have Claude pick something from an endless backlog 24/7...
You need some time to chain tasks. Endless tasks, can't be in random order and there is usually a link between tasks/context.
Doesn’t Taskmaster require an API key? It doesn’t work with a subscription.
You can technically hack the API key from the subscription, but that’s probably brittle.
Or is there some other meta I’m missing?
https://github.com/eyaltoledano/claude-task-master/pull/805
Nice, thanks!
My extension in vscode have auto resume already, you use GUI to write workflows and run them right them. https://marketplace.visualstudio.com/items?itemName=Codingwo...
You can even pause. I will public a CLI that is doing same base on same syntax. And it use github claude action yaml syntax: https://github.com/codingworkflow/claude-runner/blob/main/.g...
Seems convenient. I have a couple of different Claude accounts, so I switched between them when one gets exhausted. Sometimes they both get exhausted. If other people have a couple of accounts then that would be a nice feature to add to this: switching between accounts and then resuming when either of them becomes available again.