I can’t say anything.
1 Like
So … you earn Daily Bread to feed Baby Jesus by completing your Koine Greek lessons every day? And if you neglect your studies for 40 days and 40 nights, Satan appears and starts plying him with tempting loaf-shaped rocks?
6 Likes
found an interesting paper from Oxford last year about the perception of human-AI relationships from people that have one vs those that don’t, via self report on Replika.
https://arxiv.org/pdf/2311.10599
Some interesting comparison there, like self report of their effect on social health, non-users stay to normalish distribution, no surprise there:
What’s more interesting is both group’s reaction for a more human-like AI, totally polar opposite there.
4 Likes
Also a fascinating video couple days ago from Professor David Kipping, who runs an astronomy lab over at Columbia university, and his perspective on AI. He has other great videos unrelated to AI on his channel too that’s also worth checking out.
Interesting video talk, thanks for sharing.
“I don’t care, the advantage is too great,” is not nearly as startling a standpoint as he seems to find it. Being willing to throw out privacy concerns and ethical concerns is something clever people have been ready to do for some pretty trivial benefits.
We already knew that the subjective boost to productivity from seeing a simple prompt spit out lots of results is intoxicating – and this video confirms that it’s the case among some elite physicists as well as some elite software developers – but that objectively, actual productivity boosts can be significantly undercut or even completely undone by the work he mentions of cross-checking and fixing errors and hallucinations.
Even when they’re real, productivity boosts in things like managing your emails or even solving differential equations doesn’t necessarily speed us toward major breakthroughs – it can reduce academic drudgery without thereby creating “super scientists.” A lot of academic output is (at best) moderately interesting while making (at most) a marginal contribution to anything transformative.
So I await the promised “avalanche of discovery” with interest but substantial skepticism. If AI does provide the insight that gets us to cheap fusion reactors, great. That would totally be worth it. But so far, seems like the boosts to science are being delivered through specialized machine learning software (like protein folding, or maybe the search for exoplanets AI software that this guy worked on a decade-ish ago), rather than the giant scale LLMs. And that feels to me like a continuation of the scientific role software has been playing for over a half-century now, rather than an exponential shift upward.
In some fields (he talks about college admissions) productivity boosts largely cancel each oher out – applicants use ChatGPT to write their applications, staff use it to summarize and analyze those applications, and we ultimately end up with a lot more AI use but not a lot of actual value created. Sympathetic as I am to the vets who’ve been using GPT to access disability benefits, I’d bet on it just being move one in an arms race with the gatekeepers, in which AI will be wielded by both and the equilibrium outcome ends up not far from where we started.
Unless that’s all cut off by the “explosion of cost” the video creator thinks is likely coming. I agree with him that that would be likely even if the economics of LLMs worked the same way as other subscription services. But as we’ve discussed upthread, given that marginal costs per additional user are significantly higher for LLMs, it’s even likelier that access to LLMs eventually becomes something you have to pay a lot for.
His closing idea that an AI superintelligence might yield a world of magic – a world where the technologies we rely on are built on ideas and models that no human brain has ever grasped, or will ever grasp – is terrific as sci-fi. I don’t mean that in a derogatory way; sci-fi ideas are worth grappling with, and reality sometimes unfolds along those lines. But I continue to bet that we’re a long way from an AI singularity, so I’m not going to put much actual worry into it.
PS: The folks who are handing over control of their emails and personal data to AI agents really might want to reconsider.
And here’s a good (alarming) piece on the growing opacity of AIs and the implications for security.
10 Likes
A year ago, Starwish asked Open AI Deep Research to analyze CoG games by genre. The result was fluently written but had flaws you could drive a truck through – getting genres tangled, claiming that sci-fi was a top-selling genre (bringing in HGs to bolster the case), underselling the superhero genre. It ended up as a significant data point in my skepticism about the usefulness of AI in research:
Well, this week the blog Astral Codex Ten is running an AMA – Ask Machines Anything – for Claude’s latest paid version, as an effort to convince skeptics who only have access to free versions that the best AI has become very, very good indeed. I thought I’d see how much of a difference twelve months had made, and asked a version of Starwish’s prompt.
The result was, I have to admit, entirely superior across the board. Nothing jumped out at me as wrong, nor really even any major omissions. Fair play to our robot overlords – they’re improving at a rapid clip.
Meanwhile, I’ve also read an interesting article that puts more flesh on the bones of my continuing skepticism about the idea that AI is going to lead to a new scientific revolution – not so much in this case because of AI’s dysfunctions, but because of those in how we do science.
7 Likes
What is the COG stance on using AI to write, My apologies for the question , it has been a long time since I have been active here
edit: found the answer to my question
This is basically correct, but the numbers it uses to get here seem… suspect. The word counts at least I know are definitely wrong (it says Rebels is longer than Book of Hungry Names, as an example), and using Steam follows as proxy for popularity just feels bad, knowing most of CoG’s revenue is from mobile purchases (at least it was, I assume that’s still the case but don’t know modern figures). Definitely an improvement though (it seemed to not get caught on Robots as an outlier, for one), would be curious to see the takeaways from HG.
6 Likes
Is that really wrong, though? I mean, maybe literally, to some extent, but should it really be expected to… I don’t know, jump out at the disinterested reader?
(
)
1 Like
Oh dear.
“I was promised the Babel Fish and got Speedy Gonzales.”
8 Likes
I’m still digesting these news, so I’ll drop it here with no comments. What are y’all’s thoughts on this?
It’s funny to think the original plot fot the Matrix was that the machines used humans as processors, but the Wachowskis thought it was too confusing for the average movie-goer at the time so they changed it so that humans were used as batteries. We would make highly inneficient batteries, but now science is confirming fiction that we would actually make highly efficient processors.
I made this post to help people and this is what you take away from it 
If it makes you feel better I didn’t give it the code, I pasted the error from the actual terminal which is NOT CS.
4 Likes
So eh, Openai just used an general purpose LLM, which is not an hybrid system of symbolic reasoning plus LLM like Alpha Geometry, to disapprove an believed central conjecture for Erdos problem, you can read it more here:
https://openai.com/index/model-disproves-discrete-geometry-conjecture/
A shortened summary of the chain of thought process, which is 125 pages already, can be found here:
Now they’ve solved some minor ones a month ago, but my understanding is that this is a way bigger deal, and really publish worthy for a top one mathematics conference if a human discovered this.
The guy who maintains the Erdos problem website, who trashed Openai’s solutions on other Erdos conjecture last time a month ago, seems to be impressed with it
and that seems to hold true for every mathematicians, so I think I’m right in trusting them for the implications of this.
It seems one major factor, is that the LLM can just keep going at it with the assumption that the current solution is false, while a human would give up after a couple failed attempts, but is such tenacity not part of the research capability? many world changing discoveries were made through trial and error.
I don’t expect LLM to just solve every problem next year of course, but every progress is incremental. It’s not even 10 months ago I think, where IMO gold was “solved”, and this achievement really dwarfs that.
3 Likes
Being able to go 24/7 without needing to worry about earning money for rent probably helps there.
2 Likes
Fascinating – thanks for sharing! The AI-skeptical take had already dropped in my Inbox:
My takeaway is that it’s genuinely impressive and helpful, and comes in exactly the kind of space where computerized tools thrive:
- mathematics/ logic/ programming
- a problem where, as you note, the main advantage is inhuman tenacity – the ability to systematically brute-force solutions down lots of pathways that a human might judge not “worth their time to explore”
There’s a bunch of worthwhile work to be done in that space. The mathematician Marcus checks in with says, “I can think of any number of results I’ve worked on in my career where I could have moved faster or been more comprehensive if I had access to [tools like this].” Which is great.
While I think this is a point against broad AI skepticism (“this is worthless stochastic parrotry”) I don’t think it strikes against “AI bears” in terms of bubble wariness. It’s hard to assess the cost-effectiveness of solutions to an Erdos conjecture, since academic mathematics isn’t a profit-making enterprise; but it does very little to shift my bet that LLM Gen AI is going to turn out to be deeply unprofitable, and that the wake of the bubble bursting will be a bunch of improved sector-specific machine learning tools rather than a general intelligence that could replace human workers at large scale.
3 Likes
Whatever you think of AI it is very good at finding errors in code, especially when you tell it what is going wrong, and especially when there isn’t a lot of code.
I also think you are missing the point that it correctly identified the problem with the code and how to fix it and the fix worked. OP is just bring it to the devs attention so they can fix it properly.
I’ve seen other people recently having this exact issue. Something obviously went wrong with a recent template update.
2 Likes
I think you’ll find that some people oppose the use of generative AI for any purpose whatsoever, not just those that legitimately infringe on intellectual property and human creativity. From that perspective it doesn’t matter whether AI is genuinely effective at some things, and anything beneficial that it accomplishes is fruit from a poisonous tree.
8 Likes
Because literally everything allegedly useful could be done without the energywaste and plagiarism.
GenAi actively drains funding and other resources from the development of actual useful AIs
1 Like
I view it like fire, or nuclear power. Incredible destructive possibilities, but it enables many beneficial things not possible without it. AI use for unfolding proteins, medical diagnostics, and material science have already been as potentially life saving as the discovery of penicillin.
I also think there is genuine benefit to near instant translation between different cultures and groups.
I mean, it sounds like we agree. I just thought it was odd you’d criticize an author for using AI to help them diagnose and fix computer code they couldn’t understand and just wanted to fix so they could write. That’s not generative AI.
1 Like