I wouldn’t characterize the dialogues that way. Sure, the bot’s initial output is “go get checked for COPD at the hospital, and if they confirm my preliminary diagnosis, here’s what you’re likely to be prescribed” – but when it’s then asked to recommend medicines the patient can pick up for himself, it straightforwardly recommends some “suitable” medicines. It does hedge that with caveats like “follow medical advice, always consult a doctor before taking meds,” which is great, but I don’t think it’s right to say it’s not being asked what it would prescribe/recommend.
And even if it were, I don’t think that would make the difference you think it does. Should the bot’s highly inaccurate predictions of human doctors’ actions be more encouraging than if it had been given the mandate to overprescribe in its own right? Should we really expect its response to be meaningfully different if asked to make its own prescriptions? (What it should really say is “I can’t make any recommendations because a conclusive diagnosis would require real-world tests of your body that I can’t perform until they give me access to better robotics,” but that’s obviously not what it’s trained to do.)
Yes, in this specific case it looks like an algorithmic tweak of “just go with the first answer on your list” would lead to an improvement – but that’s hardly the general case. As you know better than just about anyone here, a huge amount of work over the past few years has gone into finding ways to encourage AIs to do precisely the opposite – to not go with the initial top-of-mind answer but think through it to avoid errors. A single study in which the “first response” rubric would outperform randomly chosen middle-income country physicians doesn’t mean that we’re just one “very easy” tweak away from reliable medical AIs.
I don’t either! And yet it seems to be so. As best I can see, because LLMs are black boxes that we force-evolve rather than deterministically code, linking them up to more specialized tools has proven nearly as tricky as hallucination reduction. Maybe we’ll end up finding that a LLM that can play chess at Stockfish’s level is as evolutionarily unlikely as a winged shark.
Glad you can appreciate this despite the video including other perspectives opposite to your own. The creators didn’t take the “theft” idea seriously enough to keep them from buying and using AI subscriptions, and it was only when the AI research bots catastrophically failed them that they made and put out this warning video.
Your blithe final paragraph ignores the fact the output of AI slop is exponentially greater and faster than inaccurate human slop ever was. If “low-effort” means not fact-checking everything it gives you, then very few people are ever going to invest in “high-effort” use of AI. It’s not (mostly) “bad actors” pumping out what AI gives them, it’s normal people.
Here in Nepal, almost the only ads YouTube serves me are for AI services – make a video! write a book! create a website! – and of course not a single ad says “be sure to double-check for hallucinations.” Maybe the people making those ads are reckless and bad, but the users are mostly going to be normies reassured that their robot guru will sort out all the facts they need to put out there onto the net.
If some change in our doctor training system meant that suddenly the ratio of slop doctors to good doctors went from 5:1 to 500:1, it would be a real problem for my ability to “go see a doctor.”
