Artificial Intelligence

[ysabetwordsmith]

Professors Staffed a Fake Company Entirely With AI Agents

As Business Insider first reported, the results were dismal. The best-performing model was Anthropic's Claude 3.5 Sonnet, which struggled to finish just 24 percent of the jobs assigned to it. The study's authors note that even this meager performance is prohibitively expensive, averaging nearly 30 steps and a cost of over $6 per task.

Google's Gemini 2.0 Flash, meanwhile, averaged a time-consuming 40 steps per finished task, but only had an 11.4 percent rate of success — the second highest of all the models. The worst AI employee was Amazon's Nova Pro v1, which finished just 1.7 percent of its assignments at an average of almost 20 steps.


While corporations may wish to replace human employees with software, it is not yet feasible for complex tasks.  Only the simplest jobs are really at risk.

Comments

[lilyhargrave]

That is somewhat reassuring to know!

[mdlbear]

Another article on the same site says that AI Chatbots Are Becoming Even Worse At Summarizing Data. Why am I not surprised?

Well ...

[ysabetwordsmith]

One explanation is Hapsburg AI. The AI content is splattered far and wide, deliberately made hard to identify and avoid. But when you trait new AI on old AI-generated material, then you tend to get gibberish. So they shot themselves in the foot there.

Another is that humans are less good at many logical tasks such as summarizing data than they used to be, as education degrades due to low investment. I've seen some painfully bad summaries of scientific studies. You can't teach what you don't know, so AI learning from idiots will be inept.

Re: Well ...

[gatheringrivers]

...Hapsburg AI.

LOL! I'm going to have to remember that, thank you for putting it that way! :)

Re: Well ...

[ysabetwordsmith]

Here's a reference:
https://futurism.com/ai-trained-ai-generated-data-interview

[gatheringrivers]

Sadly, it won't stop them from TRYING at least.

Well ...

[ysabetwordsmith]

Not immediately, but eventually. AI is ruinously expensive to program, train, and run not just in employee hours but in energy costs. If they can't find a way to make it profitable, then it will collapse sooner or later, leaving only the handful of areas where it is actually helpful. Remember the dotcom bust? Like that.

Computers and humans are just good at doing totally different things. If it's a task based on logic, precision, or math then a computer will often excel. But if it requires creativity, intuition, or making something from scratch then you're better off with a human. So while we may see AI stick around for certain things like telling you which of 20,000 lidar images have ruins in them, it is not good enough to do most human jobs.