Hey Freelance Friends!

I spent most of this edition doing something no client was paying me for: chasing down whether a handful of statistics I wanted to use were actually true. True, as in I could trace it back to a name, a date, and an institution willing to put their reputation behind it.

Two of the numbers I wanted to use didn't survive the chase. They came from sites that exist purely to repackage other people's research into cleaner-looking lists, no methodology, no accountability, just a headline that felt right. I cut them. That decision, alone, took longer than writing this section did.

The scarce resource isn't content anymore. It's trustworthy synthesis. And trustworthy synthesis, it turns out, is a skill, not a subscription.

Here's what sent me down that rabbit hole in the first place. In March 2025, researchers at Columbia's Tow Center ran 200 queries through eight AI search tools and asked them to do something embarrassingly basic, correctly identify the headline, publisher, date, and URL of a real news article. The tools got it wrong more than 60% of the time. Grok, specifically, failed 94% of the time. Open-book tests where the AI refused to open the book correctly.

For twenty years, the internet's problem was access, find the source, find the fact, find the file. Google solved that. That bottleneck is gone now, and what replaced it is worse: anyone can generate a fluent, confident paragraph about anything in four seconds, so the question stopped being "can I find this" and became "did anyone actually check." Most of the time, the honest answer is no.

The word makes it sound rare, dreamlike, almost charming. The reality is blunter: AI systems confidently invent things and say them with the same tone they'd use for something true. There's no flicker, no hedge, no "I'm not sure." Just clean, fluent, wrong.

Here's how widespread that actually is, with receipts:

News retrieval: A landmark EBU/BBC study tested four major AI assistants, ChatGPT, Copilot, Gemini, Perplexity, across 3,000 responses in 14 languages. 45% of answers had a significant issue. 81% had some kind of problem, once minor issues were counted too. Gemini was the worst performer, with serious issues in 76% of its responses.

Source citation: Columbia's Tow Center found AI search tools failed basic news-sourcing tasks over 60% of the time across 200 tests, and that's the average. Grok alone hit 94%.

Medical terminology: Researchers at the Icahn School of Medicine at Mount Sinai built 300 fictional clinical cases, each with one fabricated detail, a made-up lab test, a fictitious symptom, an invented syndrome, and fed them to six leading AI models. Hallucination rates ranged from 50% to 83%. The models didn't just repeat the fake detail. They elaborated on it, confidently, offering clinical-sounding explanations for conditions that don't exist. A one-line warning in the prompt cut the errors substantially, which tells you the failure isn't inevitable, it's a default nobody's forced to accept.

Academic citation: Multiple studies on AI-assisted research synthesis have flagged a recurring pattern, fabricated citations that look exactly like real ones, down to the formatting.

A large language model isn't built to be right, it's built to be plausible. Under good conditions, plausible and right line up most of the time. Under thin, ambiguous, or contradictory conditions, the model keeps generating fluent language anyway, because fluency was the job. Truth was never actually the spec.

Humans read fluency as credibility. It's not malicious, it's just how brains and sentences have always worked together, and it's exactly the seam AI slides through at scale.

Verification used to be a job title. Now it's becoming a background skill everyone needs, the way "basic computer literacy" quietly became a requirement nobody announced out loud.

Here's who gets paid more in a world flooded with plausible-sounding noise:

The pattern: none of these are "AI skills." They're investigative habits, source-tracing, primary-document checking, healthy suspicion of anything too clean, that used to live exclusively in journalism, law, and academia. Now they're spreading into every job that touches a screen, because every job that touches a screen now also touches a model that occasionally makes things up with total conviction.

This also reframes the AI-skills-gap conversation people keep having wrong. It's not really coders versus non-coders anymore. The actual divide is people who can evaluate information quality versus people who can't, and that line cuts straight through both groups. Plenty of highly technical people fail this test constantly, because "sounds sophisticated" and "is accurate" feel identical from the inside until you check.

A study of AI hallucinations in healthcare queries, the MedHalu research, found that laypeople trying to catch a fabricated medical claim performed no better than chance, and sometimes worse. Experts caught the same errors reliably, but only because they combined prior domain knowledge with actively cross-checking trusted sources. A separate study at a UK business school tested 211 students on a graded assessment containing a planted AI hallucination, only 20% caught it. The students who did were the ones with the strongest existing academic skills, not the ones who used AI more cautiously.

Read that twice, because it's the part that should actually worry you. The people most exposed to bad information are the same people least equipped to catch it. I only caught this week's two bad stats because I already knew enough about how these studies get cited to smell something off. If it had been a topic I knew nothing about, I'd have published them and never known. That's not a comfortable thing to admit in an article about verification, but it's the honest one.

The freelancer using an AI-generated contract template doesn't know the clause is fictional. The student doesn't know the citation doesn't exist. The small business owner doesn't know the tax guidance is wrong. The patient doesn't know the diagnosis explanation is confidently false.

This is precisely backwards from how protection is supposed to work. Usually expertise is a luxury good. Right now it's functioning as a safety net, and the people without it are operating with none.

This part isn't a conspiracy, just an incentive structure doing exactly what it was built to do. Engagement systems want speed, confidence, and resolution. Verification is slow. Caveats kill momentum. Uncertainty doesn't perform well in a feed.

AI search results feel psychologically efficient because they hand you a clean answer instead of five contradictory sources and a shrug. But reality usually is five contradictory sources and a shrug, disputed evidence, incomplete data, honest "we don't know yet." A synthesized answer that compresses all of that into one tidy paragraph isn't more accurate. It's just better-dressed.

The long-term risk researchers are flagging isn't really "people believe one wrong thing." It's that people stop tracing sources at all, because the interface feels trustworthy enough that checking starts to feel paranoid instead of normal. When that happens, authority quietly shifts from "the evidence supports this" to "the chatbot said it nicely."

Across freelance forums, research groups, and online communities, a pattern has emerged that nobody centrally designed:

When the evidence is genuinely incomplete, leave it uncertain instead of forcing a clean answer

Nobody calls it a search workflow anymore. It's an editorial workflow, the exact one newsrooms have run for a century, except now millions of people with zero journalism training are reinventing it from scratch, one suspicious paragraph at a time, because the tools gave them no other option.

Technical fixes are underway: retrieval-grounded models, citation-tracing tools, automated factuality checkers. They'll help. They won't finish the job. Several researchers argue fabrication is structurally baked into how these systems generate language at all, which means the fix was never going to be "wait for the next model update."

The longer-term answer looks more like what every prior information crisis produced: institutions building deliberate verification layers around an imperfect system. Peer review. Editorial standards. Citation norms. Legal evidentiary rules. None of those exist because someone trusted the raw information supply. They exist because nobody did, and they built scaffolding around that distrust instead of waiting for it to resolve itself.

We're at the start of rebuilding that scaffolding for a synthetic information environment. Some of it will be institutional, verification desks inside newsrooms, citation audits inside academic publishing, security review inside engineering teams. Some of it is going to be personal, six habits you run on autopilot every time something online sounds a little too confident.

Previous generations learned how to search. This one is learning how to doubt productively, which is not the same as doubting reflexively. Productive doubt asks specific questions and stops once it has real answers. It doesn't spiral into "nothing is true." It just refuses to accept "this sounds right" as a finished thought.

Written out, the whole emerging skill sounds insultingly simple. It's exactly why most people still aren't doing it, and exactly why I still cut two stats this week instead of just believing what I read.

Credibility used to be ambient, assumed, background, free. It's becoming infrastructure now, something you build, maintain, and occasionally repair by hand, badly, on no schedule anyone applauds.

— The Profreelance Crew

Sources: Jaźwińska & Chandrasekar, "AI Search Has a Citation Problem," Tow Center for Digital Journalism, Columbia Journalism Review (March 2025); Deck, "AI search engines fail to produce accurate citations in over 60% of tests, according to new Tow Center study," Nieman Journalism Lab (March 2025); European Broadcasting Union & BBC, "News Integrity in AI Assistants" study (October 2025); Omar et al., "Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support," Communications Medicine 5, 330, Icahn School of Medicine at Mount Sinai (2025); "MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models" (2024); "Distinguishing Fact from Fiction: Student Traits, Attitudes, and AI Hallucination Detection in Business School Assessment," UK business school study (2025).

P.S. This week on the Profreelance website

A Year of Showing Up Badly Until It Became Something Else

A year ago I started writing this newsletter mostly to prove I could do something and be consistent. Turns out that's a low bar and also, somehow, the hardest one I've cleared in a while.

Nobody warns you what this feels like when there was no plan for any of this.

This year has brought fewer headlines borrowed from somewhere braver, and a byline that stopped performing and started just... being mine.

I went back through the archive to see what actually held up. More than I expected, less than I'd like to admit.

[Read the full piece →]