By Jay Heisler and Jake Robinson
Imagine you’re an overworked OSINT professional. Shouldn’t be too hard, since you’re a reader of this blog! You have five human trafficking cases sitting in your inbox, and you need to figure out who the bad guys are as fast as you can. It’s literally a life-or-death situation. Fortunately, your favorite LLM is just a chat window away, and you feed it the information you know on one case and let it search while you load up your favorite link analysis tool to start on case #2 in another browser window.
A few minutes later, your chat bot has served up a nice dossier for your first suspect and it sounds great. You spot check a few of the sources and they look good to go. You slap your findings in an email and send it out to law enforcement. Thirty minutes later, you get an email – the physical address you just claimed was used by the trafficking ring doesn’t exist and never did. Your reputation is sullied and your reports are no longer trusted. Just be glad an innocent person wasn’t arrested! A couple of years ago, you would have checked every fact and reviewed every figure. While AI has helped you triple your output, you’ve lost something along the way. You’re not alone.
Overall, literacy rates were declining even before the pandemic. The percentage of US adults scoring at or below the Level 1 literacy level has increased from 19% in 2017 to 28% in 2023. Adults at Level 1 struggle with complex texts and multi-step instructions. The decline has been both deep and broad. Most OECD countries reported a drop in literacy scores measured by the Survey of Adult skills going back even farther, between 2013 and 2023. The US showed a 12.4 point decline, and Canada showed a 0.5 point decline. Only six OECD countries showed an increase, with Finland topping the list at +15.2 points.
Beyond the declining numbers of students who are capable of reading, broad anecdotal evidence points to a trend where fewer and fewer college students are willing to read. A 2024 article in The Atlantic noted one college professor’s observation that “students have become overwhelmed by the reading… (they) now seem bewildered by the thought of finishing multiple books a semester.” The article also claims that middle- and high-school students are encountering fewer books in classrooms.
So what does any of this have to do with OSINT? “Declining rates of literacy go hand in hand with a decrease in critical thinking,” said Canadian OSINT expert Jordan Saikaley. He says the rise of investigators using AI to help with investigations has exacerbated the problem. “The more illiterate people rely on AI-generated answers and take the information it provides… with no fact checking, the more truth is lost,” he said. “That would have a terrible impact on the outcome of legal trials, if AI generated evidence was used with no verification.”
Readers of this blog know that OSINT tradecraft requires slow, methodical, and in-depth reading and understanding of oceans of information and a keen ability to separate fact from falsehood. This intuition is backed by science: in their paper “Looking at Literacy in a Post-Truth World”, a group of Belgian researchers cited data linking critical thinking with literacy, and data that showed that more than 90% of 15-year-old students are unable to critically reflect on written information.
Furthermore, there’s early evidence that the use of LLMs to assist writing and research very likely leads to a decrease in cognitive engagement and brain connectivity when recalling and evaluating information that a person writes. This is an alarming finding in a field that is increasingly reliant on LLMs yet absolutely dependent on information being correct and judgments being defensible. “Any time an analyst uses bad information in their reports, that significantly impacts not only their reputation, but the reputation of their organization if that information gets out,” said Jenna Dolecek, an OSINT investigator for Justice Rapid Response. “I was already seeing fatigue in the field before AI, but now it has gotten worse because analysts can turn their brains off and just let AI handle it.. I really do fear for the future of the field… I understand the burnout… but we have to remember why we’re doing this work in the first place and for whom.”
Despite the doom and gloom, not all OSINT practitioners view the rise of LLMs and the fall of information literacy as a foregone conclusion. “I use AI a lot, for a range of different tasks,” said Aaron Roberts, co-founder of Webamon and UK OSINT Community executive committee member. “I think if you have the appropriate training and you understand the technology, I think you’ll see the divide between people who are sharing something hallucinated, and people who understand what constitutes intelligence… Ultimately,” he said, “you need human-driven analysis and the broader context you have as a human to understand if something looks legit or if something looks suspicious.”
So what does this mean for the future of OSINT? After all, how do you reverse the downward trend in deep reading that likely has just as much to do with the rise of 24-hour news and social media as with the more recent AI phenomenon? Maybe it really is a training problem, as Roberts says. “The AI skills gap is really a ‘critical thinking’ gap” claims the title of a 2025 Fortune article. That article cites quotes from several executives emphasizing the critical need to invest in companies’ human capital, “The required skill set is shifting from whether workers can do certain tasks to thinking more strategically.” “Knowledge is sort of now free in some ways. Thinking now has to really kick in,” according to executive Mark Beasley.
Given the new cultural landscape, the challenge is how to train new OSINT practitioners in complex skills that they are no longer likely to obtain through deep reading of, and critical thinking about, a large number of books. The skills gained through reading have never been more critical, and yet we have set ourselves up to be uniquely unprepared to gain them.
The next hallucinated address might not cost just a wasted email or a diminished reputation. The costs could be much greater. Even though AI can write a dossier in a few minutes, it still doesn’t always get it right. And even as AI gets better at getting it right, human judgment and critical thinking will still be absolutely necessary to make sure analyses are relevant, accurate, and human.
Bios:
Jake Robinson is a former CIA Targeting Officer, US Army Intelligence Analyst, and is currently the Principal Product Manager at Clearview AI.
Jay Heisler is a journalist who teaches part time at a university in Nova Scotia, in Canada. His journalism has been published in CNBC, Voice of America and Small Wars Journal, plus in daily Canadian newspapers.
Leave a Reply