Open Source Intelligence Analysis — What Businesses Need to Know About OSINT
Open source intelligence is one of the best ways to stay informed about developing threats, but it can be hard to wrangle. Learn how your business can effectively analyze OSINT to mitigate risks and maintain business continuity.
While social media platforms have developed and trained algorithms to provide a personalized feed of information tailored to our unique interests, corporate security and safety leaders are often left to their own devices trying to make sense of this ever-growing world of unstructured data. Monitoring and analyzing social conversations and content from countless other data sources is of critical concern as organizations attempt to quickly identify pertinent information that poses a material risk to their business or people. Sara Pratley, Alertmedia’s SVP of Global Intelligence, puts it succinctly:
Nearly everyone who has faced making a critical decision with incomplete information has longed for clearer or better data. At its foundation, open source intelligence is meant to provide decision-makers with more complete, timely, and actionable data—connecting dots and for a more accurate picture. But this breadth of intelligence is also rife with misinformation, demanding rigorous analysis by experts.
In this post, we’ll cover what OSINT is, why it has grown in popularity, and what businesses need to know about leveraging it as part of their risk monitoring and employee safety strategies.
Download The Business Case for Human-Vetted Intelligence
What Is Open Source Intelligence?
Open source intelligence, or OSINT, refers to information derived from sources available to the general public. Here’s how the U.S. Department of State defines it:
“Open source intelligence (OSINT) is intelligence that is produced from publicly available information and is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement.”
The term “open source” is used broadly to refer to public information that can be accessed and used freely without permission or special accommodations. This includes both public sources— popular websites, social platforms, and forums—as well as less discoverable sources of information, such as publicly accessible files and documents, webpages hidden behind a login/paywall, and more.
With 5.25 billion people using the internet—roughly 66% of the world’s population—the volume of information published daily is almost beyond comprehension. On average, individuals generate approximately 1.7MB of new data every second, and researchers estimate that the vast majority of the world’s data was created in the past two years alone. Consider the following:
This wealth of information also represents an invaluable source of insight for governments, intelligence agencies, and private businesses looking for a competitive advantage or to keep tabs on what’s happening in the world around them. While sifting through billions of data sources to find a handful of valuable and relevant pieces of information may sound like looking for a needle in a haystack, OSINT researchers and analysts have developed various tools and methods to help discover and take advantage of the information available to them.
Open source intelligence tools and analysis
While it’s easy to think about OSINT as the words or pictures seen on a website or in a social media post, its scope is far greater. Created in 2016, the OSINT Framework provides a helpful overview of the breadth and depth available to researchers and analysts, ranging from usernames, phone numbers, and email addresses to transportation records, public registries, and the global terrorism database.
Some of the available sources OSINT researchers use for gathering information include
Consumer search engines (e.g., Google, Bing, Yahoo, etc.) are invaluable tools for OSINT researchers. However, specialty search engines like Shodan are also used to find obscured data from various “Internet of Things” (IoT) devices—like webcams, smart TVs, license plate readers, and more—to aid cybersecurity investigations and other research. TweetDeck is also a popular OSINT tool used to curate real-time social content based on keyword, content type, or location. Tools like Spiderfoot and Maltego help OSINT investigators and information security (infosec) teams by automating data collection, creating graphical links, and visualizing the relationship between individuals and topics of interest.
Passive vs. active OSINT
Given the sheer volume of information publicly available on the web, there’s no shortage of OSINT available to researchers and analysts. However, there are limits to what researchers can access without taking some proactive steps—whether it be creating an account to access specific social media platforms or being invited to specific moderated communities, such as invite-only groups, messaging services, subreddits, or message boards. Depending on the techniques used to collect the information, it is considered either passive or active OSINT.
History of OSINT
There is some debate about the origins of open source intelligence, but the concept of monitoring and leveraging publicly available information sources for intelligence gathering and national security purposes dates back to at least the 1930s. At the time, the British government asked the British Broadcast Corporation (BBC) to launch a new service that would capture and analyze print journalism from around the world. Originally referred to as its Digest of Foreign Broadcast, the service eventually became known as BBC Monitoring, which still exists today. According to a report published in the Journal of U.S. Intelligence Studies, by 1943, this organization monitored approximately 1.25 million broadcast words every day, serving as a “modern Tower of Babel” with the stated goal to “listen to the voices of friend and foe alike.”
It wasn’t until the 1980s that the U.S. military coined the term OSINT, which grew from a desire to deliver more timely, dynamic intelligence to inform decisions on the battlefield. In 1994, the CIA formally acknowledged OSINT as tradecraft by establishing the Intelligence Community Open Source Program (COSPO).
OSINT for Cybersecurity and Physical Threat Detection
While governments and intelligence agencies continue to use open source information for various purposes, its origins in the private sector can be traced back to the 1980s. Today, security researchers and teams at companies of all sizes use this intel to reduce risk and inform decision-making in the physical world and cyberspace.
Physical OSINT
Physical OSINT is what most people think of when they hear about open source intelligence. Often, imminent physical events and threats incite mass amounts of information broadcast to the public, which people can action in multiple ways.
Monitoring emergencies and developing events
When it comes to emergencies and other potentially disruptive events, a fast and effective response can be the difference between a positive or tragic outcome. OSINT can help security and business continuity leaders identify critical events as or even before they happen, providing timely information to employees in harm’s way before they’re impacted.
Perhaps the most memorable modern example of a timely, effective response was during the Arab Spring protests of 2010 and 2011.
The protests initially started as small, isolated pockets of unrest that became widely publicized. As governments cracked down on traditional media and public gatherings, protestors turned to social media, which boosted their visibility even more. This had the paradoxical effect of invigorating political movements in other countries, resulting in several regime changes like in Egypt and Tunisia.
At the same time, businesses in those areas that previously relied on government support to protect their employees were suddenly left in the lurch as the governments concerned themselves with the protests. Luckily, savvy companies used those same social media platforms to keep abreast of developments and get their employees out of dangerous areas before tensions erupted further.
Investigating security incidents
Corporate security teams commonly use OSINT when investigating or responding to reported security incidents, including active shooter situations, suspicious activity, bomb threats, and more. Once the situation is reported, security teams may use OSINT to verify critical details, such as the location or time an incident occurred, or to provide the most up-to-date information about rapidly developing events on the ground.
Cybersecurity OSINT
Hackers and other digital bad actors are closely connected to OSINT, for better and for worse. As cyberattacks have become more widespread and well-known, the impetus for hacking victims to publicly share details about the attack is much stronger. From companies like Sony and Colonial Pipeline to government agencies, no organization is immune to unauthorized digital intrusion.
By collaborating with other legitimate organizations and governments to share information on these attacks, organizations and individuals contribute to the overall OSINT ecosystem and have the potential to prevent the unnecessary spread of computer viruses or other malware.
OSINT can also flow directly from the threats themselves: Cybercriminals’ online activity can help companies quickly detect when sensitive company information is leaked. This “paydata” is often discussed or published on deep web or dark web message boards, enabling security teams to investigate cyber threats and learn about the vulnerabilities hackers may have exploited to access the information.
Unfortunately, it also works the opposite way. Most companies have a wealth of information publicly available to anyone with malicious intent. From employees’ social media profiles to media coverage, hackers are keen to exploit seemingly innocent information to inform their attacks, including social engineering attacks like phishing attempts or password hint analysis.
4 Challenges to OSINT Analysis
While open source intelligence is a well-established field with countless use cases and applications, the majority of organizations are ill-equipped to navigate the ever-changing field of OSINT on their own. There are many reasons why organizations might find themselves unsuited to this approach to risk monitoring—from navigating legal and ethical debates around OSINT data collection to the extensive human and financial resource requirements to monitor online conversations and world events effectively, 24/7/365. Here are some of the ways OSINT falls short.
1. OSINT data collection is highly time-consuming
While collecting OSINT data is far simpler now than ever before, making sense of it remains a highly specialized skill set requiring years of training. Unfortunately, due to the volume of data generated every day, OSINT analysts must sift through copious amounts of information from a wide range of sources to piece together a coherent story about what’s actually happening at any given time in a given location. Sara Pratley knows from deep experience: “It takes expertise—and sometimes numerous experts to manage the pure amount of information and perform careful vetting.”
Meanwhile, if the situation is urgent—such as in an emergency or business-critical event—the time required to gather and analyze the information may be longer than the time in which the organization can take meaningful action to alter the outcome, thus calling into question the value of the collection process itself.
2. OSINT is noisy and difficult to filter
Not only does it take time, but the need to constantly monitor, search, and filter voluminous troves of available information to identify relevant information is also tedious for even trained analysts. While AI, machine learning, and other specialized tools help researchers parse information faster, it is challenging for any sized organization to identify all critical, time-sensitive events as they happen. This is particularly true for multi-location businesses that monitor hundreds of distinct facilities or thousands of remote employees.
3. Unverified OSINT can’t always be trusted
False positives are another primary concern within the OSINT community. For example, last year, a video of a woman in Singapore rejecting the country’s COVID-19 mask mandate went viral. Shortly after, the woman in question was taken into custody by local police; however, various internet figures took it a step further, identifying her as the CEO of a digital security firm based on publicly available sources—reports that turned out to be false. Unfortunately, the misinformation resulted in her company and its employees having to endure damage to the company’s brands and personal threats before the initial reports were corrected.
4. Artificial Intelligence can mislead (but also reveal)
The recent advancements in artificial intelligence (AI) have sent reverberations throughout our daily lives, meaning open source information is highly vulnerable to manipulation. OSINT analysts rely on up-to-the-minute reports for the situations they’re monitoring, but what if someone fabricated that information—and convincingly so? The AI technology has proven itself capable of working as a high-speed disinformation assembler, intentionally and unintentionally, using text, images, and deepfakes to disseminate falsehoods. Sara Pratley explains that researchers must focus even more on corroborating information and fact-checking sources to preclude bad insights. This extra attention can slow things down significantly.
On the other hand, AI has been used to sniff out faked media successfully. At best, it seems like we’ll be witness to an AI arms race between bad actors and legitimate organizations, with OSINT analysts caught in the murky spaces between.
What to Look for in a Threat Intelligence Tool
Fortunately, there are several ways organizations can leverage the benefits of OSINT to improve situational awareness and aid decision-making without hiring a dedicated team of analysts to comb through the data. Many are turning to purpose-built threat intelligence solutions that use OSINT data while also providing a layer of curation and verification to provide businesses with a cleaner, more actionable view of what’s happening.
If your organization is considering investing in an intelligence solution, here are five things to consider when evaluating your options.
Speed
Whether you have a small security team or a large GSOC, knowing as quickly as possible about critical events that pose a risk to your employees or business is crucial. When assessing tools, ask how intelligence is delivered and how easy it is to implement so you can alert those in harm’s way. Is threat intelligence offered 24/7/365? Can it be integrated with your organization’s mass notification system to streamline response and recovery workflows? Activating your emergency response isn’t only about how quickly you have verified information to deliver; it also requires a mechanism to notify those impacted.
Depth and breadth of coverage
Your provider should offer both around-the-clock monitoring and comprehensive coverage of a wide range of threat types. What types of incidents are captured? What data sources are used? Does the provider cover all major geographies? Use this information in conjunction with a business continuity checklist to understand if there are any gaps in your preparedness efforts so you can address them before moving forward.
Noise-free, accurate information
If the phrase “drinking from a fire hose” came to mind when reading about the volume of data created daily, you’re not alone. Trying to monitor and make sense of everything posted on the web—even if you were to monitor only a fraction of available data sources—is guaranteed to leave your organization overwhelmed and with little more insight than if you had done nothing at all. How does your provider analyze and filter information? What parameters do they use to determine relevance? How does the provider differentiate between unverified and verified sources? Consider requesting a demo to see real-world threat alerts around one of your locations so you can independently assess whether the information provided is adequate.
Location-aware risk assessments
Moving from raw data to actionable intelligence is all about context. For organizations to truly understand the impact of threats, they must also understand if, how, and to what extent each threat poses a risk to their people, facilities, and assets. Be sure to ask potential threat intelligence providers how threat data integrates with location intelligence. Do they support real-time data syncing with your HRIS? How does the solution account for threats near remote employees or business travelers?
Analyst verification
We’ve stressed the danger of information overload in OSINT operations, where analysts have more on their plate than what they have time for. Thankfully, high-quality threat intel tools provide direct analyst access, where trained intelligence professionals can jump in and assist your organization’s OSINT efforts with their specialized knowledge, allowing you to scale your capabilities without expanding your internal team.
Addressing Information Overload
With a world’s worth of information available at our fingertips, learning how to parse and interpret it becomes all the more important. Using OSINT as part of a holistic approach to threat intelligence and monitoring can help your organization reduce risk and keep employees safe during emergencies and other critical events. Learn more about how AlertMedia’s Threat Intelligence helps customers monitor, analyze, and quickly communicate threats to their people. Request a demo to see for yourself.
