FAQ
Logotyp Agencji Bezpieczeństwa Wewnętrznego

Open source intelligence on the internet – categorisation and evaluation of search tools

Data publikacji: 18.12.2024

Przegląd Bezpieczeństwa Wewnętrznego, 2024, Numer 31 (16), s. 383 - 412

https://doi.org/10.4467/20801335PBW.24.030.20807

Autorzy

Daniel Mider
Uniwersytet Warszawski
, Polska
https://orcid.org/0000-0003-2223-5997 Orcid
Wszystkie publikacje autora →

Tytuły

Open source intelligence on the internet – categorisation and evaluation of search tools

Abstrakt

This article presents a comprehensive analysis and systematic review of search tools used in open source intelligence (OSINT). Three main categories of software were evaluated: systems integrated with operating system or web browser, standalone applications, and repositories of links to specialised tools. A critical evaluation of representative examples from each category was conducted, taking into account their functionality, effectiveness, and limitations. The analysis identified significant gaps in the current instrumentation and formulated postulates regarding potential directions for the development of the OSINT operator skills. The optimal development direction should focus on open source, modular tools with a low entry threshold, enabling community participation in their refinement and customisation for analysts’ needs. The findings serve as a knowledge compendium for OSINT researchers, practitioners, and enthusiasts.

Bibliografia

Pobierz bibliografię

Abramczuk K., Kąkol M., Wierzbicki A., How to Support the Lay Users Evaluations of Medical Information on the Web?, in: Human Interface and the Management of Information: Information, Design and Interaction, S. Yamamoto (ed.), Cham 2016, pp. 3–13. https://doi.org/10.1007/978-3-319-40349-6_1.

Bazzell M., Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information, Charleston 2018.

Bazzell M., OSINT Techniques: Resources For Uncovering Online Information, [n.p.] 2023.

Block L., The long history of OSINT, “Journal of Intelligence History” 2023, vol. 23, no. 2, pp. 95–109. https://doi.org/10.1080/16161262.2023.2224091.

Dorn A.W., United Nations Peacekeeping Intelligence, in: The Oxford Handbook of National Security Intelligence, L.K. Johnson (ed.), Oxford 2010, pp. 275–295.

Forge J., A Note on the Definition of “Dual Use”, “Science and Engineering Ethics” 2010, vol. 16, no. 1, pp. 111–118.

Hargittai E., Hinnant A., Digital Inequality: Differences in Young Adults’ Use of the Internet, “Communication Research”, 2008, vol. 35, no. 5, pp. 602–621. https://doi.org/10.1177/0093650208321782.

Hulnick A.S., Fixing the Spy Machine. Preparing American Intelligence for the Twenty-First Century, Westport 1999.

Lowenthal M.M., Intelligence. From Secrets to Policy, Washington 2007.

Maddrell P., Spying on Science: Western Intelligence in Divided Germany 1945-1961, Oxford 2006.

Mercado S.C., Sailing the Sea of OSINT in the Information Age, “Studies in Intelligence” 2004, vol. 48, no. 3, pp. 45–55.

Mider D., Mappa Mundi ukrytego Internetu. Próba kategoryzacji kanałów komunikacji i treści (Eng. Mappa mundi of the hidden internet. Categorising internet communication channels), “PTINT Praktyka i Teoria Informacji Naukowej i Technicznej” 2015, vol. 23, no. 1, pp. 3–16.

Mider D., Sztuka wyszukiwania w Internecie – autorski przegląd wybranych technik i narzędzi (Eng. The art of searching on the internet. Review of selected techniques and tools), “Studia Politologiczne” 2019, vol. 54, pp. 191–229.

Mider D., Garlicki J., Mincewicz W., The Internet Data Collection with the Google Hacking Tool – White, Grey or Black Open-Source Intelligence, “Internal Security Review” 2019, no. 20, pp. 280–300.

Nasheri H., Economic Espionage and Industrial Spying, Cambridge 2004.

National Research Council, Computers at Risk: Safe Computing in the Information Age, Washington 1991.

Olcott A., Open Source Intelligence in a Networked World (Continuum Intelligence Studies), New York 2012.

Open Source Intelligence Market Size, Share, Competitive Landscape and Trend Analysis Report by Source, Technique and End User: Global Opportunity Analysis and Industry Forecast, 2020-2027, Allied Market Research, May 2020.

Rosenzweig P., McNulty T.J., Shearer E., Whistleblowers, Leaks, and the Media: The First Amendment and National Security, Chicago 2013.

Schaurer F., Störger J., Guide to the Study of Intelligence. The Evolution of Open Source Intelligence (OSINT), “The Intelligencer: Journal of U.S. Intelligence Studies” 2013, no. 3, pp. 53–56.

Steele R.D., Open source intelligence, in: Handbook of Intelligence Studies, New York 2007, pp. 129–147.

Steele R.D., The Open-Source Everything Manifesto: Transparency, Truth, and Trust, Berkeley 2012.

Turaliński K., Wywiad gospodarczy i polityczny. Podręcznik dla specjalistów ds. bezpieczeństwa, detektywów i doradców gospodarczych (Eng. Economic and political intelligence. A handbook for security professionals, investigators and economic advisers), Warszawa 2015.

Tylutki K., The information of a mass destruction range – OSINT in intelligence activities, “Internal Security Review” 2018, no. 19, pp. 384–404.

Wyniki pracy wywiadu naukowo-technicznego MSW PRL 1971–1989 (Eng. The results of the operations of the scientific-technical intelligence of the Polish People’s Republic 18711989), M. Sikora (comp.), Katowice–Warszawa 2019.

Internet sources

A Consumer’s Guide to Intelligence, Office of Public Affairs CIA, 1999, https://archive.org/details/consumersguide_tenet/mode/2up [accessed: 28 VI 2024].

AFP, How Bellingcat became Russia’s ‘biggest nightmare’, France24, 7 IX 2022, https://www.france24.com/en/live-news/20220907-how-bellingcat-became-russia-s-biggest-nightmare [accessed: 28 VI 2024].

Awesome OSINT For Everything, https://github.com/Astrosp/Awesome-OSINT-For-Everything [accessed: 28 VI 2024].

Bellingcat’s Online Investigation Toolkit, https://heystacks.com/doc/612/bellingcats-online-investigation-toolkit-bitlybcat (spreadsheet) [accessed: 28 VI 2024].

BlackArch, https://blackarch.org/index.html [accessed: 28 VI 2024].

BrowserAudit, https://browseraudit.com [accessed: 28 VI 2024].

CherryTree, https://www.giuspen.com/cherrytree [accessed: 28 VI 2024].

Chertoff M., Simon T., The Impact of the Dark Web on Internet Governance and Cyber Securityhttps://www.cigionline.org/static/documents/gcig_paper_no6.pdf [accessed: 28 VI 2024].

Ciancaglini V. et al., Deep Web and Cybercrime: It’s Not All About TOR, Trend Micro, 12 XI 2014, https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/deep-web-and-cybercrime-its-not-all-about-tor [accessed: 28 VI 2024].

Colquhoun C., A Brief History of Open Source Intelligence, Bellingcat, 14 VI 2016, https://www.bellingcat.com/resources/articles/2016/07/14/a-brief-history-of-open-source-intelligence/ [accessed: 28 VI 2024].

Electronic evidence of war crimes. The role of journalists, media and social media, webinar organised by Group of Friends on the Safety of Journalists and Media Freedom in Strasbourg and the Council of Europe, 25 XI 2022, https://www.coe.int/en/web/kyiv/-/electronic-evidence-of-war-crimes-and-the-role-of-journalists-media-and-social-media [accessed: 28 VI 2024].

FOCA, https://github.com/ElevenPaths/FOCA [accessed: 28 VI 2024].

Higgins E., How Open Source Evidence was Upheld in a Human Rights Court, BellingCat, 28 III 2023, https://www.bellingcat.com/resources/2023/03/28/how-open-source-evidence-was-upheld-in-a-human-rights-court/ [accessed: 28 VI 2024].

Hunchly, https://www.hunch.ly [accessed: 28 VI 2024].

IntelTechniques, https://inteltechniques.com [accessed: 28 VI 2024].

Kali Linux, https://www.kali.org [accessed: 28 VI 2024].

Mackinnon A., Bellingcat Can Say What U.S. Intelligence Can’t, Foreign Policy, 17 XII 2020, https://foreignpolicy.com/2020/12/17/bellingcat-can-say-what-u-s-intelligence-cant/ [accessed: 28 VI 2024].

Malfrats OSINT Map, https://map.malfrats.industries [accessed: 28 VI 2024].

Maltego, https://www.maltego.com [accessed: 28 VI 2024].

Market share of leading desktop search engines worldwide from January 2015 to January 2024, Statista, 2024, https://www.statista.com/statistics/216573/worldwide-market-share-of-search-engines/ [accessed: 28 VI 2024].

Matthews O., Fact Cats. The inside story of how it got the Skripal scoop, The Spectator, 20 X 2018https://www.spectator.co.uk/article/fact-cats/ [accessed: 28 VI 2024].

MetaOSINT Chart, https://metaosint.github.io/learn-more [accessed: 28 VI 2024].

NATO Open Source Intelligence Handbook v 1.2https://archive.org/details/NATOOSINTHandbookV1.2/page/n1/mode/2up [accessed: 28 VI 2024].

Open Source Intelligence Market Size, Share, Growth, and Industry Analysis, By Type (Video Analytics, Text Analytics, Visualization Tool, Cyber Security, Web Analysis, Social Media Analysis, and Others), By Application (Private Sector, Public Sector and Other), Regional Insights, and Forecast to 2032, Business Research Insights, March 2024, https://www.businessresearchinsights.com/market-reports/open-source-intelligence-market-109546 [accessed: 28 VI 2024].

OSINT Framework, https://osintframework.com [accessed: 28 VI 2024].

OSRFramework, https://github.com/i3visio/osrframework [accessed: 28 VI 2024].

Otwarte Źródła (Eng. Open sources), https://osintframework.pl [accessed: 28 VI 2024].

Paley N., Copying is an act of love. Please copy and sharehttps://copyheart.org [accessed: 28 VI 2024].

ParrotOS Security, https://www.parrotsec.org [accessed: 28 VI 2024].

PrivacyTests, https://privacytests.org [accessed: 28 VI 2024].

Recon-ng, https://github.com/lanmaster53/recon-ng [accessed: 28 VI 2024].

Reinsel D., Grantz J., Rydning J., The Digitization of the World. From Edge to Corehttps://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf [accessed: 28 VI 2024].

Spiderfoot, https://github.com/smicallef/spiderfoot [accessed: 28 VI 2024].

Spiderfoot, https://login.hx.spiderfoot.net [accessed: 28 VI 2024].

The Ultimate OSINT Collection, https://start.me/p/DPYPMz/the-ultimate-osint-collection [accessed: 28 VI 2024].

TheHarvester, https://github.com/laramies/theHarvester [accessed: 28 VI 2024].

Trace Labs, https://www.tracelabs.org/initiatives/osint-vm [accessed: 28 VI 2024].

 Tsurugi Linux, https://tsurugi-linux.org/index.php [accessed: 28 VI 2024].

VIS, https://vis.occrp.org [accessed: 28 VI 2024].

Volume of data/information created, captured, copied, and consumed worldwide from 2010 to 2025, Statista, June 2021, https://www.statista.com/statistics/871513/worldwide-data-created/ [accessed: 28 VI 2024].

White E., Closing cases with open-source: Facilitating the use of user-generated open-source evidence in international criminal investigations through the creation of a standing investigative mechanism, Cambridge University Press, 7 IX 2023, https://www.cambridge.org/core/journals/leiden-journal-of-international-law/article/closing-cases-with-opensource-facilitating-the-use-of-usergenerated-opensource-evidence-in-international-criminal-investigations-through-the-creation-of-a-standing-investigative-mechanism/981CEFF9D5AF80B6FD0A75BE6A1A384C [accessed: 28 VI 2024].

Williams H.J., Blum I., Defining Second Generation Open Source Intelligence (OSINT) for the Defense Enterprise, RAND, 17 V 2018, https://www.rand.org/pubs/research_reports/RR1964.html [accessed: 28 VI 2024].

Case law

Case of Ukraine and the Netherlands v. Russia, 8019/16, 43800/14, 28525/20, Archive of the European Court of Human Rights, 30 XI 2022, https://hudoc.echr.coe.int/eng#{%22itemid%22:[%22001-222889%22]} [accessed: 28 VI 2024].

United States Court of Appeals, United States of America v. Bogdan Nicolescu; Radu Miclaus, https://www.opn.ca6.uscourts.gov/opinions.pdf/21a0231p-06.pdf [accessed: 28 VI 2024].

Informacje

Informacje: Przegląd Bezpieczeństwa Wewnętrznego, 2024, Numer 31 (16), s. 383 - 412

Typ artykułu: Oryginalny artykuł naukowy

Tytuły:

Angielski: Open source intelligence on the internet – categorisation and evaluation of search tools

Autorzy

https://orcid.org/0000-0003-2223-5997

Daniel Mider
Uniwersytet Warszawski
, Polska
https://orcid.org/0000-0003-2223-5997 Orcid
Wszystkie publikacje autora →

Uniwersytet Warszawski
Polska

Publikacja: 18.12.2024

Status artykułu: Otwarte __T_UNLOCK

Licencja: CC-BY-NC-SA  ikona licencji

Udział procentowy autorów:

Daniel Mider (Autor) - 100%

Informacje o autorze:

Dr hab. Daniel Mider – Adiunkt na Wydziale Nauk Politycznych i Studiów Międzynarodowych Uniwersytetu Warszawskiego. Specjalizuje się w problematyce wywiadu jawnoźródłowego w internecie, kryptoaktywów, cyberprzestępczości, socjologii internetu i socjologii przemocy politycznej.

Korekty artykułu:

-

Języki publikacji:

Angielski