Deep Web vs Dark Web vs Darknet: When we talk about the internet, we define it as a network of networks. Actually, the network of networks with capital letters is due to its size and extension. The Internet is a decentralized network of interconnected networks.
Computers, servers and all kinds of devices are connected to each other through kilometres of wiring or through wireless networks. But beyond this definition, the Internet is an amalgamation of services and content.
Some of them are difficult to access. Others, are hidden behind layers of security. And others, directly, illegal in most countries. To encompass them, concepts such as Deep Web, Dark Web and Darknet are used.
Global in scope and with millions of users, according to Statista, in January 2021 the internet population was 4.66 billion users. 59.5 percent of the population. And not everyone has the same intentions or interests.
There are individuals, companies, public bodies, security agencies, criminal groups, adults, and children… As is often said, the Internet is the digital reflection of our real world. For better and for worse.
As a consequence of what the internet is and how it is organized, there are areas of the internet that are not available to everyone. Sometimes it is simply due to private networks of companies or public administrations. Sometimes they are content or addresses that escape the indexing of search engines, our main gateway to the Internet.
So, is the Deep Web, Dark Web and Darknet the same thing? Is everything illegal or tricky in these hidden faces of the internet? How much is true and how much is an urban legend in the less public corners of the internet?
Deep Web, the Deep Internet
The Internet is made up of a series of services and content. What we know as the Web or the World Wide Web is only a part of the Internet. And the Deep Web is a part of the World Wide Web with capital letters.
We owe the concept of the Deep Web as we know it today to the American computer scientist Michael Bergman, who has directed companies such as The WebTools or BrightPlanet.
The latter is dedicated to helping companies and individuals to take advantage of the information hidden precisely in the Deep Web. In turn, Bergman is inspired by the Invisible Web concept, coined by Jill Ellsworth in 1994 and which was cited later.
Moreover, the term Deep Web already appears at the end of the 90s in various publications. But let’s get to the heart of it. What is that part of the internet with an exotic name? Basically, the Deep Web is that part of the web that is not indexed by internet search engines.
There are two ways to access the web. Typing an address in the address bar of your browser or doing a search in a search engine. That search engine will offer you links to web pages. Links that we can also find on social networks or on other pages. Links that search engines have found during their indexing processes.
This arduous task is automatically handled by bots called web crawlers or spiders. They scan pages and links from lists of known links. From those primary links, they find other addresses that they send to the search engine. And it indexes, organizes, categorizes, prioritizes or undoes them.
In practice, almost all of us go directly to Google, Bing or DuckDuckGo. Why type a link if you can find it with a search of a few seconds? What’s more, normally the result we want is on the first page of the results.
There are those who, half-jokingly, half-seriously, consider that the other results pages are also part of the Deep Web. And it is that what is not in the search engines, it is difficult for it to reach us.
Why does the Deep Web occur?
Sometimes it is something that those responsible for unindexed pages want. To do this, simply include a file called robots.txt . In principle, bots looking for new links should pay attention to that file and not include that link or page. That file indicates what part of the page is indexable or whether the entire page should be left out of indexing.
Sometimes, these are pages for internal use by companies, public administrations or individuals. That is, to access them you need credentials. Web crawlers don’t perform miracles, so if one link isn’t available or accessible, they move on to another. If an entire page is password protected, its content stays out of search engines. In this case for security.
To be clear, the Deep Web is what remains outside the reach of search engines. But it does not mean that it is illegal content. Moreover, every day you access the Deep Web without realizing it.
When you check your mail from the browser, look at your bank account, access private Facebook groups or restricted Instagram profiles, when you enter an internet forum… All this content is Deep Web and is perfectly legal.
As for the size of this part of the Internet that remains outside of search engines, Michael Bergman’s 2001 research pointed to between 400 and 550 times larger than the World Wide Web that we know. About 7,500 terabytes of information. More than 200,000 pages. And, I repeat, these data are from 2001.
Today the figures should be much higher since the private and commercial internet has developed a lot these years.
Bergman’s own study makes a ranking of some of the pages that host the most content on the Deep Web. For example, US public agencies such as NASA, the NOOA (National Office of Oceanic and Atmospheric Administration), the Library of Congress or the Census, companies such as Alexa, IBM, or Amazon…
Dark Web and Darknet, the Dark Internet
Now that we know that the Internet is made up of different services and contents, one of them the Web or World Wide Web and that this, in turn, is divided between the part that we know (sometimes called Clearnet ) and the part that we don’t, all go deeper.
There are those who use the simile of a sea or ocean in which we submerge little by little. On the surface, the Web is offered by search engines such as Google, Bing, Yahoo!, Baidu or Yandex. Below, is the Deep Web, the part that we see because we are customers, employees or users, but that not everyone knows about due to privacy or security limitations.
But there is a third content that is part of the Web and that is hidden for mostly legal or political reasons, sometimes. It is known as the Dark Web and sometimes the media confuses both concepts, Deep Web and Dark Web. In part, because the second is part of the first.
That is, it is web content that does not appear in search engines. But it differs in that you cannot access it from your browser. You need a special browser or a specific application. The two most popular are TOR (and its ONION links) and also I2P (and its I2P links). The main reason is that the content exposed on the Dark Web is usually illegal.
Another peculiarity of the Dark Web is that it is hosted on what is known as the Darknet, literal translation, dark network. These are networks that operate on the Internet but are accessible through a specific software that uses its own protocol. That is, the Web uses the HTTP and WWW protocols.
For its part, a Darknet uses alternative protocols, usually encrypted or with several layers to protect the privacy of the members of the network. This is the case of Freenet, I2P, and TOR ( The Onion Router ).
The Dark Web remains hidden but functions much like the Web we access from search engines.
In the words of Kaspersky, a computer security specialist, “The Dark Web is the hidden set of Internet sites that can only be accessed through a specialized web browser. It is used to keep internet activity private and anonymous, which can be useful in both legal and illegal applications.
While it is used by some to evade government censorship, it has also been known to be used for highly illegal activities.” Each page has its own address and can link to other pages and there are even search engines that index TOR or I2P links. DuckDuckGo, for example, has a version of the TOR network.
From activism to criminal
Interestingly, the origin of the Internet and the Dark Web have a lot in common. The Internet as such was created from the ARPANET, the first network of networks that used the TCP/IP protocol. The goal was to create a secure, decentralized communications system that could continue to function despite losing network elements in an armed conflict.
Behind the project was the United States Department of Defense, although over time researchers and scientific bodies joined.
For its part, the origin of TOR ( The Onion Router ) is also in the search for an anonymous and secure communications system. The US government was looking for a way to facilitate anonymous communication in countries run by dictatorships or oppressive regimes.
The project passed through various government hands until it was patented. This generated a parallel project, what we know as TOR, with the purpose of offering this technology for free and with an open license.
Over time, both technologies have served laudable ends. The Internet has brought millions of people, companies, and administrations closer together has made things much easier for us, and has democratized communication and the dissemination of content. However, there are those who use it for questionable purposes.
The same goes for TOR. There are activists and journalists who use it to safeguard their identity in non-democratic countries. But cybercriminals also take advantage of your anonymity.
Wikipedia lists the content that can be found on the Dark Web: Ransomware, botnets, and illegal markets ( darknet markets for drugs, medicines, weapons, extreme content, Bitcoin-related services, and infected computers). , illegal offensive content, fraud, and a long etcetera.
Come on, everything fits on the Dark Web, whether legal or illegal. Some of the most popular content is the sale of bank card numbers, databases with credentials or personal data, or the sale of payment service accounts such as Spotify, Netflix, or HBO.
Cybercrime on the Visible Internet
Fiction and the media have intentionally or inadvertently simplified what the Deep Web, Dark Web, and Darknet mean. Also giving them an image that makes them an exclusive tool for cybercriminals.
Precisely, in 2011 the Dark Web and the Darknet were made known to the general public with the closure of the Silk Road, an illegal market for drugs and medicines. But as we have seen, nothing is further from reality. The Internet and what it contains is much more complex than the light ( Clearnet ) and the dark ( Darknet ).
On the one hand, illegalities occur on the Internet without the need to go to the Darknet. The dissemination of passwords and personal content without the permission of their owners or the dissemination of payment service accounts, for example, is the order of the day. We can also find forums in which illegal content is spread or in which exchanges are made that violate various international laws.
But this happens on web pages and also on social networks or instant communication networks such as WhatsApp or Telegram. Illegal, explicit videos are even posted on portals such as Facebook or YouTube.
On the other hand, depending on the country, accessing the internet that we all see on a daily basis involves using browsers that are compatible with TOR or I2P links to bypass censorship.
Media such as the BBC, Deutsche Welle, The Guardian, ProPublica or The New York Times have an Onion version for those who cannot access the World Wide Web and are forced to the anonymity offered by the Dark Web through the Darknet.
