Netlas vs Shodan: Platforms Comparison

IoT search engines are essential tools for any red team specialist. With several solutions available on the market, each offering its own set of pros and cons. This article will review and compare two of them: Netlas and Shodan.

Bellow you’ll find a technical, in-depth analysis of the capabilities offered by both search engines. It evaluates them across three key parameters: capabilities, volume and relevance of data, and enterprise features. The goal is to assist specialists in making an informed choice while highlighting the strengths and weaknesses of each platform.

Capabilities

Let’s begin with a brief overview of the capabilities of both search engines. It’s important to note that this article will not cover query syntax or other technical details; you can find that information in the documentation for Netlas and Shodan. Instead, we will focus on how the applications differ and the specific purposes for which they are best suited.

Search Engine

The primary function of both of these tools is to search for internet-connected devices. In Netlas, this is accomplished using the Responses Search Tool, while in Shodan, the Shodan Search Engine is employed.

The following images will illustrate these tools in action.

Using these tools, you can analyze the responses from hundreds of millions of servers to requests made by Netlas and Shodan crawlers. This capability allows users to gain insights into the configuration and vulnerabilities of various devices connected to the Internet.

External Attack Surface Management

In addition to searching for IoT elements, each of the search engines provides tools for implementing external attack surface management.

For Netlas, these tools include the Attack Surface Discovery Tool and the Private Scanner. While you can find more detailed information about these tools in our article “Complete Guide on Attack Surface Discovery”, I will provide a brief description here.

The Attack Surface Discovery Tool is designed to simplify the construction of an attack surface. It visually represents all objects and their connections on an intuitive graph, making it accessible for users with minimal knowledge of information security or computer networks. To add new nodes, you simply click on an existing one, which provides a list of possible searches; you can then select one to include related objects in the graph based on your search criteria.

Additionally, nodes can be grouped, allowing users to search for multiple objects of the same type simultaneously. You can also exclude nodes, which means they can be hidden without impacting the overall construction of the graph.

The following image illustrates an example of a graph created using the Attack Surface Discovery Tool.

The Private Scanner enables users to obtain the most up-to-date information on their chosen hosts by leveraging the capabilities of Netlas. Once a scan is initiated, the targets from the specified surface are scanned across more than 1,200 ports, and all collected data is organized into a separate private index. Users can search this index at any time, just as they would in the Public Data.

One of the key advantages of the Private Scanner is the ability to launch a scan directly from the Discovery Tool with a single click. This feature allows users to scan the surface immediately after completing its construction.

The images below showcase the interface of the Private Scanner.

In contrast, Shodan provides tools for attack surface management, including Scanner and Monitor. Shodan’s Scanner allows users to perform on-demand scans of IP addresses or networks, while Shodan Monitor enables continuous monitoring of specified IP ranges or domains, providing real-time alerts for any changes or new vulnerabilities detected.

The Shodan Scanner operates similarly to the Private Scanner in Netlas. Users can enter multiple IP addresses in CIDR format, after which the specified targets will be scanned. Unfortunately, the input is restricted to IPs and ranges, meaning that users cannot scan domain names directly.

Shodan Monitor, on the other hand, is a dashboard that provides an overview of key indicators related to your attack surface, such as open ports, potential vulnerabilities, and other relevant metrics. This tool allows users to keep track of their monitored assets and receive updates on any changes or new threats that may arise.

Similar to Netlas Private Scanner, Shodan Monitor allows users to search their own separate index, which consists of scan responses. This functionality enables users to easily access and analyze the results of their scans, facilitating better management of their attack surface and the identification of potential vulnerabilities.

CLI and API

Both search engines provide the capability to perform searches via the command line or API. To install the corresponding tools, you can use the following commands:

pip install netlas

pip install shodan

After entering the appropriate API keys, you can utilize the tools without needing to open a browser, as demonstrated in the following images.

You can also harness the power of these search engines within your Python scripts. For guidance on how to implement this, be sure to read the relevant documentation for Netlas and Shodan.

Other Ways to Use

In addition to the web version and API, both search engines provide various usage methods. While there are many integrations available, two of the most valuable methods worth highlighting are:

1. Browser Plugins: Both engines offer plugins that can be installed in Google Chrome ( Netlas, Shodan) and Mozilla Firefox ( Netlas, Shodan). While their functionality is somewhat limited compared to the full capabilities of the search engines, these plugins allow you to quickly check whether the site you are visiting is safe while browsing.
2. Modules for Maltego: Maltego is a leading investigative tool that integrates data from hundreds of sources, including those discussed in this article. By connecting Shodan or Netlas to Maltego, you gain access to their data in the form of transformations, allowing for in-depth analysis and visualization of relationships and connections within your investigations.
3. Data Files and Streaming API. For users who prioritize complete anonymity, both search engines offer ways to access data in-house. In the case of Netlas, this is achieved through the Datastore and Streaming API, based on Redis Streaming. The Datastore option allows users to purchase datasets — CSV or JSON files containing specific types of data, such as the most popular databases or all scanned WHOIS records. A full list of available datasets is provided on the relevant page. Notably, all datasets are free for subscriptions above the Corporate level. The Streaming API, available as a paid feature for Enterprise subscriptions, allows users to receive live, real-time data for immediate use. Shodan offers similar features, including the Firehose API and Bulk Data Files (BDF). The Firehose API functions similarly to Netlas’ Streaming API, providing live data streams. The BDF feature provides data received daily. These features available to users with an Enterprise subscription.

Overall, you can observe that the capabilities of both search engines are quite similar. Next we will discuss some minor features.

Minor Capabilities

This short section will include features that were difficult to attribute to a specific area. These are small things that make it easier to use or make the user experience more enjoyable.

Netlas Tools

Let’s start with additional Netlas tools. In addition to the Responses Tool, Discovery Tool, and Private Scanner mentioned above, these include the following:

• IP/Domain Info. This tool allows the user to obtain brief information about a domain name or IP address, including open ports, reputation, and basic fields from the WHOIS protocol.

• DNS Search. The main task of this tool is to provide users with information about DNS records. Supports both Forward DNS Search and Reverse DNS.

• IP WHOIS Search. Provides search by WHOIS protocol fields. Netlas collects WHOIS data for every existing IP address, which can be very valuable in investigations or Surface builds.

• Domain WHOIS Search. A tool similar to the previous one. The main difference is that it contains WHOIS protocol data for domain names.

• Certificates Search. Using this tool, the user can search the fields of SSL certificates. Very often, this allows you to detect certain devices and services if they have specific certificates.

Netlas Team Access

Another interesting point in working with Netlas is the use of team access. After joining a team, users can see private scans and ASD graphs that the administrator shared with them. An approximate view of such access will be presented in the following image.

This setup significantly streamlines group work, as any team member can access the shared results. Without it, users would need to send surfaces as documents, which could slow down the workflow considerably.

Shodan Minor Features

Shodan also offers some small but useful features that may surprise you.

One such feature is the additional tools available under its umbrella. These tools are not directly related to the search engine itself but complement its functionality. Here is a short list of main tools:

• InternetDB API. A standalone tool, free for non-commercial use, that provides a brief summary of IP addresses, including open ports, vulnerabilities, and tags. It’s essentially a stripped-down version of the main search engine, offering only basic data without additional details.
• nrich. CLI tool, the functionality of which is similar to the previous tool.
• GeoNet tools. Another useful feature of Shodan allows users to evaluate the performance of networks on servers located in different regions around the world. This helps assess global connectivity and server responsiveness.

In addition, Shodan offers very tiny improvements, such as, for example, changing the search engine theme if you are tired of the standard light and dark ones.

Features Comparsion Summary

Finally, you can see a quick summary of the various features in the following table:

Technical Differences

Before comparing the volume and relevance of data, it’s important to highlight the different scanning methods used by Netlas and Shodan. In this section, we will examine their distinct approaches to scanning, data labeling, and the number of ports scanned by each search engine.

Scan Differences

Netlas first assumes the presence of a corresponding protocol for the scanned port and then verifies if the service matches the expected protocol. If it doesn’t, the service is recorded as RAW; otherwise, it is recorded as the initially detected protocol.

For Shodan, we were unable to find detailed information on the exact scanning process. Based on our understanding, we can assume that Shodan first detects the service and then scans it, assigning the RAW designation only when no known protocol is detected. Additionally, due to the lack of protocol filters in the search, we couldn’t accurately estimate the percentage of RAW data.

Shodan may show slightly more accurate results when searching by protocols. However, the RAW data in Netlas is minimal — only a fraction of a percent — so this difference is unlikely to have a significant impact except in exceptional cases.

Number of Scanning Ports

Next, it’s important to note that the number of ports scanned by both search engines is limited.

When scanning for Public Data, Netlas scanners examine no more than 146 ports per IP address, consisting of 141 TCP ports and 5 UDP ports. A detailed list can be found in the relevant section of the documentation. While this may appear limited, the issue can be partly resolved by using the Private Scanner, which increases the number of scanned ports to ~1300.

At the same time, Shodan continuously scans 1237 ports, and the full list can be accessed via the corresponding API request. However, it’s important to note that the frequency of port scanning varies. For instance, a commonly used port like 80 might be checked daily, while less popular ports, such as 9203, may only be scanned once a month. More details on this can be found in the next subsection.

Scanning Timing

As mentioned earlier, Shodan does not scan all ports with the same frequency. During our research, we observed cases where some ports on a single IP address were scanned just a day ago, while others were scanned several weeks ago. This is due to Shodan’s crawlers prioritizing more frequently used ports.

In contrast, Netlas crawlers treat all ports equally important. Therefore, the date of scanning an IP address is also the scan date for each of its open ports, regardless of their popularity.

These differences can potentially mislead users when comparing the relevance of data from both systems. If the focus is solely on the date of the last IP address scan, Shodan might appear to have an advantage. However, as demonstrated in the experiment described in the next section, the freshness of the data is generally comparable if more attention is paid to the ports themselves.

Volume and Relevance of Data

Despite the numerous capabilities of Netlas and Shodan, both search engines are primarily utilized for searching responses. In this section, we will explore how the data returned by each search engine differs.

While it would have been possible to conduct a comparison using the request *, we determined that this approach wouldn’t provide enough relevance. Instead, we performed four groups of tests: focusing on 5 popular ports, 5 popular web servers, 5 popular databases, and 10 recent vulnerabilities. All tests were conducted using data from both search engines that were current at the time of writing.

Volume Tests

So, let’s begin with the analysis of ports. The test results are presented in the table below.

Overall, we can see that the search engines perform comparably in this aspect. Netlas excels in searching for web applications, while it lags behind in the detection of SSH and CWMP protocols. This disparity can be attributed to their differing approaches to searching, as well as the fact that Netlas scans both domain names and IP addresses, whereas Shodan focuses specifically on IPs and ports.

The next comparison focuses on popular web servers.

It is evident that in this case, the advantage lies with Netlas. These results align well with the findings from the previous tests, where we examined ports; specifically, more instances of ports 80 and 443 correlate with a greater number of web servers.

Next, the third type of software we analyzed was Database Management Systems (DBMS).

In this category, Shodan holds the advantage, with Netlas only managing to gain a significant edge in the case of Elastic, which is explained by the presence of a popular web interface for this database.

Finally, the last comparison focuses on vulnerabilities, specifically searching for potentially vulnerable products. To conduct this analysis, we selected ten relatively recent CVEs and developed search queries for both Netlas and Shodan. The results of these searches are shown in the table below.

Here, we see a close comparison, with only a few cases where one search engine clearly outperforms the other. Overall, Netlas emerged as the winner, which can be attributed to its strong performance in identifying web interfaces for vulnerable services, which were the primary focus of the search.

Overall, it is clear that the two search engines maintain a cautious parity in terms of data volume and coverage of various applications. While Netlas excels significantly in aspects related to HTTP/HTTPS protocols, it demonstrates comparable or lower results in other areas. This discrepancy is somewhat mitigated by the presence of the Private Scanner, which enables users to gather data that may have been overlooked during the initial scan.

Ultimately, the choice between Netlas and Shodan should be guided by the specific goals of the work: whether the focus is on studying vulnerabilities in websites or in more specialized software.

Data Relevance

When it comes to data relevance, both search engines offer similar performance.

According to the Shodan Help Center, the standard search provides data that is no older than 30 days. In comparison, Netlas crawlers typically go through the entire Internet every 3 weeks. You can confirm this by reviewing the crawl periods on the About page.

However, this information only reflects the range of the response ages. To verify the accuracy of this, we generated 50 random IP addresses, with data available in both Netlas and Shodan. The average age of records for these addresses is shown in the little table:

The results indicate that, based on this small test, the average relevance of Shodan’s data is slightly better than that of Netlas. However, it’s important to keep in mind the restrictions outlined in the Technical Differences section. In our study, we focused only on ports that matched in both search engines, but alternative approaches could be employed, such as considering the scan date for IP only or the timing of the first ports scanned.

Engines Limitations

After examining the capabilities of both search engines, it’s important to address their limitations. Not all features are accessible to every user; some are restricted to paid subscriptions. In this section, we will present the subscription plan tables for Netlas and Shodan, along with a summary table indicating the required plan for various features.

Netlas Plans

Shodan Plans

It is evident that Netlas subscriptions are more affordable, while Shodan offers a one-time payment plan. However, pricing alone doesn’t convey the full picture, so it is important to provide a table outlining the features associated with each subscription.

Features

Additionally, it’s important to consider the search restrictions associated with each subscription. We won’t go into detail, as you can find this information on the respective pages for Netlas and Shodan, but here’s a brief overview.

Shodan Membership is a decent option for trying out the search engine, but it has its limitations. Users are restricted to 100 searches per month and 100 scan coins per month.

The higher-tier subscriptions are generally comparable in terms of search restrictions. For instance, the Freelancer plans on both search engines allow for up to a million results per month, 5,120 scan coins, and access to 20 pages of results in the web version of the application. We won’t cover all subscription options; instead, we’ll focus on the features available at the Enterprise level. More details will follow in the next point.

Ease of Use

This sub-item also touches on some of the limitations of the search engines. While it is subjective and won’t impact the results of our study, it’s important to mention.

As one of the oldest products of its kind, Shodan has a high entry threshold for users. Among its drawbacks, we would highlight the following points.

• Queries Creation. Most queries in Shodan are simply IP addresses, products, or other terms entered into the search bar. This works fine for basic searches, but when you need to create more complex patterns, it becomes limiting. While Netlas offers flexible query customization and allows you to search by any field of any detected protocol, Shodan provides very limited filters. Only specific protocols such as HTTP, NTP, Telnet, SSL, SNMP, and SSH have dedicated filters. For other protocols, users must rely on entering the necessary strings into the search, hoping to avoid false positives. Another drawback of the Shodan Search Engine is the lack of autocomplete functionality in the search bar. In contrast, Netlas offers suggestions in the Responses Tool, making searches faster and more efficient. Here is an example of how the suggestions look in Netlas:

And here’s how it looks like in Shodan:

Here, autocomplete only offers previously performed searches. The lack of the ability to create a query on the go, combined with the need to refer to documentation, raises the entry threshold and reduces the overall effectiveness of using the tool.

• Documentation. Another issue encountered while writing this article was Shodan’s documentation. Although the Help Center contains a wealth of information, it primarily focuses on using the API, CLI, and describing integrations. It also covers several use cases and provides a brief guide on creating them. However, finding detailed information about certain tools, like the Firehose API, or guides on creating more complex queries, was not intuitive. Additionally, the available data was insufficient, which meant much of the learning had to be done through practical experience during the writing process. In our opinion, this also raises the entry threshold for new users.

Enterprise Features

Let’s examine the two highest subscription tiers. These levels are designed for large organizations, and both Netlas and Shodan provide approximately comparable features at this tier. This comparison will be most effectively illustrated in a two pivot tables. We’ll start with Corporate tier plans.

Next, we’re going to Enterpise tier.

The tables show that Netlas holds certain advantages even within its most expensive subscription. At the Corporate level, it offers access for multiple users, search by hostnames (as in Community Plan…), and a more favorable pricing structure compared to Shodan.

* - There is a distinction between the number of users and the number of API keys. Multiple users can share a single key, but they will all be subject to the limits of the subscription plan and a maximum of one request per second. To overcome these limitations, increasing the number of keys is necessary. When purchasing the Netlas’s Corporate plan, a company effectively acquires five separate licenses, allowing for greater flexibility and higher request volumes.

Summary

In summary, the choice between these search engines largely depends on the user’s goals. If the focus is on researching websites and web services, Netlas is undoubtedly the better option. However, for those interested in more obscure applications, Shodan may provide superior results.

One of the main drawbacks of Netlas is the current absence of a monitoring feature, while Shodan restricts many functions (such as searching by domain names or DNS records) to higher subscription tiers.

Overall, Netlas is likely the best choice for small companies, offering lower costs with a comparable amount of data, making it ideal for startups with limited budgets. However, for large companies, using both tools may be the most effective solution. While it may not be inexpensive, combining Netlas and Shodan ensures access to the most comprehensive information, meeting a wide range of needs.