Shulou(Shulou.com)05/31 Report--

The editor will share with you what OnionSearch is. I hope you will get something after reading this article. Let's discuss it together.

OnionSearch

OnionSearch is a URL search script for onion domain names. The tool is based on Python 3 and can help researchers crawl URL addresses in different .onion search engines.

Tool requirements

Python 3

Currently supported search engines

Ahmia

Darksearchio

Onionland

Notevil

Darksearchenginer

Phobos

Onionsearchserver

Torgle

Onionsearchengine

Tordex

Tor66

Tormax

Haystack

Multivac

Evosearch

Deeplink

Tool installation PyPI installation: pip3 install onionsearchGitHub installation: git clone https://github.com/megadose/OnionSearch.gitcd OnionSearch/python3 setup.py install tools use usage: onionsearch [- h] [--proxy PROXY] [--output OUTPUT] [--continuous_write CONTINUOUS_WRITE] [--limit LIMIT] [--engines [ENGINES [ENGINES.] [ -- exclude [EXCLUDE [EXCLUDE...]] [--fields [FIELDS [FIELDS...] [--field_delimiter FIELD_DELIMITER] [--mp_units MP_UNITS] search positional arguments: search The search string or phrase optional arguments:-h -- help show this help message and exit-- proxy PROXY Set Tor proxy (default: 127.0.0.1)-- output OUTPUT Output File (default: output_$SEARCH_$DATE.txt) Where $SEARCH is replaced by the first chars of the search string and $DATE is replaced by the datetime-continuous_write CONTINUOUS_WRITE Write progressively to output file (default: False)-limit LIMIT Set a max number of pages per engine to load-engines [ENGINES [ENGINES...]] Engines to request (default: full list)-- exclude [EXCLUDE [EXCLUDE...]] Engines to exclude (default: none)-- fields [FIELDS [FIELDS...]] Fields to output to csv file (default: engine name link), available fields are shown below-- field_delimiter FIELD_DELIMITER Delimiter for the CSV fields-- mp_units MP_UNITS Number of processing units (default: core number minus 1) [...] Multiprocessing behavior

By default, the script will be run using the "mp_units = cpu_count ()-1" parameter. This means that if our device CPU has four cores, it will run three crawlers at the same time. We can set the value of the "mp_units" parameter at will, but it is recommended to use the default value.

Sample use of tools

Request the query "computer" from all search engines:

Onionsearch "computer"

Request the query "computer" from all search engines, but exclude "Ahmia" and "Candle":

Onionsearch "computer"-- exclude ahmia candle

Request "computer" from all search engines, including "Tor66", "DeepLink" and "Phobos" at the same time.

Onionsearch "computer"-- engines tor66 deeplink phobos

The content is the same as the above query, but only three pages per search engine:

Onionsearch "computer"-- engines tor66 deeplink phobos-- limit 3 output result default output

"by default, search results are stored in CSV format, which contains the following data:"

"engine", "name of the link", "url" custom output columns

We can also use the "--fields" and "--field_delimiter" parameters to specify data items in the output file:

"--fields" can help us add, remove, and reorder output items:

"engine", "name of the link", "url", "domain"

Or:

Demonstration of "engine" and "domain" tools

After reading this article, I believe you have a certain understanding of "what is OnionSearch". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.