The fourth article in the ongoing series on FOSS security tools introduces the subdomain discovery tool called subfinder.
Subfinder is optimised for speed and is easy to use on websites. Designed to use passive online sources to maximise the results, it includes wildcard elimination modules. You can produce the results to standard output, file or JSON formats. It is written using the Go programming language and released under the MIT licence.
We will use Go version 1.22.4 installed on Parabola GNU/Linux-libre (x86-64) for the subfinder examples.
Installation
The Parabola GNU/Linux-libre (x86-64) is a free, lightweight, Libre rolling-release distribution which is based on Arch Linux.
$ cat /etc/os-release NAME=Parabola PRETTY_NAME=”Parabola GNU/Linux-libre” ID=parabola ID_LIKE=arch BUILD_ID=rolling VARIANT=”x86_64 SystemD Edition” VARIANT_ID=”x86_64-systemd” ANSI_COLOR=”1;35” HOME_URL=”https://www.parabola.nu/” DOCUMENTATION_URL=”https://wiki.parabola.nu/” SUPPORT_URL=”ircs://irc.libera.chat/#parabola” BUG_REPORT_URL=”https://labs.parabola.nu/” LOGO=”parabola-logo”
You need to first install Go on the system using the ‘pacman’ package management tool as follows:
$ sudo pacman -S go
The GOPATH and PATH environment variables need to be set in ~/.bashrc.
GOPATH=$HOME/go PATH=$PATH:$GOROOT/bin:$GOPATH/bin
You will then need to source the ~/.bashrc file for the settings to take effect:
$ source ~/.bashrc
You can verify the version of Go installed using the following command:
$ go version go version go1.22.4 linux/amd64
You can now install subfinder using the “go” executable as shown below:
$ go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest go: downloading github.com/projectdiscovery/subfinder/v2 v2.6.6 go: downloading github.com/projectdiscovery/subfinder v2.6.6+incompatible go: downloading github.com/projectdiscovery/fdmax v0.0.4 ... github.com/projectdiscovery/utils/update github.com/projectdiscovery/subfinder/v2/pkg/runner github.com/projectdiscovery/subfinder/v2/cmd/subfinder
Docker
You can also use the subfinder Docker image instead of doing a local installation in order to use the tool. For example:
$ sudo docker pull projectdiscovery/subfinder:latest
If you want to pass specific flags to the invocation with Docker, you can pass the options to the Docker command as follows:
$ sudo docker run projectdiscovery/subfinder:latest -d google.com
Version
The ‘-version’ option of subfinder prints the installed version of subfinder. As of writing this article, the latest version is v2.6.6.
$ subfinder -version [INF] Current Version: v2.6.6 [INF] Subfinder Config Directory: /home/guest/.config/subfinder
Usage
The ‘-h’ option of subfinder provides a quick summary and description of the various option flags supported by the tool.
$ subfinder -h Subfinder is a subdomain discovery tool that discovers subdomains for websites by using passive online sources. Usage: subfinder [flags] Flags: INPUT: -d, -domain string[] domains to find subdomains for -dL, -list string file containing list of domains for subdomain discovery SOURCE: -s, -sources string[] specific sources to use for discovery (-s crtsh,github). Use -ls to display all available sources. -recursive use only sources that can handle subdomains recursively (e.g. subdomain.domain.tld vs domain.tld) -all use all sources for enumeration (slow) -es, -exclude-sources string[] sources to exclude from enumeration (-es alienvault,zoomeyeapi) FILTER: -m, -match string[] subdomain or list of subdomain to match (file or comma separated) -f, -filter string[] subdomain or list of subdomain to filter (file or comma separated) ...
Sources
The ‘-ls’ or ‘-list-sources’ option provides the list of all available sources as demonstrated below:
$ subfinder -ls __ _____ __ _______ __/ /_ / __(_)___ ____/ /__ _____ / ___/ / / / __ \/ /_/ / __ \/ __ / _ \/ ___/ (__ ) /_/ / /_/ / __/ / / / / /_/ / __/ / /____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/ projectdiscovery.io [INF] Current subfinder version v2.6.6 (latest) [INF] Current list of available sources. [39] [INF] Sources marked with an * need key(s) or token(s) to work. [INF] You can modify /home/guest/.config/subfinder/provider-config.yaml to configure your keys/tokens. alienvault anubis bevigil * binaryedge * bufferover * c99 * censys * certspotter * chaos * ... waybackarchive whoisxmlapi * zoomeyeapi * facebook * builtwith *
Domain
You need to use the ‘-d’ or ‘-domain string[]’ option to specify the domain for which you want to find all the sub-domains. In the following example, we are searching for all the sub-domains for google.com.
$ subfinder -d google.com ... googleproxy-66-249-80-193.google.com googleproxy-64-233-172-24.google.com ww.google.com firebaserules.clients6.google.com googleproxy-66-102-7-180.google.com googleproxy-64-233-173-21.google.com ... [INF] Enumerating subdomains for google.com [INF] Found 5875 subdomains for google.com in 2 seconds 434 milliseconds
Verbose
You can request for a more verbose output using the ‘-v’ option of subfinder as demonstrated below:
$ subfinder -d google.com -v ... [alienvault] tuf-autopush.corp.google.com [alienvault] staging-git.corp.google.com [alienvault] docs-qa.corp.google.com ... ... [crtsh] docs.google.com [crtsh] mail.google.com [crtsh] plus.google.com ... [dnsdumpster] google-proxy-74-125-212-200.google.com [dnsdumpster] google-proxy-64-233-172-200.google.com [dnsdumpster] google-proxy-66-249-82-200.google.com [INF] Found 5875 subdomains for google.com in 2 seconds 459 milliseconds
Match
The ‘-m’ or ‘-match’ option allows you to match only a specific string in the subdomain or list of subdomains. In the following example, we are searching for the subdomains that have ‘googleproxy’ in the name.
$ subfinder -m “googleproxy-*” -d google.com ... googleproxy-66-249-82-201.google.com googleproxy-64-233-172-33.google.com googleproxy-66-102-7-221.google.com googleproxy-66-249-84-51.google.com ... googleproxy-66-249-84-91.google.com googleproxy-66-249-88-34.google.com googleproxy-64-233-172-180.google.com [INF] Found 1761 subdomains for google.com in 4 seconds 692 milliseconds
Filter
You can omit certain patterns in the output using the ‘-f’ or ‘-filter string[]’ option to exclude a subdomain or list of subdomains. For example:
$ subfinder -f “googleproxy-*” -d google.com ... mailiu0-f50.google.com ratelimited-proxy-209-85-238-74.google.com mailua0-f193.google.com mailua0-f254.google.com lens.google.com ratelimited-proxy-203-208-38-180.google.com mail-vs1-f10.google.com [INF] Found 4114 subdomains for google.com in 4 seconds 700 milliseconds
Rate limit
The ‘-rl’ or ‘-rate-limit int’ option can restrict the number of HTTP requests sent per second to the website. It accepts an integer as an argument. For example:
$ subfinder -rl 2 -d google.com ... rate-limited-proxy-74-125-150-100.google.com rate-limited-proxy-108-177-69-120.google.com r2.sn-npoeene6.c.docs.google.com mailio0-f171.google.com ... [INF] Enumerating subdomains for google.com [INF] Found 5875 subdomains for google.com in 10 seconds 343 millisecond
Concurrency
You can also specify the number of concurrent goroutines to be executed using the ‘-t int’ option. It accepts an integer argument, and the default value is 10.
$ subfinder -t 20 -d google.com ... mailqt0-f203.google.com mail-vs1-f110.google.com cag.ext.google.com ratelimited-proxy-66-249-91-10.google.com ... [INF] Enumerating subdomains for google.com [INF] Found 5875 subdomains for google.com in 11 seconds 616 milliseconds
Output
subfinder supports multiple output formats to show the results. The ‘-o’ or ‘-output string’ option writes the results to a file. For example:
$ subfinder -m “googleproxy-*” -d google.com -o /tmp/google.txt
The ‘-oJ’ or ‘-json’ flag converts each entry into a JSON structure that can be consumed by downstream applications.
$ subfinder -m “googleproxy-*” -d google.com -oJ
...
{“host”:”googleproxy-66-249-83-11.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-93-163.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-88-170.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-80-64.google.com”,”input”:”google.com”,”source”:”anubis”}
[INF] Found 1761 subdomains for google.com in 3 seconds 562 milliseconds
You can also use the ‘-oD’ or ‘-output-dir string’ option to write the output to a directory. The ‘-cs’ or ‘-collect-sources’ option includes all the sources in the results when used only with the ‘-json’ flag. The ‘-oI’ or ‘-ip’ option includes the host IP address in the output.
Statistics
The information on source statistics can be obtained using the ‘-stats’ flag. A sample execution is given below:
$ subfinder -d google.com -stats ... [INF] Found 5875 subdomains for google.com in 7 seconds 292 milliseconds [INF] Printing source statistics for google.com Source Duration Results Errors ──────────────────────────────────────────────────────── alienvault 7.291s 144 0 anubis 51ms 5197 0 builtwith 0s 0 0 crtsh 2.025s 105 0 digitorus 863ms 21 0 dnsdumpster 2.438s 1 0 hackertarget 1.31s 405 0 leakix 1.022s 2 0 virustotal 0s 0 0 ...
Configuration
The subfinder configuration is present in ~/.config/subfinder/config.yaml. The first few lines of the file are given below:
# subfinder config file # generated by https://github.com/projectdiscovery/goflags # domains to find subdomains for #domain: [] # file containing list of domains for subdomain discovery #list: # specific sources to use for discovery (-s crtsh,github). use -ls to display all available sources. #sources: [] # use only sources that can handle subdomains recursively (e.g. subdomain.domain.tld vs domain.tld) #recursive: false # use all sources for enumeration (slow) #all: false # sources to exclude from enumeration (-es alienvault,zoomeyeapi) #exclude-sources: [] ...
You can explicitly specify a configuration file using the ‘-config string’ option of subfinder. The provider configuration file is present in ~/.config/subfinder/provider-config.yaml. For example:
bevigil: [] binaryedge: [] bufferover: [] builtwith: [] c99: [] censys: [] certspotter: [] chaos: [] chinaz: [] dnsdb: [] ...
The ‘-pc’ or ‘-provider-config string’ option can be used to specify a different configuration file. The list of resolvers can be specified using the ‘-r string[]’ option. You can also include them in a file and pass it to the ‘-rL’ or ‘-rlist string’ option of subfinder.
Optimisation
The ‘-timeout int’ option makes the subfinder wait for the given number of seconds before timing out. The default value is 30 seconds. The ‘-max-time int’ option specifies the time in minutes to wait for the results. The default value is 10 minutes. If you do not want colour in the output, you can use the ‘-nc’ or ‘-no-color’ option. You can also use the tool as a library in your application code.
You are encouraged to read the subfinder documentation to learn more on its usage and flag options.
