SpiderFoot running in Docker

A simpler way to get SpiderFoot running is to use Docker. This post will explain how to get your SpiderFoot Docker container up and running.


Installing Python-based applications can sometimes be a bit of a pain, especially when there are many third party library dependencies. In the case of SpiderFoot, it’s been designed from the beginning to have as few external Python dependencies as possible, but even installing the few that are needed trip some people up occasionally.

For this reason, using Docker can be a fast way to get up and running without having to worry about managing dependencies. This post will explain a bit about Docker and then get straight into running SpiderFoot within a container.

What is Docker?

From the outset it should be made clear that Docker is not a virtualization technology like VirtualBox or VMWare - it doesn’t interpret the code running within the container or emulate any devices. Instead, it uses the operating system’s built-in capabilities for resource (CPU, memory, disk, network) isolation, providing the container with what looks to be a complete operating system when in reality only a subset is being exposed.

From start to finish, the process of getting a Docker container running looks like this:

  1. Create a Dockerfile, defining how the application looks inside a container
  2. Build a Docker image using the Dockerfile
  3. Run a container based on the created image

SpiderFoot ships with a Dockerfile, so all you have to deal with are steps 2 and 3.

Creating a Docker Image

Using the Dockerfile supplied with SpiderFoot, we can create the image that will be used to run the container:

docker build -t spiderfoot .

This can take up to a few minutes, but the end result should be an image created with the tag of spiderfoot (that’s what the -t flag above did):

steve@dev:~$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
spiderfoot          latest              2731095e2907        7 hours ago         89.47 MB
alpine              3.7                 3fd9065eaf02        6 weeks ago         4.148 MB

You can ignore the alpine image, because that is just a ‘layer’ that was used for the SpiderFoot image to be created with. Alpine Linux is a minimal Linux distribution ideal for use with Docker because it keeps the images small.

We can now run SpiderFoot!

Running SpiderFoot

If you look at the Dockerfile you’ll see that the port it will listen on within the container is 5001. But that’s within the container - what we need to do is map a port on the server running the container to that port, so that we can connect to it from outside the container. That’s what the -p argument will do. Below you can see we are mapping port 5009 on the local server to port 5001 which is the port SpiderFoot is listening on within the container.

docker run -p 5009:5001 -d spiderfoot

The -d argument tells docker to detach from the container. In other words, it puts the running container into the background so we can do other things.

Let’s verify it’s running:

steve@dev:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
d98b5712ab93        spiderfoot          "/usr/bin/python ./sf"   7 hours ago         Up 7 hours>5001/tcp   modest_stonebraker

We can also shell into the container, just for curiosity’s sake:

steve@dev:~$ docker exec -it d98b5712ab93 /bin/bash
~ $ ps auxw
    1 spiderfo   0:28 /usr/bin/python ./sf.py
   25 spiderfo   0:00 /bin/sh
   29 spiderfo   0:00 ps auxww
~ $

So now that SpiderFoot is running and mapped to a port we can connect to, let’s use it:

steve@dev:~$ python ./sfcli.py -s http://localhost:5009

  _________      .__    .___          ___________            __
 /   _____/_____ |__| __| _/__________\_   _____/___   _____/  |_
 \_____  \\____ \|  |/ __ |/ __ \_  __ \    __)/  _ \ /  _ \   __\
 /        \  |_> >  / /_/ \  ___/|  | \/     \(  <_> |  <_> )  |
/_______  /   __/|__\____ |\___  >__|  \___  / \____/ \____/|__|
        \/|__|           \/    \/          \/
                Open Source Intelligence Automation.
                by Steve Micallef | @binarypool

[*] Version 2.11.
[*] Server http://localhost:5009 responding.
[*] Loaded previous command history.
[*] Type 'help' or '?'.

To see this in action, check out the asciinema video below: