Vultr DocsLatest Articles

How to Host Searx with Gunicorn and Nginx on OpenBSD

OpenBSDWeb ServersGunicorn
HM
Updated on November 21, 2023
How to Host Searx with Gunicorn and Nginx on OpenBSD header image

Searx is a free metasearch engine application that lets you host aggregated web results with a key focus on privacy. A metasearch engine can query multiple search engines to produce results without user tracking or profiling.

In this article, you will install Searx with Gunicorn and Nginx, then configure Morty as a web content sanitizer, and Filtron as the reverse HTTP proxy on an OpenBSD server.

Prerequisites

Create Users

Create new user accounts for Searx, Morty, and Filtron with no login access. For the user searx, the home directory should point to the service's root directory.

Create the Searx home directory.

$ doas mkdir /usr/local/searx

Create a new user searx.

$ doas useradd -d /usr/local/searx/ -s /sbin/nologin -u 3000 searx

Grant the user ownership rights to the home directory.

$ doas chown -R searx:searx /usr/local/searx/

Create a new user morty.

$ doas useradd -s /sbin/nologin -u 30001 morty

Create a new user filtron.

$ doas useradd -s /sbin/nologin -u 30002 filtron

Installation

Install Nginx

$ doas pkg_add nginx

Allow Nginx to start at boot time.

$ doas rcctl enable nginx

Start Nginx.

$ doas rcctl start nginx

Install Python, Pip, Git, and Go

$ doas pkg_add python3 py3-pip git py3-lxml go

Enable as default pip.

$ doas ln -sf /usr/local/bin/pip3.8 /usr/local/bin/pip

Install Gunicorn

$ doas pip install gunicorn

Install Morty

$ go get github.com/asciimoo/morty

Move Morty to a system-wide directory.

$ doas mv go/bin/morty /usr/local/bin/

Install Filtron

$ go get github.com/asciimoo/filtron

Move Filtron to a system directory.

$ doas mv go/bin/filtron /usr/local/bin/

Install Searx

$ doas git clone "https://github.com/searx/searx.git"

Move the file to the searx home directory.

$ doas cp -r searx /usr/local/searx/searx-files

Install Searx requirements.

$ doas pip install -r /usr/local/searx/searx-files/requirements.txt

Configurations

Configure Filtron

Configure Filtron by making a copy of the default configuration file.

Create a new /etc/filtron directory.

$ doas mkdir -p /etc/filtron

Copy the configuration file.

$ doas cp $HOME/go/pkg/mod/github.com/asciimoo/filtron@v0.2.0/example_rules.json /etc/filtron/rules.json

Configure Searx

Create a new Searx configuration directory.

$ doas mkdir -p /etc/searx/

Make a copy the settings template to the directory.

$ doas cp /usr/local/searx/searx-files/utils/templates/etc/searx/use_default_settings.yml /etc/searx/settings.yml

Generate a secret key and save it to the file.

$ doas sed -i -e "s/ultrasecretkey/$(openssl rand -hex 16)/g" "/etc/searx/settings.yml"

Also, uncomment the last lines within the settings.yml file to enable Morty.

# uncomment below section if you have running morty proxy
#result_proxy:
#    url : http://127.0.0.1:3000/
#    key : !!binary "your_morty_proxy_key"

Save and close the file.

Configure Nginx

First, backup the current Nginx configuration file.

$ doas cp /etc/nginx/nginx.conf /etc/nginx/nginx.ORIG

Then, open and edit the main Nginx configuration file using your favorite editor.

$ doas nano /etc/nginx/nginx.conf

Add the following lines of code within the HTTP block.

http {

#HTTP Server Block Begins

server {
 listen 80;
 listen [::]:80;
 server_name search.example.com;

location /.well-known/acme-challenge/ {
 rewrite ^/.well-known/acme-challenge/(.*) /$1 break;
 root /acme;
     
 }

 }

}

Save and close the file.

Next, request for a free Let's Encrypt SSL certificate to secure the server with encrypted HTTPS traffic.

Create a new acme-client configuration file.

$ touch doas /etc/acme-client.conf

Open the file.

$ doas nano /etc/acme-client.conf

Then, paste the following contents:

authority letsencrypt {
  api url "https://acme-v02.api.letsencrypt.org/directory"
  account key "/etc/ssl/private/letsencrypt.key"
}

domain search.example.com {
  alternative names { search.example.com }
  domain key "/etc/ssl/private/search.example.com.key"
  domain certificate "/etc/ssl/search.example.com.crt"
  domain full chain certificate "/etc/ssl/search.example.com.pem"
  sign with letsencrypt
}

Now, request a free certificate for the subdomain.

$ doas acme-client -v search.example.com

Once successful, edit the Nginx configuration file again to accept HTTPS traffic. On port 443.

Paste the following lines of code within a new server block:

http {

server {
  listen 443 ssl;
  listen [::]:443 ssl;
  server_name search.example.com;

location / {
    proxy_pass         http://127.0.0.1:4004/;

    proxy_set_header   Host             $host;
    proxy_set_header   Connection       $http_connection;
    proxy_set_header   X-Real-IP        $remote_addr;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
    proxy_set_header   X-Scheme         $scheme;
    proxy_set_header   X-Script-Name    /searx;
}

location /static/ {
        alias /usr/local/searx/searx-files/searx/static/;
}

location /morty {
        proxy_pass http://127.0.0.1:3000/;

        proxy_set_header   Host             $host;
        proxy_set_header   Connection       $http_connection;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header   X-Scheme         $scheme;
}

ssl_certificate /etc/ssl/search.example.com.pem;
ssl_certificate_key /etc/ssl/private/search.example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE+AESGCM:DHE+AESGCM:ECDHE+ECDSA+AES+SHA256;

}

}

Also, redirect all HTTP requests to HTTPS by including a 301 redirect within the first HTTP server block.

Your HTTP block should now be similar to the one below:

#HTTP Server Block Begins

server {
listen 80 default_server;
listen [::]:80;
server_name search.example.com; 
return 301 https://$host$request_uri;
}

Save and close the file.

Check the Nginx configuration file for errors.

$ doas nginx -t

Restart Nginx.

$ doas rcctl restart nginx

Setup System Services

To control Searx, Morty, Filtron, and Gunicorn as services on the server, set up new files in the rc.d directory.

Create the following files within the /etc/rc.d directory using your preferred text editor.

Gunicorn Searx:

$ doas nano /etc/rc.d/gunicornsearx

#!/bin/ksh

RUN_DIR="/var/run/gunicornsearx"
daemon="/usr/local/bin/gunicorn"
gunicornsearx_flags="-b 127.0.0.1:8001 --chdir /usr/local/searx/searx-files/searx --pythonpath /usr/local/searx/searx-files -p /var/run/gunicornsearx/gunicornsearx.pid -D searx.webapp"
gunicornsearx_user="searx"

. /etc/rc.d/rc.subr

pexp="/usr/local/bin/python.*"

# Process ID File

rc_pre() {
        if [[ ! -d /var/run/gunicornsearx ]]; then
            mkdir $RUN_DIR
            chown -R searx:searx $RUN_DIR
        fi
    }

rc_stop() {
    if [[ -f $RUN_DIR/gunisearx.pid ]]; then
        kill $(cat $RUN_DIR/gunicornsearx.pid)
        rm $RUN_DIR/gunicornsearx.pid
    fi
}

rc_cmd $1

Save and close the file.

Morty:

$ doas nano /etc/rc.d/morty

#!/bin/ksh

daemon="/usr/local/bin/morty"
morty_user="morty"

rc_bg=YES

. /etc/rc.d/rc.subr

rc_cmd $1

Save and close the file.

Filtron:

$ doas nano /etc/rc.d/filtron

#!/bin/ksh

daemon="/usr/local/bin/filtron"
filtron_flags="-api '127.0.0.1:4005' -listen '127.0.0.1:4004' -rules '/etc/filtron/rules.json' -target '127.0.0.1:8001'"
filtron_user="filtron"

rc_bg=YES

. /etc/rc.d/rc.subr

rc_cmd $1

Save and close the file.

Make the service scripts executable.

$ doas chmod +x /etc/rc.d/gunicornsearx /etc/rc.d/morty /etc/rc.d/filtron

Now, enable each of the services to start at boot time.

$ doas rcctl enable filtron 
$ doas rcctl enable morty 
$ doas rcctl enable gunicornsearx

Then, start all services one by one.

$ doas rcctl start filtron
$ doas rcctl start morty 
$ doas rcctl start gunicornsearx

Configure Firewall

Back up the current packet filter firewall configuration file.

$ doas cp /etc/pf.conf /etc/pf.conf.bak

Then, remove the default configuration file.

$ doas rm /etc/pf.conf

Next, edit a new pf.conf firewall configuration file using any text editor.

$ nano /etc/pf.conf

Paste the following rules to harden your server and only allow SSH, HTTP, HTTPS traffic.

# declare tcp ports
tcp_services = "{ sshd, http, https }"

# blocks all other unused ports
block all

# Allow Traffic from specified tcp ports
pass in on egress proto tcp to port $tcp_services

# Allow outgoing traffic
pass out

Save and close the file.

Confirm the current firewall table with the following command.

$ doas pfctl -sr

Reload the firewall.

$ doas pfctl -f /etc/pf.conf

Test the Server

Searx is up and running. Visit your subdomain to test the service.

http://search.example.com

Conclusion

Congratulations, you have hosted Searx on OpenBSD with Gunicorn and Nginx, then secured your server with HTTPS. For further in-depth documentation about the service, visit the Searx Github repository.