wingsite
CGI

CGI (Common Gateway Interface) is an old but still very useful technology. It lets you add dynamic pages to an otherwise static site without having to make the entire thing dynamic, isn’t that cool? It can work with interpreted scripts or binary files and is called when someone visits a page or submits a form with everything you might need passed to your CGI script via environment variables - with the exception of form data. As a bonus, that means you can easily test out your scripts by simply setting some environment variables and running your script!

The guides I came across when looking into CGI were all pretty old and only focused on the apache and nginx web servers. They also tended to use perl in any examples. I’ll be using caddy here and simple bash examples, but of course, you can use whatever you want because that’s one of the beauties of CGI :D

setting up your web server

Since we’ll be using caddy, which doesn’t directly support running CGI scripts (but does have a transport layer for it), we’ll be offloading the actual processing to a little program called fcgiwrap and reverse proxying to it. The nice thing about this setup is that you’ll only need one fcgiwrap process active and you can pass it as many CGI scripts as you want!

fcgiwrap should be in your repos, so go ahead and install it with your package manager. Depending on your distro, it may start up a service on it’s own. If that’s the case and you plan on just letting your OS handle it completely, you should look at the service file for where it created the socket, you’ll need it when setting up the web server.

If you want to set it up yourself, it’s pretty easy, as it doesn’t have many options. The most basic setup is simply fcgiwrap -s /path/to/fcgiwrap.socket but you can also specify the number of preforked processes with -c like so fcgiwrap -c 16 -s /path/to/fcgiwrap.socket. Obviously the more processes you prefork the harder it will be on your processor.

One important point is to make sure fcgiwrap has permission to access any scripts/binaries you’ll be using, you’ll more than likely want to run it under the same user/group that you’re running your web server as.

Now for the caddy config, this is obviously going to vary depending on what you’re doing. I’ll show 2 setups here that should cover most cases though.

First off, the most common scenario: you’re running a static site and want a couple of dynamic pages. Let’s assume the following directory structure:

- mysite/
-- cgi/
--- guestbook
-- html/
--- index.html
--- guestbook.html

We’ll also say that your fcgi socket is located at /run/fcgiwrap.socket and that the guestbook.html file has a form that sends a POST request to https://example.com/form/guestbook.

You’ll notice that the CGI script isn’t in the same directory as our HTML. This is considered best practice as it means people won’t be able to download your script, whether that actually matters to you or not, is of course, up to you. Here’s a simple Caddyfile for serving a static site:

example.com {
    root * /var/www/mysite/html
    file_server
}

We’ll make use of caddy’s handle directive to only have a specific URL respond with our script.

example.com {
    handle /form/guestbook {
        reverse_proxy unix///run/fcgiwrap.socket }
            transport fastcgi {
                env SCRIPT_FILENAME /var/www/mysite/cgi/guestbook
            }
        }

    root * /var/www/mysite/html
    file_server
}

We’ll come back to the SCRIPT_FILENAME later in the environment variables section.

This sets up a single, static URL, that will respond with a single script. In simple cases this could be all you need, you could also duplicate it and modify the handle to deal with more pages, which would be fine if you only have 2 or 3 instances, but could get messy.

Let’s have a look at something slightly more dynamic. Caddy has various placeholders available to use. Let’s say your cgi directory actually had several scripts and you wanted to call from a /wiki/* path for adding, removing and editing pages. So now your cgi directory looks like this

- cgi/
-- add
-- delete
-- edit

We can set up a Caddy file like this:

example.com {
    handle /wiki/* {
        reverse_proxy unix///run/fcgiwrap.socket }
            transport fastcgi {
                env SCRIPT_FILENAME /var/www/mysite/cgi/{file}
            }
        }

    root * /var/www/mysite/html
    file_server
}

I’ve used the {file} placeholder here which will be substituted for the last part of a URL. For example, when visiting https://example.com/wiki/edit, {file} will equate to edit and run the script at mysite/cgi/edit. The {path} placeholder can also be useful here, but it substitutes the entire path, so it would return /wiki/edit which might be desirable if you mirrored your website paths in your CGI directory. It’s worth looking at the placeholders page and trying out various techniques to see what best suits your situation.

environment variables

Now let’s get onto actually to writing some CGI applications. One of the most important things you’ll be using are the environment variables that will get passed to your program to give you some context to work with. I got most of these from the CGI RFC page. I’m reproducing them here with simpler explanations of what they actually do and so I can separate out some I think are mostly irrelevant. I won’t list some that don’t actually seem to be used any more, or have been replaced.

HTTP request information

Variable Description
PATH_INFO The requested path, e.g, https://example.com/wiki/add would equate to /wiki/add
QUERY_STRING The query string of the URL, sans ?. In https://example.com/?msg=hello&user=1 it would be msg=hello&user=1
REQUEST_METHOD The request method used, e.g, GET, POST, DELETE, etc.
HTTPS Empty if the request is HTTP or on if the request is HTTPS

requester information

Variable Description
REMOTE_ADDR The IP address making the request
REMOTE_HOST If the requester’s IP resolve to a domain that will be returns, otherwise it’s just the IP address again

server information

Variable Description
SERVER_NAME The hostname/domain name of the server, e.g, example.com or localhost
SERVER_PORT The port the request was made to, e.g, 80 or 443
SERVER_PROTOCOL The HTTP version being used
SERVER_SOFTWARE The name and version of the web server, e.g, Caddy/v2.10.2

everything else

Variable Description
GATEWAY_TYPE The CGI version (most likely CGI/1.1)
PATH_TRANSLATED This is supposed to translate URL encoded elements to their, e.g, %20 into a space. In my experience, PATH_INFO does this anyway. More usefully, it will also result in the full path to the requested file in the filesystem, e.g, /var/www/mysite/cgi/wiki/add
SCRIPT_NAME This should be set by the web server and will appended to the DOCUMENT_ROOT to find a CGI script
DOCUMENT_ROOT The system file path to the site root, that is, where your HTML files are being served from
SCRIPT_FILENAME The path to the CGI script

You’ll probably have noticed in the initial examples that I passed SCRIPT_FILENAME to the CGI process to point to the scripts, another option is passing SCRIPT_NAME which will be appended to DOCUMENT_ROOT to find scripts and is fine if you keep the scripts together with your HTML. I prefer the first method but it’s up to you.

AUTH_TYPE is listed on the RFC page which is supposed to tell you if an SSL connection is being made but it always returns blank. I found HTTPS elsewhere which does the similar and actually works. It will be blank for http pages and on for https.

Here’s a simple shell script you can run on a server to get an idea of what the variables look like in real use:

#!/usr/bin/env sh

echo -e "Status: 200 OK\nContent-Type: text/plain\n\n"

echo "HTTP request information"
echo "========================"
echo "PATH_INFO: $PATH_INFO"
echo "QUERY_STRING: $QUERY_STRING"
echo "REQUEST_METHOD: $REQUEST_METHOD"
echo "HTTPS: $HTTPS"
echo ""
echo "Requester information"
echo "====================="
echo "REMOTE_ADDR: $REMOTE_ADDR"
echo "REMOTE_HOST: $REMOTE_HOST"
echo ""
echo "Server information"
echo "=================="
echo "SERVER_NAME: $SERVER_NAME"
echo "SERVER_PORT: $SERVER_PORT"
echo "SERVER_PROTOCOL: $SERVER_PROTOCOL"
echo "SERVER_SOFTWARE: $SERVER_SOFTWARE"
echo ""
echo "Everything else"
echo "==============="
echo "GATEWAY_INTERFACE: $GATEWAY_INTERFACE"
echo "PATH_TRANSLATED: $PATH_TRANSLATED"
echo "DOCUMENT_ROOT: $DOCUMENT_ROOT"
echo "SCRIPT_NAME: $SCRIPT_NAME"
echo "SCRIPT_FILENAME: $SCRIPT_FILENAME"

That’s almost everything you’ll need to write scripts, however, if you intend on using HTML forms you should also note that form data will be passed to your script via stdin. This will result in a string that looks the same as a QUERY_STRING.

One other thing to remember is that your scripts should be executable!

responding to requests

All you have to do to respond to a request is print to stdout.

However, it’s good to know a little about HTTP here. The start of your response string will be headers, followed by 2 new lines, then your actually response body. You can find a list of response headers to use over at MDN but the 2 you’ll probably be using most are Status and Content-Type. Status is actually specific to CGI responses, it should be followed by a HTTP status code and text. You can find a list of them here. Content-Type is a MIME type, usually you’ll be using text/html. You can find a list of all MIME types on iana.

Each header should end in a colon then a space, the value and then a new line. Once you’re done with headers, add two new lines then your response body.

For example, in a shell script, you could use printf "Status: 200 OK\nContent-Type: text/plain\n\nhello there!" as a response.

example

We’ll use the guestbook example from above. Create a file and directory named guestbook/sign.html in your html directory with the following:

<!DOCTYPE html>
<html>
<head><title>Sign the guestbook</title></head>
<body>
    <form method="post" action="/guestbook/sign">
        <label>Name: <input type="text" name="name" /></label><br>
        <label>Message: <input type="text" name="msg" /></label><br>
        <input type="submit" value="submit" />
    </form>
</body>
</html>

Brutally simple. Now for the script, make a new file in your cgi directory called sign:

#!/usr/bin/env bash

respond() (
    printf "Status: $1\nContent-Type: text/plain\nLocation: /guestbook/book.txt\n\n$2"
)

# Reject non-POST requests
if [[ "$REQUEST_METHOD" != "POST" ]]; then
    respond "405 Method Not Allowed"
    exit 0
fi

raw_form=$(cat -)

# Split query string into parts
IFS='&' read -a query_pairs <<< "$raw_form"

# Get the title and content of the new page
for pair in "${query_pairs[@]}"
do
    key=$(echo "$pair" | cut -d "=" -f 1)
    value=$(echo "$pair" | cut -d "=" -f 2)
    if [[ "$key" == "name" ]]; then
        guest_name=$(echo "$value" | tr "+" " ")
    fi

    if [[ "$key" == "msg" ]]; then
        guest_msg=$(echo "$value" | tr "+" " ")
    fi
done

echo -e "[$guest_name]: $guest_msg\n" >> "../html/guestbook/book.txt"

respond "303 See Other" "Successfully signed the guest book"

When someone visits /guestbook/sign.html, they can fill out the form and their name and message will be added to book.txt (I went with a text file here for simplicity) and then they’ll get redirect to the guestbook to see their new message.

We’re using a 303 response here along with a Location header to do the redirecting. Form data will be URL encoded and replace spaces with +’s and such, the tr command is swapping these out but you should use a proper function to do this if it’s available in your chosen language.

This is a very basic example and you should be wary of exacting what you’re allowing on public accessible scripts, security is important ;)

I hope this was somewhat useful and that you have fun with your CGI future!