WebFaction Forum / Setting up Django with nginx/FastCGI.

IAIHMB · 2008-09-13 20:17:19

Setting up Django with nginx/FastCGI is pretty straight forward, just:

1) Create a "Custom (Listening on Port)" app. In the remaining steps replace all occurrences of "(App Name)" and "(App Port)" with the name and port of the app that you created.
2) Connect to the server that your account is on using an SSH client and execute the following commands:

Code:

cd ~/webapps/(App Name)
mkdir -p bin lib/python2.5

# Install Django and flup.
cd ~/webapps/(App Name)
PYTHONPATH=lib/python2.5 easy_install-2.5 -d lib/python2.5 -s bin http://www.djangoproject.com/download/1.0/tarball/ flup

./bin/django-admin.py startproject myproject

# Install nginx.
cd ~/webapps/(App Name)
wget http://sysoev.ru/nginx/nginx-0.6.32.tar.gz
tar fxz nginx-0.6.32.tar.gz
cd nginx-0.6.32
./configure --prefix=~/webapps/(App Name)
make
make install
cd ..
rm -fr nginx-0.6.32
rm nginx-0.6.32.tar.gz

# Replace "example.com" with your domain name and "/" with your app's app path.
# Also, please note that the config code contains backslashes in the fastcgi parameters.
# If you copy and paste the entire 'cat ... EOF' block as shown here, the nginx.conf
# file will be created without the backslashes. If you are copying and pasting the config
# shown here into an existing nginx.conf, be sure to remove those backslashes.
cd ~/webapps/(App Name)
cat > conf/nginx.conf <<EOF
worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include  mime.types;

    server {
        listen       (App Port);
        server_name  example.com;

        location / {
            fastcgi_pass   unix:`echo $PWD`/myproject/myproject.sock;

            fastcgi_param  CONTENT_LENGTH   \$content_length;
            fastcgi_param CONTENT_TYPE      \$content_type;
            fastcgi_param  PATH_INFO        \$fastcgi_script_name;
            fastcgi_param  QUERY_STRING     \$query_string;
            fastcgi_param  REQUEST_METHOD   \$request_method;
            fastcgi_param  SERVER_NAME      \$server_name;
            fastcgi_param  SERVER_PORT      \$server_port;
            fastcgi_param  SERVER_PROTOCOL  \$server_protocol;
        }
    }
}
EOF

# Start Django
cd ~/webapps/(App Name)/myproject
python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=$PWD/myproject.sock > /dev/null 2>&1 &

# Start nginx
cd ~/webapps/(App Name)
./sbin/nginx

3) Add the app that you created in step one to a website and wait a couple of minutes for the changes to take affect.

That's it.

Last edited by IAIHMB (2008-09-19 14:49:51)

globophobe · 2008-09-15 00:51:33

You might also want to add something like:

This rewrite rule appends a slash to the url if one is not already there. I think nginx is better to handle this than django.

Code:

if ($request_uri ~* "^[\w\-\/]+[^\/?]$") {rewrite ^(.*)$ $scheme://$host$1/ permanent;}

This rewrite rule to normalizes the url from example\.com to www\.example\.com

Code:

if ($host ~* "^[\w-]+\.\w{2,3}$") {rewrite ^(.*)$ $scheme://www.$host$1 permanent;}

These rewrite rules would go somewhere inside:

Code:

location / {

}

glisha · 2008-11-01 20:21:41

I've tried the setup above and had couple of issues with the tutorial so I thought I'd share them.

1. I'm not sure why but when I do ./configure --prefix=~/webapps the tilde didn't expand and nginx ended up installed in /home/glisha/webapps/nginxapp/~/conf...

So I had to specify the full path for --prefix=/home/glisha/webapps/nginxapp/.

2. ./configure couldn't find pcre and it complained that I would't be able to use http_redirect. I had to download pcre and point the configure script to the source.

Code:

cd ~/webapps/nginxapp/
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.8.tar.gz
tar -xzf pcre-7.8.tra.gz
cd pcre-7.8
./configure --prefix=/home/glisha/webapps/nginxapp/pcre
make
make install # I'm not sure if I need to install or just make pcre.

That I run configure for nginx once again with pcre

Code:

./configure --prefix=/home/glisha/webapps/nginxapp --with-pcre=/home/glisha/webapps/nginxapp/pcre-8.4/
make
make install

3. I had to add the content-type passed for the fcgi in conf/nginx.conf because my forms didn't work.

Code:

fastcgi_param CONTENT_TYPE $content_type;

---

I run pylot with 10 agents for 360 sec on the same django application with apache/mod_python and on nginx/fscgi. I managed to get 8request/second with apache/mod_python and 13request/second with nginx and the memory footprint is great.

Apache took about 50MB+15MB for memcache and nginx/fcgi only 20MB without memcache.

I don't know why doesn't my application hit memcached when I run it thru nginx, but I'll ask in another thread.

redmonkey · 2008-12-13 08:23:12

IAIHMB wrote:
Code:
./configure --prefix=~/webapps/(App Name)

This only created directories in the right place if I gave it a fully qualified prefix path:

Code:

./configure --prefix=/home/redmonkey/webapps/(App Name)

Also, I noticed that adding a "/" to the end of your path causes configure to make paths with double slashes in them, so I left off the final slash i.e

Code:

--prefix=<stuff>/webapps/unusability

instead of

Code:

--prefix=<stuff>/webapps/unusability/

Simple stuff really.

Thanks a lot IAIHMB

tinodb · 2009-03-13 06:47:18

Hi,

I can't get this to work. I get the following error when I try to start nginx:

Code:

[emerg] 28855#0: invalid host in upstream "/home/tinodb/webapps/django/projects/dsac/dsac.sock" in /home/tinodb/webapps/django//conf/nginx.conf:15

The socket file does exist (it's created by manage.py).

Also using

Code:

unix:`echo $PWD`/myproject/myproject.sock;

in the nginx.conf file did not work. it gave the error

Code:

2009/03/13 06:38:43 [emerg] 27847#0: invalid number of arguments in "fastcgi_param" directive in /home/tinodb/webapps/django//conf/nginx.conf:17

So I used the full path instead: /home/tinodb/webapps/django/projects/dsac/dsac.sock

What can I do to fix this?

Tino

seanf · 2009-03-14 12:43:15

tinodb wrote:
Hi,

I can't get this to work. I get the following error when I try to start nginx:
Code:
[emerg] 28855#0: invalid host in upstream "/home/tinodb/webapps/django/projects/dsac/dsac.sock" in /home/tinodb/webapps/django//conf/nginx.conf:15
The socket file does exist (it's created by manage.py).

Also using
Code:
unix:`echo $PWD`/myproject/myproject.sock;
in the nginx.conf file did not work. it gave the error
Code:
2009/03/13 06:38:43 [emerg] 27847#0: invalid number of arguments in "fastcgi_param" directive in /home/tinodb/webapps/django//conf/nginx.conf:17
So I used the full path instead: /home/tinodb/webapps/django/projects/dsac/dsac.sock

What can I do to fix this?

Tino

This error ...

Code:

2009/03/13 06:38:43 [emerg] 27847#0: invalid number of arguments in "fastcgi_param" directive in /home/tinodb/webapps/django//conf/nginx.conf:17

.. has nothing to do with the path to the socket. The problem is that you left the arguments off of your various fastcgi_param directives, eg you have this...

Code:

            fastcgi_param  CONTENT_LENGTH   ;
            fastcgi_param  PATH_INFO        ;
            fastcgi_param  QUERY_STRING     ;
            fastcgi_param  REQUEST_METHOD   ;
            fastcgi_param  SERVER_NAME      ;
            fastcgi_param  SERVER_PORT      ;
            fastcgi_param  SERVER_PROTOCOL  ;

... when you should have this ...

Code:

            fastcgi_param  CONTENT_LENGTH   $content_length;
            fastcgi_param  PATH_INFO        $fastcgi_script_name;
            fastcgi_param  QUERY_STRING     $query_string;
            fastcgi_param  REQUEST_METHOD   $request_method;
            fastcgi_param  SERVER_NAME      $server_name;
            fastcgi_param  SERVER_PORT      $server_port;
            fastcgi_param  SERVER_PROTOCOL  $server_protocol;

Hope that helps!

Regards,

Sean F, WebFaction Support
--
WebFaction - Agile hosting for everyone
http://webfaction.com - http://twitter.com/webfaction

tinodb · 2009-03-15 13:57:31

Ooops, thought I used "include fastcgi_params" ...

thx, it's working now

soares · 2009-05-15 20:06:20

I can't seem to get this to work. I have nginx up and running, but when I make a request, it seems as if the request gets munged before it gets do django. For example, when I go to test.natesoares.com/test/, I get:

Code:

TypeError at \/test/

'NoneType' object is not iterable

Request Method:     \GET
Request URL:     http://test.natesoares.com%5C/test/%5C/test/?%5C
Exception Type:     TypeError
Exception Value:     

'NoneType' object is not iterable

Don't know what's going on here. My nginx.conf is just:

Code:

worker_processes  1;  

events {
    worker_connections  1024;
}

http {
    include  mime.types;

    server {
        listen       7322;
        server_name  natesoares.com;

        location / { 
            fastcgi_pass   unix:/home/soares/webapps/overviewer/overviewer/overviewer.sock;

            fastcgi_param  CONTENT_TYPE     \$content_type;

            fastcgi_param  CONTENT_LENGTH   \$content_length;
            fastcgi_param  PATH_INFO        \$fastcgi_script_name;
            fastcgi_param  QUERY_STRING     \$query_string;
            fastcgi_param  REQUEST_METHOD   \$request_method;
            fastcgi_param  SERVER_NAME      \$server_name;
            fastcgi_param  SERVER_PORT      \$server_port;
            fastcgi_param  SERVER_PROTOCOL  \$server_protocol;
        }
    }   
}

Any ideas?

globophobe · 2009-05-15 20:42:27

There is likely an error in your python code.

If you believe that there is an error in your nginx conf then you should enable error logging. In your conf, add:

Code:

error_log /path/to/your/error.log

soares · 2009-05-15 22:21:39

I don't think it's a python issue, because django isn't getting the correct request. As I said above, when I go to test.natesoares.com/test, I get the following error from django:

Code:

...
Request Method:     \GET
Request URL:     http://test.natesoares.com%5C/test/%5C/test/?%5C
...

Django hasn't been given the correct URL. So yes, the error thrown (and the one in the logs) is a django error, but the error is thrown by django because django isn't given the right url. I don't know enough about nginx to know what's going on or why test.natesoares.com would be converted to the url that django got. Any ideas?

globophobe · 2009-05-16 02:28:49

I believe your problem may be caused by your use of backslashes in your fastcgi parameters.

Code:

fastcgi_param  CONTENT_TYPE     \$content_type;

Should be:

Code:

fastcgi_param  CONTENT_TYPE     $content_type;

seanf · 2009-05-16 07:56:52

If you copy and paste everything from the original post, those backslashes will be escaped when the 'cat .... EOF' stuff executes and they won't appear in the final config file.

I've added a comment to the original post to clarify the backslash issue.

soares · 2009-05-16 11:25:17

That makes sense. However, I took the slashes out, re-ran manage.py runfcgi, and re-ran sbin/nginx, and the problem is still there. Is there something else I have to do to re-load the new conf file?

seanf · 2009-05-16 11:38:32

Re-running sbin/nginx does not restart nginx. You need to find the PID of your nginx process, eg:

Code:

ps -u your_username -o pid,command | grep nginx

Then kill the master nginx process, eg if the PID was 4321:

Code:

kill 4321

THEN you can re-start nginx with sbin/nginx to activate the new config.

frozenskys · 2009-07-14 07:15:18

Hi,
I am using this setup, with the addition of the nginx upload progress module, and it works really well (I've even mirrored the whole setup on my Ubuntu Dev server at home). However I can't work out how to get the real IP address of the client to be written in the nginx logs. Can anyone point me in the right direction?

Regards,
Richard.

seanf · 2009-07-15 08:47:41

frozenskys wrote:
I am using this setup, with the addition of the nginx upload progress module, and it works really well (I've even mirrored the whole setup on my Ubuntu Dev server at home). However I can't work out how to get the real IP address of the client to be written in the nginx logs. Can anyone point me in the right direction?

The real IP is contained in the 'X-Forwarded-For' header. For example, here is the Apache logfile directive we set up when you install one of our regular Django apps:

Code:

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

FSX · 2009-07-30 03:53:22

seanf wrote:
Re-running sbin/nginx does not restart nginx. You need to find the PID of your nginx process, eg:
Code:
ps -u your_username -o pid,command | grep nginx
Then kill the master nginx process, eg if the PID was 4321:
Code:
kill 4321
THEN you can re-start nginx with sbin/nginx to activate the new config.

It might be better to use:

Code:

sbin/nginx -s quit
sbin/nginx

Or reload the config:

Code:

sbin/nginx -s reload

tinodb · 2009-07-30 04:20:29

Or even better, send a HUP signal.

kill -HUP `cat /var/run/nginx.pid` will reload the config, and make sure you have no interuption in serving your pages.

Here is my ningx start/stop/restart script:

Code:

NAME=nginx
BASEDIR="/home/<user>/webapps/django/sbin/"
PIDFILE="$BASEDIR/../$NAME.pid"

case "$1" in
  start)
    echo -n "Starting $NAME: "
    $BASEDIR$NAME
    echo "$NAME Started."
    ;;
  stop)
    echo -n "Stopping $NAME: "
    if [ -f $PIDFILE ]; then
       kill -QUIT `cat $PIDFILE`
       rm -f -- $PIDFILE
    fi
    echo "Stopped."
    ;;
  restart)
    echo -n "Restarting $NAME: "
    if [ -f $PIDFILE ]; then
       kill -HUP `cat $PIDFILE`
    else
       $BASEDIR$NAME
    fi
    echo "restarted $NAME."
    ;;

  *)
    echo "Usage: $N {start|stop|restart}" >&2
    exit 1
    ;;
esac

exit 0

erikvorhes · 2009-08-11 16:27:38

How would one modify nginx.conf to serve multiple Django projects?

seanf · 2009-08-11 17:12:59

erikvorhes wrote:
How would one modify nginx.conf to serve multiple Django projects?

Use multiple server entries in your nginx config, pointing to different sockets (and different domains if necessary) eg:

Code:

    server {
        listen       43210;
        server_name  firstdomain.com;

        location / {
            fastcgi_pass   unix:`echo $PWD`/firstproject/firstproject.sock;

            fastcgi_param  CONTENT_LENGTH   $content_length;
            fastcgi_param  CONTENT_TYPE     $content_type;
            fastcgi_param  PATH_INFO        $fastcgi_script_name;
            fastcgi_param  QUERY_STRING     $query_string;
            fastcgi_param  REQUEST_METHOD   $request_method;
            fastcgi_param  SERVER_NAME      $server_name;
            fastcgi_param  SERVER_PORT      $server_port;
            fastcgi_param  SERVER_PROTOCOL  $server_protocol;
            
        }
    }
    server {
        listen       43210;
        server_name  seconddomain.com;

        location / {
            fastcgi_pass   unix:`echo $PWD`/secondproject/secondproject.sock;

            fastcgi_param  CONTENT_LENGTH   $content_length;
            fastcgi_param  CONTENT_TYPE     $content_type;
            fastcgi_param  PATH_INFO        $fastcgi_script_name;
            fastcgi_param  QUERY_STRING     $query_string;
            fastcgi_param  REQUEST_METHOD   $request_method;
            fastcgi_param  SERVER_NAME      $server_name;
            fastcgi_param  SERVER_PORT      $server_port;
            fastcgi_param  SERVER_PROTOCOL  $server_protocol;
        }
    }

and run multiple Django FastCGI processes:

Code:

cd ~/webapps/(App Name)/firstproject
python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=$PWD/firstproject.sock > /dev/null 2>&1 &
cd ../secondproject
python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=$PWD/secondproject.sock > /dev/null 2>&1 &

Hope that helps!

bwooceli · 2009-08-22 22:26:23

So I've made it so far as to get a friendly "Welcome to nginx" message when I hit the subdomain I've pointed at the app, but it doesn't show my project... when I run manage, I get the following result, is that correct?

Code:

[apt9online@web67 leel]$ manage runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/leel/leel.sock > /dev/null 2>&1 &
[1] 15291
[apt9online@web67 leel]$

in the access and error logs, it only shows the browser requesting the favicon and the error that it couldn't find one

Last edited by bwooceli (2009-08-22 22:30:23)

David L · 2009-08-23 04:59:09

That's correct. The output of '[1] 15291' is just telling you the PID of the process it created

bwooceli · 2009-08-24 09:09:54

Still not able to get anything but "Welcome to nginx", so a couple of stupid follow-up questions.

- After running manage fastcgi, should I be able to see that process running then (because I don't)
- Should manage fastcgi create an actual .sock file in my directory (because it doesn't appear to)

Thanks for bearing with me on this.

frozenskys · 2009-08-24 09:27:02

1) Yes you should see the django fcgi process if it has started OK
2) Yes if the config is correct you should get a .sock file created.

It sounds like the django project isn't set up properly and is failing - to see why try starting it without the redirect e.g.

Code:

manage runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/leel/leel.sock

Also I would check the settings in your nginx.conf because if the django process is down you should get a "500 Server Error" page from nginx not the "welcome to niginx" page.

Regards,
Richard.

bwooceli · 2009-08-24 11:00:42

Thanks, that got me down the right troubleshooting path and I'm up and running So can someone help enlighten me on a couple other general questions and contexts? When I look at my running processes now, I see that each sock is represented twice.

Code:

  RSS   PID CMD
10848  1254 python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/motu/motu.sock
10444  1255 python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/motu/motu.sock
  932  3230 nginx: worker process
10168 11762 python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/leel/leel.sock
15924 11763 python2.5 manage.py runfcgi maxchildren=1 maxspare=1 method=prefork socket=/home/apt9online/webapps/9online/leel/leel.sock
  908 14150 ps -u apt9online -o rss,pid,cmd
  840 17404 nginx: master process ./sbin/nginx
 1564 31894 sshd: apt9online@pts/0
 1524 31896 -bash

I'm just trying to get my general knowledge/know-how up, what does this actually mean?

#1 2008-09-13 20:17:19

Setting up Django with nginx/FastCGI.

Code:

#2 2008-09-15 00:51:33

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

Code:

#3 2008-11-01 20:21:41

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

Code:

#4 2008-12-13 08:23:12

Re: Setting up Django with nginx/FastCGI.

IAIHMB wrote:

Code:

Code:

Code:

Code:

#5 2009-03-13 06:47:18

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

Code:

#6 2009-03-14 12:43:15

Re: Setting up Django with nginx/FastCGI.

tinodb wrote:

Code:

Code:

Code:

Code:

Code:

Code:

#7 2009-03-15 13:57:31

Re: Setting up Django with nginx/FastCGI.

#8 2009-05-15 20:06:20

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

#9 2009-05-15 20:42:27

Re: Setting up Django with nginx/FastCGI.

Code:

#10 2009-05-15 22:21:39

Re: Setting up Django with nginx/FastCGI.

Code:

#11 2009-05-16 02:28:49

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

#12 2009-05-16 07:56:52

Re: Setting up Django with nginx/FastCGI.

#13 2009-05-16 11:25:17

Re: Setting up Django with nginx/FastCGI.

#14 2009-05-16 11:38:32

Re: Setting up Django with nginx/FastCGI.

Code:

Code:

#15 2009-07-14 07:15:18

Re: Setting up Django with nginx/FastCGI.

#16 2009-07-15 08:47:41

Re: Setting up Django with nginx/FastCGI.

frozenskys wrote:

Code:

#17 2009-07-30 03:53:22

Re: Setting up Django with nginx/FastCGI.

seanf wrote:

Code:

Code:

Code:

Code:

#18 2009-07-30 04:20:29

Re: Setting up Django with nginx/FastCGI.

Code:

#19 2009-08-11 16:27:38

Re: Setting up Django with nginx/FastCGI.

#20 2009-08-11 17:12:59

Re: Setting up Django with nginx/FastCGI.

erikvorhes wrote:

Code: