WebFaction Forum / Cron + Django Script

kconragan · 2008-02-21 17:41:27

I've been trying to get this to work for a few hours now with no luck so looking for some help.

I have the following python script:

Code:

from django.core.management import setup_environ
import settings
setup_environ(settings)

from conragan.shoebox.views import update_twitter
update_twitter()

When I run the script through a shell, it works with no problems. However, called from a cron job, nothing is happening. The cron job is as follows:

Code:

1,21,41 * * * * /home/kconragan/webapps/django/apache2/bin/start > /dev/null 2>&1
* * * * * python2.5 /home/kconragan/webapps/django/conragan/twitter_updater.py > /home/kconragan/cron.out 2>&1

No errors or anything are being output to cron.out. Any ideas?

kconragan · 2008-02-21 22:45:32

OK, I added the following to my crontab:

Code:

PYTHONPATH=/home/kconragan/lib/python2.5:/home/kconragan/webapps/django:
DJANGO_SETTINGS_MODULE=conragan.settings

I'm now at least getting an error output to cron.out:

Traceback (most recent call last):
File "/home/kconragan/webapps/django/conragan/twitter_updater.py", line 5, in ?
from conragan.shoebox.views import update_twitter
File "/home/kconragan/webapps/django/conragan/../conragan/shoebox/views.py", line 3, in ?
import twitter
File "/home/kconragan/lib/python2.5/twitter.py", line 14, in ?
import simplejson
ImportError: No module named simplejson

Why would the script not be able to find simplejson through cron when it can find it through shell? Ideas how to fix?

Last edited by kconragan (2008-02-21 22:46:45)

seanf · 2008-02-21 22:55:58

Hi.... make sure your crontab is in unix format (:set ff=unix in vim). You might also want to search for ^M characters and delete them wherever you see them.

Regards,

Sean F

kconragan · 2008-02-22 00:25:11

Thank seanf. Unfortunately, didn't seem to help. For what it's worth, I edited the crontab using nano, not VIM. I did open the crontab in vim and execute :set ff=unix, but again, no change.

Could this be a problem with how simplejson is installed? I assumed it was installed correctly since running the script through a shell returned no errors. Any other ideas?

IAIHMB · 2008-02-22 15:06:30

Could this be a problem with how simplejson is installed? I assumed it was installed correctly since running the script through a shell returned no errors. Any other ideas?

I think that this is the problem:

[kconragan@web28 ~]$ find lib/python2.5/ -name simplejson
lib/python2.5/django/utils/simplejson
lib/python2.5/simplejson-1.7.3/build/lib.linux-i686-2.5/simplejson
lib/python2.5/simplejson-1.7.3/build/temp.linux-i686-2.5/simplejson
lib/python2.5/simplejson-1.7.3/simplejson
lib/python2.5/simplejson-1.7.3/docs/simplejson
[kconragan@web28 ~]$

When you easy_install Python modules they're not necessarily installed in a directory that's in your PYTHONPATH. Try adding the following to the top of your crontab:

PYTHONPATH=/home/kconragan/lib/python2.5/:/home/kconragan/lib/python2.5/simplejson-1.7.3/:/home/kconragan/webapps/django/

Hope it helps.

kconragan · 2008-02-22 15:17:58

Thanks for the tip.

I was just about to update this message stating I had fixed the issue by changing to:

Code:

0,30 * * * * /usr/local/bin/python2.5  /home/kconragan/webapps/django/conragan/twitter_updater.py > /home/kconragan/cron.out 2>&1

I'll give your suggestion a go though as it seems a bit cleaner.

patrickbeeson · 2008-05-11 19:45:01

I'm having a very similar issue with my cron that runs a python script within my Django project.

Here is the error I'm getting:

Traceback (most recent call last):
File "/home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/update_feeds.py", line 55, in <module>
update_feeds(options.verbose)
File "/home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/update_feeds.py", line 8, in update_feeds
from knoxd.apps.aggregator.models import Feed, FeedItem
ImportError: No module named knoxd.apps.aggregator.models

And here is the cron I'm running:

PYTHONPATH=/home/pbeeson/webapps/django/lib:/home/pbeeson/lib/python2.5 DJANGO_SETTINGS_MODULE=knoxd.settings

0,30 * * * * /usr/local/bin/python2.5 /home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/update_feeds.py > /home/pbeeson/cron.log 2>&1

The cron writes the error to the file, but I can't figure out why I'm getting the error.

I've also tried running the following cron for a bash script that runs the python script:

1 * * * * /home/pbeeson/webapps/django/knoxd/feed_updater

Here's the bash script:

#!/bin/bash
. ~/.bash_profile

/usr/local/bin/python2.5 /home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/update_feeds.py

Any help would be greatly appreciated since this cron was running fine on Dreamhost (ugh)!

likebike · 2008-05-11 20:18:45

Hi Patrick,

It looks to me like you need to add /home/pbeeson/webapps/django to your cron's PYTHONPATH. The resulting crontab would be this:

PYTHONPATH=/home/pbeeson/webapps/django/lib:/home/pbeeson/lib/python2.5:/home/pbeeson/webapps/django:/home/pbeeson/webapps/django/globalapps
DJANGO_SETTINGS_MODULE=knoxd.settings

0,30 * * * * /usr/local/bin/python2.5 /home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/update_feeds.py > /home/pbeeson/cron.log 2>&1
0 12 * * 7 python /home/pbeeson/webapps/django/knoxd/apps/aggregator/bin/delete_old_feeditems.py > /home/pbeeson/cron.log 2>&1
9,29,49 * * * * /home/pbeeson/webapps/django/apache2/bin/start > /dev/null 2>&1

I've already done it for you.

Regards,

~Chris Sebastian

stranger1 · 2008-05-26 23:30:11

Hey Patrick, I created a python file where I imported all my django settings and various django models and called the sync functions in it. In the crontab I just ran 'python sync.py'. This will avoid the DJANGO settings problem.

Also when I try to sync my flickr, twitter and brightkite, it runs fine. But twitter fails sonetimes due to the overtraffic. Is there a way to get an email when the cronjob creates errors. Sounds too much but wondering if it possible.

Regards,

patrickbeeson · 2008-05-28 06:31:42

I put in a ticket with the same question about getting an email of a cron output. Here's what I got:

You'll need to direct stderr to a script that sends the email, eg:

1 * * * * python /home/pbeeson/webapps/django/project/update_feeds.py > /home/pbeeson/cron.log 2>/home/pbeeson/email_script.py

email_script.py could be something like:

#! /usr/bin/python2.5
import sys
import smtplib
server = smtplib.SMTP('smtp5.webfaction.com')
server.login('mailbox_name', 'mailbox_password')

msg = (
"To: you@yourdomain.com\r\n"
"From: you@yourdomain.com\r\n"
"Subject: Cron job error\r\n"
"Content-type: text/plain\r\n"
"\r\n"
"%s") % sys.argv[1]

server.sendmail('you@yourdomain.com', 'you@yourdomain.com', msg)
server.quit()

stranger1 · 2008-05-29 12:36:59

Fantastic.. I thought its not possible. Thanks patrick for sharing this. I wrote the same email script using gmail.
Now I observed that when my cron script ran. The output of the cron is overwriting my email_script. I observed that the content in my email_script disappeared. This is because its overwriting my email_script.

...............sync.py > /home/yashh/cron.out > /home/yashh/email_script.py

Is this the right format or should the emailscript be on another line

Last edited by stranger1 (2008-05-29 13:11:24)

stranger1 · 2008-05-30 02:42:52

Ok Finally got it working. I could draw some inspiration from your approach and make one similar one. I am putting the output of cron job in cronresults.txt and then in my email_script.py I read the cronresults. If any data exists then mail me the content else do nothing. Its working. Thank you for the idea..

SpyderFCS · 2009-02-19 13:07:36

I setup my crontab to run a updater.py script I have. However, I have two completely separate projects that I am running via two individual django w/ mod_wsgi apps. Since they are seperate, how can I have a cron job that uses different PYTHONPATHS and Django_Settings_Modules for each script?

seanf · 2009-02-19 13:55:24

SpyderFCS wrote:
I setup my crontab to run a updater.py script I have. However, I have two completely separate projects that I am running via two individual django w/ mod_wsgi apps. Since they are seperate, how can I have a cron job that uses different PYTHONPATHS and Django_Settings_Modules for each script?

Preface your commands with the environment variables, eg:

Code:

0,30 * * * * PYTHONPATH=/home/you/webapps/django:/home/you/webapps/django/lib/python2.5 DJANGO_SETTINGS_MODULE=myproject.settings /usr/local/bin/python2.5 /home/you/path/to/your/script.py > /dev/null 2>&1
0,45 * * * * PYTHONPATH=/home/you/webapps/django2:/home/you/webapps/django2/lib/python2.5 DJANGO_SETTINGS_MODULE=myproject2.settings /usr/local/bin/python2.5 /home/you/path/to/your/other/script.py > /dev/null 2>&1

#1 2008-02-21 17:41:27

Cron + Django Script

Code:

Code:

#2 2008-02-21 22:45:32

Re: Cron + Django Script

Code:

#3 2008-02-21 22:55:58

Re: Cron + Django Script

#4 2008-02-22 00:25:11

Re: Cron + Django Script

#5 2008-02-22 15:06:30

Re: Cron + Django Script

#6 2008-02-22 15:17:58

Re: Cron + Django Script

Code:

#7 2008-05-11 19:45:01

Re: Cron + Django Script

#8 2008-05-11 20:18:45

Re: Cron + Django Script

#9 2008-05-26 23:30:11

Re: Cron + Django Script

#10 2008-05-28 06:31:42

Re: Cron + Django Script

#11 2008-05-29 12:36:59

Re: Cron + Django Script

#12 2008-05-30 02:42:52

Re: Cron + Django Script

#13 2009-02-19 13:07:36

Re: Cron + Django Script

#14 2009-02-19 13:55:24

Re: Cron + Django Script

SpyderFCS wrote:

Code:

Board footer