Do python scripts that update things online need a virtual environment to run?
I have been trying for a while to schedule a python script in cron to run twice a day. The script uses a couple API's to pull information from a website and update a google spreadsheet with the information. The script is working- I am able to run it manually without a problem, but when I schedule it to run from a cron job I don't see any output.
After a bit of troubleshooting I came across this page -
I went through all the suggested steps and got to the bottom where it says:
"For python you might find that your web app is using a virtual environment you need to invoke in your crontab."
How do I "invoke a virtual environment" in ubuntu? I am just at a point now where I don't know how to proceed. If it helps I am running my script off of a raspberry pi 3b on Ubuntu 20.04.3 LTS and my script is using the Googlesheets API.
EDIT: Things I've done so far to troubleshoot:
My user has permissions to run cron
I am using direct paths in my cron job
I've checked chron guru to make sure my cron schedule was set correctly and it is.
I get output from a cron job that prints the date and time to a file (
* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt) so I know that's working.I've given my script executable permissions with
chmod +x Script.pyI've made sure my user has permission to run the script
I've made sure there is a blank line at the end of the cron file
I've created an executable shell file that executes the script
I've made sure that cron is using the same time as the system time
tail -f /var/log/syslogshows output for my script at the scheduled times but when I check if my spreadsheet was updated it isn'tI've added shebang at the top of my python script
1 Answer
Python virtual environments are self-contained directory trees that contain a Python installation for a particular version of Python, plus a number of additional packages.
Let's say your script uses the "requests" package to make HTTP requests. If you installed requests in a virtual environment, then you will need to make sure the virtual environment is active when your script runs from cron. Otherwise you will get ImportError: No module named requests.
On the other hand, if you installed requests system-wide, and are not using virtual environments, then you also don't need to worry about them in cron context.
Here's what I would do to troubleshoot your particular problem: I would add logging statements in in the script to check assumptions and hypotheses: does the script start at all? Does it run fully? If it makes HTTP requests, what HTTP statuses is it getting back?
Perhaps the script throws an exception, but the exception doesn't get logged anywhere. This snippet only captures stdout:
* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt Change it to also capture stderr:
* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt 2>&1
