Join Log in
Score:0
Pocketmouse avatar Ubuntu

Do python scripts that update things online need a virtual environment to run?

ng flag
Pocketmouse

I have been trying for a while to schedule a python script in cron to run twice a day. The script uses a couple API's to pull information from a website and update a google spreadsheet with the information. The script is working- I am able to run it manually without a problem, but when I schedule it to run from a cron job I don't see any output.

After a bit of troubleshooting I came across this page - https://cronitor.io/cron-reference/cron-troubleshooting-guide

I went through all the suggested steps and got to the bottom where it says:

"For python you might find that your web app is using a virtual environment you need to invoke in your crontab."

How do I "invoke a virtual environment" in ubuntu? I am just at a point now where I don't know how to proceed. If it helps I am running my script off of a raspberry pi 3b on Ubuntu 20.04.3 LTS and my script is using the Googlesheets API.

EDIT: Things I've done so far to troubleshoot:

  • My user has permissions to run cron

  • I am using direct paths in my cron job

  • I've checked chron guru to make sure my cron schedule was set correctly and it is.

  • I get output from a cron job that prints the date and time to a file (* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt) so I know that's working.

  • I've given my script executable permissions with chmod +x Script.py

  • I've made sure my user has permission to run the script

  • I've made sure there is a blank line at the end of the cron file

  • I've created an executable shell file that executes the script

  • I've made sure that cron is using the same time as the system time

  • tail -f /var/log/syslog shows output for my script at the scheduled times but when I check if my spreadsheet was updated it isn't

  • I've added shebang at the top of my python script

73
0 + 0
user535733 avatar
cn flag
user535733
A virtual environment *might* be related to a python script failing to provide expected output. However, there are many common reasons that are more likely. A script that merely fiddles with Googlesheets has no apparent reason to need a virtual environment. I suspect you are on the wrong path. Stick with basic cron and python troubleshooting until you have more definite answers.
1
Reply
Pocketmouse avatar
ng flag
Pocketmouse
Do you have any suggestions on what to try next? I've updated my post with things I've tried already (list was too long to post here in the comments). I am at a bit of a disadvantage here when it comes to Ubuntu as I am not as familiar with it as I am python. I'm having to google everything and so far everything I've found doesn't help.
0
Reply
raj avatar
cn flag
raj
Please note that the article you quoted talks about Python **web app**. Web apps are apps that run on a webserver, and if written in Python they actually often use Python virtual environment (venv). That's not your case. If you just call your script from a command line, it probably does not use venv (you would surely remember creating the venv for it if it were the case :)). It is rather a "classical" cron problem - value of `$PATH` or other variables is different in cron than in interactive shell and because of this your script may be unable to access some modules. Investigate it.
0
Reply
user535733 avatar
cn flag
user535733
Your troubleshooting has demonstrated that there is nothing wrong with cron. 99% of the time, cron is working properly, and you're in that group. Your troubleshooting has also demonstrated that you understand how to use and troubleshoot cron properly (good job!) Therefore, it's time to focus on your script: Add debugging output to your script that will show up in a local log or file. Use the debug output to narrow down which part of your script is problematic.
0
Reply
Score:0
Pēteris Caune avatar Ubuntu
in flag
Pēteris Caune

Python virtual environments are self-contained directory trees that contain a Python installation for a particular version of Python, plus a number of additional packages.

Let's say your script uses the "requests" package to make HTTP requests. If you installed requests in a virtual environment, then you will need to make sure the virtual environment is active when your script runs from cron. Otherwise you will get ImportError: No module named requests.

On the other hand, if you installed requests system-wide, and are not using virtual environments, then you also don't need to worry about them in cron context.

Here's what I would do to troubleshoot your particular problem: I would add logging statements in in the script to check assumptions and hypotheses: does the script start at all? Does it run fully? If it makes HTTP requests, what HTTP statuses is it getting back?

Perhaps the script throws an exception, but the exception doesn't get logged anywhere. This snippet only captures stdout:

* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt

Change it to also capture stderr:

* * * * * /path/to/python3.8 /path/to/script/Script.py >> log.txt 2>&1
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.