Cron jobs not running on Kubernetes + Alpine linux
I am trying to run a cron job on Alpine Linux docker image which is then pushed in Kubernetes container. The Cron job is that I want the container to restart itself after every 15 mins. So my command is pkill processname.
Little bit about Alpine Cron jobs: For Alpine Linux cron is followed periodically that is it has already created folders /15min. /hourly. /daily. /weekly. /monthly. So I have to just copy my script in one of the folders to run it.
I created the script and as requested on Alpine FAQ's pages I didn't give any name extension to the file and copied it with execute permissions in the folder. When I try and run command run-parts --test /etc/periodic/15min it shows me that it has one script that needs to be executed. Although that script never executes. When I try and run that docker image on my local machine, not exactly with that script but a different one where I am just creating a file in the folder, it works well.
I have also tried and run crond -f -l 8 on Kube cluster and ti still does not work.
I would really appreciate help on this part what I am missing or where is it going wrong.
71 Answer
Wellcome to SuperUser.
First of all, maybe you're facing it the wrong way. If the goal is to restart the complete container every 15 minutes, maybe you should check the links bellow.
Question about restarting pods with multiple possible solutions
Kubernetes concepts - Explanation about how kubernetes works and the names people uses for things there.
Just for completity, and in case you actually want to run a periodic task, check specific Kubernetes docs, they state how to create cronjobs and debug them. I'd also suggest to read about Kubernetes limitations in documentation, as there may be something actively preventing to run your cronjobs
GCP Documentation about cronjobs
Kubernetes documentation about Cronjobs
I'm giving you a resume about how to create a cronjob task, but you should carefully read the above if you're not familiar with kubernetes and follow the links inside, they've environments ready to play and learn the basics of kubernetes.
It is more elaborated inside, but it basically gives you an example command and the way to deploy it. I've modified it to be once every 15 minutes and run a pkill. You've to create a yaml file with the following contents (check syntax for more complex cron behaviours):
# cronjob.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata: name: hello
spec: schedule: "*/15 * * * *" jobTemplate: spec: template: spec: containers: - name: hello image: busybox args: - /bin/sh - -c - pkill yourprocess restartPolicy: OnFailureand then deploy it using
kubectl apply -f [FILENAME]Use the following command to check if your cronjob was deployed:
kubectl describe cronjob [CRON_JOB]Post describing the above processes with more screenshots and examples
After this point this is a very generic answer. You can post comments and I'll try to help you out, but I currently have no access to an Alpine Linux installation, so I may not be helpful. This is the Alpine Linux documentation page addressing your issue
Tracing your error: The simple way
There's a very simple way to narrow down whats going out, although it is not very precise. There are two chances here: There's a problem with CRON itself, or you have a problem with your script (permissions, for instance). The process is simple:
- Create a task that creates an empty file inside a folder where you're ABSOLUTLY SURE everyone can read and write.
- If the new file appears when the task is suposed to run, it means that your cron is executing properly and that you have a problem with your script. Go to "Script problems" section. If not, you've a problem with cron. Go to Cron problems section
Tracing your error: Logs
The best way to solve problems in any system is looking at the logs. Software like cron normally stores logs on syslog (normally located at /var/log/syslog), unless configured otherwise. This post speaks about cron logging folders.
If you find out where your cron instance is logging and there's nothing there, it means that your cron service is not running at all. There's a Alpine Linux specific solution for this problem in Cron problems. If you can see errors, you should look at them and try to narrow down your problem. If not, you should make sure that your script is actually not running.
Cron problems
Ok, now you know cron is not working properly. There are various possible scenarios:
- Cron is not running at all
Maybe the cron service is not started. Check if the service is running. If it is not, try to start it manually. Check if it has started properly and then add it to startup process. The way to do so varies depending on the OS. In Alpine Linux, this should do the trick:
rc-service crond start && rc-update add crond- The cron folder you're using is not being used by cron
You should check the main cron config file in order to verify that the folder you're putting scripts on is actually added.
- Typos
Maybe the config line has some error with spaces on arguments or things like that, although in this case this seem not to be the scenario, as you used test commands on your folder.
Script problems
If your script runs but it is not producing the desired output, it is a good idea to perform the following steps:
- Separate your script into a file. Give it run permissions and ensure that the first line properly states the shell to use, in Alpine's case, this line should be #!/bin/sh
- Redirect the standard and error output to a file to analize what's going on.
General suggestions
- As stated in Script problems section, it is a good practise to separate your oneliners into script files and point their output to files, in order to be able to debug problems of all kinds.
- Take a close look at directory and file permissions (those directly accesed by your script and those that cron should read / user in order to execute your scripts), it is a common cause of failure, specially in containers.
EDIT: Maybe I understood your situation in the wrong way and you're in an scenario like this one
5
