Jobs
Generalities
We mainly distinguish different job types and group all tasks in the below categories. Then decide the order of categories (not jobs) when is desired to run each.
- Backups
- Maintenances
- Reporting
- Automations (triggered in some way)
- Group tasks/jobs in above categories
- Decide on a sort order (when to run each category)
- Distribute the categorie thru the time windows for batch Jobs
Then, for each category
- Sort the tasks inside the category
- Estimate run time window
- Identify the master task/job (i.e. backup for dumping jobs, logrotate for the reporting), the job that makes no sense run other tasks after it.
Tasks
- For progress and partial results, use some logging
- Buffer Stdout to generate a report
- Left STDERR to be catched by cron
- Use least privileged user if possible
Our Schedule
Reporting
From 00:00 to 00:30. The master job is logrotate, which rotates logs and so archives the data.
- logwatch
- awstats
- anteater
- pflogsumm
- git report branches (master-independent)
- MASTER logrotate / binnacle DB log rotate
Backups
From 01:00 to 02:00 The master jobs are the rsnapshot set, as it backups the data and dumps generated into the archive.
- autogit (each 6 hours)
- MySQL dumps
- LDAP dumps
- Redis dumps
- MASTER bakSnapshot
Maintenances
From 04:00 to 06:00
- Deploy code and config
- Package updates
- MASTER Maintenances
Current Schedule
Time | Frequency | Host | Group | User | Task |
---|---|---|---|---|---|
00:00 | daily | continuity | ldap-backup | root | /usr/local/lib/git/autogit.sh -d /var/backups/ldap /usr/sbin/ldap-git-backup |
00:00 | daily | continuity | php | root | [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi |
00:00 | daily | continuity | transmit | debian-transmission | /usr/local/lib/transmission/queueManager.sh &>/dev/null |
00:00 | daily | idoru | backup | jack | /usr/local/lib/git/autogit.sh -d ~/dev-bak /usr/local/lib/backup/bakDev.sh 2>/dev/null |
00:00 | daily | rydell | php | root | [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi |
00:00 | daily | wigan | php | root | [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi |
00:01 | daily | continuity | lady | root | /usr/sbin/logrotate /etc/logrotate.conf > /dev/null |
00:01 | daily | idoru | lady | root | /usr/sbin/logrotate /etc/logrotate.conf > /dev/null |
00:01 | daily | laney | lady | root | /usr/sbin/logrotate /etc/logrotate.conf > /dev/null |
00:01 | daily | wigan | lady | root | /usr/sbin/logrotate /etc/logrotate.conf > /dev/null |
00:05 | daily | idoru | backup | jack | /usr/local/lib/git/autogit.sh -d ~/srv-bak /usr/local/lib/backup/bakSrvCnfg.sh -fd"" |
00:10 | daily | continuity | DNS-maintenance | jack | /usr/local/lib/git/autogit.sh -d ~/dns-zones/ /usr/local/lib/backup/bakDNS.sh 2>/dev/null |
00:15 | daily | wigan | awstats | root | /usr/local/awstats/tools/awstats_updateall.pl -awstatsprog=/srv/sites/common/private/awstats/cgi-bin/awstats.pl now &>> /var/log/awstats.log |
00:16 | weekly | rydell | lady | root | /usr/sbin/logrotate -f /etc/logrotate.conf > /dev/null |
00:17 | daily | rydell | apticron | root | if test -x /usr/sbin/apticron; then /usr/sbin/apticron --cron; else true; fi |
00:24 | daily | idoru | apticron | root | if test -x /usr/sbin/apticron; then /usr/sbin/apticron --cron; else true; fi |
00:29 | daily | wigan | apticron | root | if test -x /usr/sbin/apticron; then /usr/sbin/apticron --cron; else true; fi |
00:31 | daily | continuity | apticron | root | if test -x /usr/sbin/apticron; then /usr/sbin/apticron --cron; else true; fi |
00:56 | daily | laney | apticron | root | if test -x /usr/sbin/apticron; then /usr/sbin/apticron --cron; else true; fi |
01:01 | yearly | idoru | backup | root | /usr/local/lib/backup/bakSnapshot.sh yearly |
01:04 | monthly | idoru | backup | root | /usr/local/lib/backup/bakSnapshot.sh monthly |
01:07 | weekly | idoru | backup | root | /usr/local/lib/backup/bakSnapshot.sh weekly |
01:10 | daily | idoru | backup | root | /usr/local/lib/backup/bakSnapshot.sh daily |
02:00 | daily | continuity | lady | root | /usr/local/sbin/syscare.sh -d -m count.zero@l3jane.net |
02:00 | daily | idoru | lady | root | /usr/local/sbin/syscare.sh -d -m count.zero@l3jane.net |
02:00 | daily | laney | lady | root | /usr/local/sbin/syscare.sh -d -m count.zero@l3jane.net |
02:00 | daily | rydell | lady | root | /usr/local/sbin/syscare.sh -d -m count.zero@l3jane.net |
02:00 | daily | wigan | lady | root | /usr/local/sbin/syscare.sh -d -m count.zero@l3jane.net |
02:10 | weekly | continuity | lady | root | /usr/local/sbin/seccheck.sh |
02:10 | weekly | idoru | lady | root | /usr/local/sbin/seccheck.sh |
02:10 | weekly | laney | lady | root | /usr/local/sbin/seccheck.sh |
02:10 | weekly | rydell | lady | root | /usr/local/sbin/seccheck.sh |
02:10 | weekly | wigan | lady | root | /usr/local/sbin/seccheck.sh |
03:00 | daily | continuity | transmit | kodi | /usr/local/lib/kodi/task.sh |
03:05 | monthly | wigan | git-maintenance | jack | /usr/local/lib/git/branchReport.sh | mail -s Branch report" jack" |
03:10 | daily | continuity | e2scrub_all | root | test -e /run/systemd/system || SERVICE_MODE=1 /sbin/e2scrub_all -A -r |
03:10 | daily | rydell | e2scrub_all | root | test -e /run/systemd/system || SERVICE_MODE=1 /sbin/e2scrub_all -A -r |
03:30 | weekly | continuity | e2scrub_all | root | test -e /run/systemd/system || SERVICE_MODE=1 /usr/lib/x86_64-linux-gnu/e2fsprogs/e2scrub_all_cron |
03:30 | weekly | rydell | e2scrub_all | root | test -e /run/systemd/system || SERVICE_MODE=1 /usr/lib/x86_64-linux-gnu/e2fsprogs/e2scrub_all_cron |
04:00 | monthly | wigan | git-maintenance | git | /usr/local/lib/git/repCheck.sh | mail -s Git Maintenance" jack" |
04:05 | daily | rydell | docker | jack | docker container prune -f --filter until=48h |
04:10 | weekly | rydell | docker | jack | docker network prune -f --filter until=24h |
04:20 | monthly | rydell | docker | jack | docker system prune -f --filter until=168h |
04:30 | weekly | rydell | docker | jack | docker volume prune -f |
04:40 | weekly | idoru | mysql | root | cd /srv/sql-dump/; /usr/local/lib/backup/bakMySQL.sh -z -i itop,kanbandb,zabbix minerals,syslog | logger -p uucp.info |
05:00 | monthly | continuity | DNS-maintenance | root | /usr/local/sbin/dns-update.sh |
07:00 | daily | wigan | kanban | www-data | cd /srv/sites/www/kanban/ && ./cli cronjob &>/dev/null |
17:09 | daily | rydell | popularity-contest | root | test -x /etc/cron.daily/popularity-contest && /etc/cron.daily/popularity-contest --crond |
23:00 | daily | idoru | SSD | root | /sbin/fstrim -v / 2>&1 | /usr/bin/logger -p news.info -t FStrim |
23:00 | daily | idoru | SSD | root | /sbin/fstrim -v /srv/sql/ 2>&1 | /usr/bin/logger -p news.info -t FStrim |
23:50 | daily | continuity | lady | root | /usr/local/bin/deploy.sh -a lady/ops/hostconfig /etc/ |
23:50 | daily | idoru | lady | root | /usr/local/bin/deploy.sh -a lady/ops/hostconfig /etc/ |
23:50 | daily | laney | lady | root | /usr/local/bin/deploy.sh -a lady/ops/hostconfig /etc/ |
23:50 | daily | rydell | lady | root | /usr/local/bin/deploy.sh -a lady/ops/hostconfig /etc/ |
23:50 | daily | wigan | lady | root | /usr/local/bin/deploy.sh -a lady/ops/hostconfig /etc/ |
23:55 | daily | continuity | lady | root | /usr/local/bin/deploy.sh -a lady/dev/shell /usr/local |
23:55 | daily | continuity | ldap-backup | root | ( cd /etc/ldap/bak/; /usr/local/lib/backup/bakLDAP.sh ) |
23:55 | daily | idoru | lady | root | /usr/local/bin/deploy.sh -a lady/dev/shell /usr/local |
23:55 | daily | laney | lady | root | /usr/local/bin/deploy.sh -a lady/dev/shell /usr/local |
23:55 | daily | rydell | lady | root | /usr/local/bin/deploy.sh -a lady/dev/shell /usr/local |
23:55 | daily | wigan | binnacle | www-data | /usr/bin/php /srv/sites/www/ignit/binnacle.php manager logrotate | logger -p syslog.info |
23:55 | daily | wigan | lady | root | /usr/local/bin/deploy.sh -a lady/dev/shell /usr/local |
23:57 | daily | continuity | lady | root | dpkg --get-selections > /etc/selections.dpkg |
23:57 | daily | idoru | lady | root | dpkg --get-selections > /etc/selections.dpkg |
23:57 | daily | laney | lady | root | dpkg --get-selections > /etc/selections.dpkg |
23:57 | daily | rydell | lady | root | dpkg --get-selections > /etc/selections.dpkg |
23:57 | daily | wigan | lady | root | dpkg --get-selections > /etc/selections.dpkg |
23:59 | daily | rydell | sysstat.dpkg-dist | root | command -v debian-sa1 > /dev/null && debian-sa1 60 2 |