Ticket resolution report for Incidents

20 Apr - 22 May

RT Created Closed Resolution time Client Subject
57958 20 Apr, 00:00 20 Apr, 10:01 0.20d Internal Check for Security Notices and System Updates
57963 20 Apr, 09:16 20 Apr, 10:39 0.14d AskBio WARNING: Security: Let's Encrypt Certificates on darkmatter.synpromics.com
57954 17 Apr, 21:37 20 Apr, 11:11 0.32d BD Group EFSRV-21761 eFiling high load
57919 13 Apr, 12:16 20 Apr, 12:35 5.03d Imoves Investigate PHP-related issues with staging site
57961 20 Apr, 08:21 20 Apr, 13:02 0.47d Reigate College rc-mss.reigate.ac.uk root volume usage
57960 20 Apr, 08:15 20 Apr, 15:41 0.74d Troika CRITICAL: System Health: Systemd State on tron.troika.lan
57970 21 Apr, 00:00 21 Apr, 08:42 0.07d Internal Check for Security Notices and System Updates
57973 21 Apr, 10:49 21 Apr, 11:33 0.07d Internal CRITICAL: Backups: Prebackup for mersey.tiger-computing.co.uk on aql-svr-zabbix-app.quickline.co.uk
57974 21 Apr, 11:31 21 Apr, 14:12 0.27d Troika UNKNOWN: Hardware: IPMI on tron.troika.lan
57975 21 Apr, 13:08 21 Apr, 15:23 0.23d OGT Storage alert: /srv/illumina_data on rodney.internal.ogtip.com
57976 21 Apr, 15:43 21 Apr, 15:54 0.02d NFW WARNING: Mail: Exim Panic Log on nfwgs2.nfw.internal
57959 20 Apr, 08:12 21 Apr, 17:13 1.90d OGT rodney1.internal.ogtip.com failed disk
57982 22 Apr, 07:30 22 Apr, 08:55 0.09d Sucden globalstar01/02.sucden.co.uk failed services
57967 20 Apr, 14:18 22 Apr, 10:04 1.58d Tangram tangram/prod pods in non ready state
57964 20 Apr, 09:22 22 Apr, 10:48 2.14d Trident Honda WARNING: Cluster: Csync2 on vmsvr2.trident.lan
57981 22 Apr, 00:00 22 Apr, 11:18 0.33d Internal Check for Security Notices and System Updates
57996 23 Apr, 00:00 23 Apr, 08:56 0.09d Internal Check for Security Notices and System Updates
57997 23 Apr, 07:58 23 Apr, 10:40 0.27d OGT rodney1 reboot/stonith problem
57998 23 Apr, 08:04 23 Apr, 10:45 0.27d Quickline aql-svr-zabbix-app.quickline.co.uk backups to mersey problem
57993 22 Apr, 17:12 23 Apr, 12:04 0.49d Tangram [FIRING: 1 for tangram/prod] KubeJobFailed in gitlab
57994 22 Apr, 17:16 23 Apr, 12:12 0.49d Tangram [FIRING: 1 for tangram/stage] ThanosCompactHalted in tcl-monitoring
57999 23 Apr, 11:16 23 Apr, 12:50 0.16d AskBio AskBio outage
58000 23 Apr, 12:29 23 Apr, 13:30 0.10d Tangram [FIRING: 1 for tangram/prod] KubePersistentVolumeFillingUp in tcl-monitoring
58001 23 Apr, 15:11 23 Apr, 15:24 0.02d Troika WARNING: Puppet: Checking-in on tron.troika.lan
58007 24 Apr, 00:00 24 Apr, 09:19 0.13d Internal Check for Security Notices and System Updates
58008 24 Apr, 08:25 24 Apr, 09:46 0.13d Tangram CRITICAL: AWS Site-to-Site VPN: Tunnel2 on dgx1.internal.etherapeutics.co.uk
57972 21 Apr, 10:15 24 Apr, 10:27 3.02d Sucden Connectivity Issue from IXN and PPS and GS2 to Azure.
58009 24 Apr, 10:56 24 Apr, 14:52 0.39d AskBio WARNING: NRPE Disk: / on edb-lab-skye.askbio.com
58022 27 Apr, 00:00 27 Apr, 09:23 0.14d Internal Check for Security Notices and System Updates
58020 26 Apr, 11:19 27 Apr, 09:47 0.18d Sucden equistar03/04 megastar03/04 NTP Timeouts
58006 23 Apr, 17:24 27 Apr, 11:40 1.43d AskBio Darkmatter ssh oddity
58059 27 Apr, 12:30 27 Apr, 12:33 0.01d NFW DOWN: nfwgs2.nfw.internal
58015 24 Apr, 15:42 27 Apr, 17:14 1.15d Tangram [FIRING: 1 for tangram/stage] ThanosCompactHalted in tcl-monitoring
58071 28 Apr, 08:05 28 Apr, 08:32 0.05d Midnight Hax CRITICAL: NRPE Disk: All Disks on woodlands.midnighthax.com
58070 28 Apr, 00:00 28 Apr, 09:27 0.15d Internal Check for Security Notices and System Updates
58072 28 Apr, 08:35 28 Apr, 09:52 0.13d Koinex DOWN: koinexserver.home
58075 28 Apr, 09:09 28 Apr, 10:43 0.16d Gravytrain CRITICAL: HTTP: www.rightsure.co.uk HTTPS on web.prod.bf2.bedfordinsurance.co.uk
58060 27 Apr, 12:50 28 Apr, 14:36 1.18d Global Brands Printer issue
58076 28 Apr, 12:36 28 Apr, 17:03 0.44d AskBio exim4 not starting on coll,jura
57962 20 Apr, 08:54 28 Apr, 17:03 6.82d Cirencester College ccolive.cirencester.local storage volume usage /var
58024 27 Apr, 10:06 29 Apr, 08:12 1.81d Tangram tangram/prod pods in non ready state
58082 29 Apr, 08:15 29 Apr, 08:27 0.02d Midnight Hax woodlands.midnighthax.com high storage volume usage /home
58081 29 Apr, 00:00 29 Apr, 08:37 0.06d Internal Check for Security Notices and System Updates
58086 29 Apr, 12:18 29 Apr, 12:33 0.03d Internal [FIRING: 1 for tcl/avon] KubeJobFailed in nextcloud
58085 29 Apr, 11:44 29 Apr, 14:53 0.31d AskBio talisker.synpromics.com root / volume usage
57944 16 Apr, 10:20 29 Apr, 16:58 9.66d AskBio ZFS pool data fragmentation on talisker.synpromics.com
58089 30 Apr, 00:00 30 Apr, 11:20 0.33d Internal Check for Security Notices and System Updates
58134 30 Apr, 12:18 30 Apr, 15:34 0.33d Internal Green monitoring check showing as a problem in Icinga
58136 01 May, 00:00 01 May, 09:14 0.12d Internal Check for Security Notices and System Updates
58078 28 Apr, 14:51 01 May, 10:07 2.53d TCV RT autoreply issues
58019 26 Apr, 02:27 01 May, 11:23 4.34d Top To Toe Re: STILL DOWN: toptotoe-staging.net is still offline ([28] Timeout was reached (Operation timed out after 30001 milliseconds with 0 bytes received))
58093 30 Apr, 09:53 01 May, 14:51 1.50d AskBio Linux vulnerability: Copy fail (CVE-2026-31431)
58109 30 Apr, 10:00 01 May, 15:49 1.58d Koinex Linux vulnerability: Copy fail (CVE-2026-31431)
58141 01 May, 16:21 01 May, 17:08 0.08d Sucden DOWN: patrs-gs2-p1l01.sucden.co.uk
58146 04 May, 00:00 05 May, 08:58 0.10d Internal Check for Security Notices and System Updates
58095 30 Apr, 09:54 05 May, 09:28 1.96d Azura Linux vulnerability: Copy fail (CVE-2026-31431)
58154 05 May, 08:21 05 May, 09:53 0.15d Globecast gcukscheddbdev and gcukscheddb01 insights client failure
58153 05 May, 08:18 05 May, 11:33 0.32d Gravytrain prod.gt1.grvytrn.co.uk certbot renew failure
58140 01 May, 16:19 05 May, 12:17 0.60d Midnight Hax CRITICAL: Disk: fstab mounts on woodlands.midnighthax.com
58144 02 May, 20:01 05 May, 14:39 0.67d Sucden CRITICAL: System Health: Systemd State on megastar04.sucden.co.uk
58096 30 Apr, 09:55 05 May, 15:25 2.55d BD Group Linux vulnerability: Copy fail (CVE-2026-31431)
58104 30 Apr, 09:58 05 May, 16:25 2.65d Globecast Linux vulnerability: Copy fail (CVE-2026-31431)
58107 30 Apr, 09:59 06 May, 10:44 3.08d ITC Compliance Linux vulnerability: Copy fail (CVE-2026-31431)
58111 30 Apr, 10:01 06 May, 10:46 3.08d Midnight Hax Linux vulnerability: Copy fail (CVE-2026-31431)
58169 06 May, 08:13 06 May, 11:00 0.28d Crystal Units crylonlis2024.crystal.local backups to thames duration
58166 06 May, 00:00 06 May, 11:16 0.33d Internal Check for Security Notices and System Updates
58113 30 Apr, 10:01 06 May, 11:42 3.17d Novadata Linux vulnerability: Copy fail (CVE-2026-31431)
58127 30 Apr, 10:08 06 May, 16:55 3.68d Trident Honda Linux vulnerability: Copy fail (CVE-2026-31431)
58160 05 May, 11:12 07 May, 09:25 1.82d Sucden strlb-gs2-p1l02.sucden.co.uk failing configuration
58179 07 May, 00:00 07 May, 09:59 0.20d Internal Check for Security Notices and System Updates
58172 06 May, 15:10 07 May, 10:13 0.50d Crystal Units Fw: can't connect lisec server
58180 07 May, 00:06 07 May, 10:42 0.27d Crystal Units backy2 cleanup failure | invalid database
58167 06 May, 00:05 07 May, 10:43 1.27d Crystal Units Backy2 cleanup failed on crylonlis001.crystal.local
58161 05 May, 12:21 07 May, 10:44 1.84d Crystal Units FW:
58157 05 May, 08:47 07 May, 10:52 2.21d Crystal Units crylonlis2024.crystal.local insights client failure
58150 04 May, 10:26 07 May, 17:01 2.90d AskBio login to RTP server
58083 29 Apr, 09:59 07 May, 17:16 5.73d Lantech ipa-m3.private.ipa.ie root storage volume usage
58064 27 Apr, 14:42 07 May, 17:20 7.26d OGT rodney.internal.ogtip.com volume usage /srv/ngs
58192 08 May, 00:00 08 May, 09:02 0.10d Internal Check for Security Notices and System Updates
58198 08 May, 10:59 08 May, 11:06 0.01d NFW CRITICAL: System Health: Systemd State on nfwgs2.nfw.internal
58197 08 May, 10:37 08 May, 11:07 0.05d TCV web4.tcv.org.uk HTTP/S checks timing out
58191 07 May, 17:46 08 May, 11:09 0.34d DLS Plastics DOWN: dls-zimbra2.dlsmedical.internal
58181 07 May, 03:08 08 May, 11:10 1.32d Internal Websites down on wey.tiger-computing.co.uk
58171 06 May, 14:08 08 May, 11:24 1.73d Sucden Storage alert: / on strlb-gs2-p1l01.sucden.co.uk > 90%
58187 07 May, 14:29 08 May, 17:20 1.29d BD Group eFiling: File Access Error
58212 11 May, 00:00 11 May, 09:28 0.15d Internal Check for Security Notices and System Updates
58217 11 May, 09:30 11 May, 11:52 0.24d OGT DOWN: damien.internal.ogtip.com
58214 11 May, 08:37 11 May, 12:32 0.39d Midnight Hax fw.midnighthax.com volume usage /boot
58219 12 May, 00:00 12 May, 09:24 0.14d Internal Check for Security Notices and System Updates
58220 12 May, 08:04 12 May, 10:51 0.28d Internal icinga error
58205 08 May, 14:56 12 May, 11:03 1.61d AskBio Storage alert: /home on talisker.synpromics.com
58229 13 May, 10:12 13 May, 10:29 0.03d Tangram CRITICAL: System Health: Systemd State on dgx1.internal.etherapeutics.co.uk
58124 30 Apr, 10:07 13 May, 10:41 8.06d TCV Linux vulnerability: Copy fail (CVE-2026-31431)
58132 30 Apr, 11:27 13 May, 10:48 7.94d Sucden Linux kernel security update (CVE-2026-31431)
58225 12 May, 23:12 13 May, 11:02 0.30d AskBio rtp-promptdv-01 is down
58226 13 May, 00:00 14 May, 09:13 1.12d Internal Check for Security Notices and System Updates
58213 11 May, 08:31 14 May, 10:56 3.24d OGT CRITICAL: System Health: Log Rotation on basil.internal.ogtip.com
58193 08 May, 08:13 14 May, 11:18 4.31d Internal dirtyfrag exploit
58224 12 May, 16:49 14 May, 12:28 1.57d AskBio Cristina
58244 14 May, 12:08 14 May, 13:51 0.17d TCV web4/mis.tcv.org.uk /boot volume filling up
58247 15 May, 00:00 15 May, 08:58 0.10d Internal Check for Security Notices and System Updates
58100 30 Apr, 09:56 15 May, 09:23 9.94d Econic Bio Linux vulnerability: Copy fail (CVE-2026-31431)
58257 15 May, 14:19 15 May, 14:52 0.06d OGT blaney.internal.ogtip.com tomcat cron failure
58269 18 May, 00:00 18 May, 10:03 0.21d Internal Check for Security Notices and System Updates
58270 18 May, 08:28 18 May, 10:59 0.25d Troika tron.troika.lan volume usage /mnt/pve/troika_vm_v2
58275 18 May, 10:35 18 May, 11:41 0.11d TCV Storage alert: /var on web4.tcv.org.uk
58276 18 May, 10:37 18 May, 14:54 0.43d OGT Storage alert: /srv/illumina_data on rodney.internal.ogtip.com
58255 15 May, 10:42 18 May, 15:02 1.43d Lantech ClamAV warnings this morning (15/05/2026)
58230 13 May, 10:12 18 May, 16:58 3.68d AskBio Backup failure on macleod.synpromics.com
58284 19 May, 00:00 19 May, 08:45 0.08d Internal Check for Security Notices and System Updates
58278 18 May, 10:57 19 May, 10:03 0.91d AskBio /tmp cleanup script errors on talisker
58091 30 Apr, 08:53 19 May, 10:04 12.12d Internal CRITICAL: Backups: Backup to woodlands.midnighthax.com on mersey.tiger-computing.co.uk
58251 15 May, 08:19 19 May, 11:18 2.30d OGT basil.internal.ogtip.com volume usage /data
58282 18 May, 16:17 19 May, 11:37 0.53d OGT Restart Docker service on Basil
58110 30 Apr, 10:00 19 May, 12:09 12.21d Lantech Linux vulnerability: Copy fail (CVE-2026-31431)
58228 13 May, 10:12 19 May, 12:31 4.23d Gravytrain Certbot failed on app.prod.qs1.grvytrn.co.uk
58285 19 May, 11:00 19 May, 13:57 0.29d Troika Unmerged config file on tron.troika.lan
58290 20 May, 00:00 20 May, 09:10 0.12d Internal Check for Security Notices and System Updates
58155 05 May, 08:31 20 May, 09:16 11.07d Quickline aql-svr-zabbix-db.quickline.co.uk Zabbix partitioning less than 10 days
58287 19 May, 11:06 20 May, 10:06 0.90d Crystal Units Storage utilisation warning: /backup on crylonlis2024.crystal.local
58296 20 May, 09:24 20 May, 10:15 0.09d Internal WARNING: Backups: BackupPC directories (NRPE) on mersey.tiger-computing.co.uk
58293 20 May, 08:14 20 May, 12:33 0.43d OGT mickey/granddad.internal.ogtip.com configuration failure
58292 20 May, 02:35 20 May, 12:38 0.46d Internal DOWN: trent.tiger-computing.co.uk
58271 18 May, 08:50 20 May, 13:36 2.48d NFW nfwgs2.nfw.internal volume usage /home
58305 21 May, 00:00 21 May, 09:31 0.15d Internal Check for Security Notices and System Updates
58308 21 May, 10:21 21 May, 10:57 0.06d Geograph geograph/prod node not ready
58291 20 May, 02:28 21 May, 11:16 1.33d AskBio CRITICAL: Networking: SSH on rtp-promptdv-01.askbio.com
58307 21 May, 09:18 21 May, 11:17 0.20d Cirencester College ccolive.cirencester.local stars HTTP/HTTPS timeouts
58315 22 May, 00:00 22 May, 09:01 0.10d Internal Check for Security Notices and System Updates
58117 30 Apr, 10:03 22 May, 09:41 14.96d Reigate College Linux vulnerability: Copy fail (CVE-2026-31431)
58273 18 May, 10:26 22 May, 11:59 4.15d OGT Run updates failure on blaney.internal.ogtip.com
58321 22 May, 14:57 22 May, 15:19 0.04d NFW DOWN: nfwgs2.nfw.internal

git hash: master (6c029d07)