15 minute update

02 Mar - 03 Apr

RT Client Subject First work started First post-work reply Time to reply
54100 BD Group CRITICAL: HTTP: formations.elementalcosec.com HTTPS on ef-v5-nodes.c.ef-platform.internal 2025-03-10 19:39:35 None 4h 20m
57640 Trident Honda vmsvr2.trident.lan storage volume usage /var 2026-03-16 10:51:47 None 1d 3h 08m
57703 OGT Storage alert: /srv/ngs on rodney.internal.ogtip.com 2026-03-19 12:22:38 2026-03-19 12:31:44 0h 09m
57638 DLS Plastics DLS Medical site down 2026-03-13 08:49:36 2026-03-13 10:15:30 1h 26m
57639 Tangram Gitlab and lost emails 2026-03-17 10:34:31 2026-03-17 10:51:55 0h 17m
57744 Geograph Geograph Gitlab upgrade (18.10.1) 2026-03-27 08:16:32 2026-03-27 08:17:31 0h 01m
57527 Troika QNAP/Firewall health-checks 2026-03-06 09:54:37 2026-03-06 10:41:51 0h 47m
57482 Globecast DNS updates 2026-03-05 11:05:48 2026-03-05 11:22:23 0h 17m
57769 NFW nfwgs2.nfw.internal samba cleanup script error 2026-04-01 13:00:40 2026-04-01 13:13:30 0h 13m
57671 JISC Upgrade servers to Debian 13 Trixie 2026-03-25 12:03:36 2026-03-25 12:11:22 0h 08m
57567 Sucden Oracle Linux Patching - 60 day cycle 2026-03-09 17:09:34 2026-03-10 14:45:52 2d 1h 36m
57467 OGT manuel.internal.ogtip.com restic scheduling 2026-03-03 12:05:31 2026-03-03 12:40:16 0h 35m
57531 Gravytrain web.prod.bf2.bedfordinsurance.co.uk Multiple HTTP/HTTPS timeouts 2026-03-02 11:57:03 2026-03-02 12:02:49 0h 06m
57707 Azura Azura - Ocado vpn certificates 2026-03-25 14:35:38 2026-03-25 14:44:30 0h 09m
56410 Sucden interstar07-10 2026-03-02 10:24:43 2026-03-02 10:30:10 0h 05m
57533 Tangram Gitlab persistent volume filling up 2026-03-02 13:31:46 2026-03-02 16:19:50 2h 48m
57476 ATA Group Spike on data usage 2026-03-02 11:43:04 None 1d 2h 17m
56752 OpenCorporates OC - Quote to remove all hardware from Sysgroup 2025-12-11 11:47:05 None 1d 2h 13m
57536 OpenCorporates CRITICAL: Networking: SSH on vm-host31.opencorporates.internal 2026-03-02 16:43:41 None 7h 16m
57524 Top To Toe gait-prod.toptotoe.com expired crl 2026-03-03 10:11:50 2026-03-03 10:42:18 0h 30m
57535 Novadata RE: Resolved: NovaData expiring VPN certs 2026-03-03 11:41:48 2026-03-03 11:48:59 0h 07m
57745 JISC Access for new employee 2026-03-31 10:51:59 2026-03-31 11:07:22 0h 15m
57532 AskBio Skye, Talisker, DarkMatter & Macleod outage 2026-03-02 17:40:33 2026-03-02 17:55:31 0h 15m
57615 AskBio Network Outage 5-6pm today 2026-03-10 11:32:36 2026-03-10 11:32:58 0h 00m
57641 BD Group Storage alert: / on ef-v5-* 2026-03-13 16:17:26 2026-03-13 16:26:55 0h 09m
57710 AskBio CRITICAL: Networking: ping4 on macleod.synpromics.com 2026-03-21 00:40:31 2026-03-21 00:48:06 0h 08m
57667 Rotastat gallus.rotastat.com heavy load/timeouts 2026-03-17 11:27:03 2026-03-17 11:29:05 0h 02m
57726 Ingeus Ingeus: login to Azure 2026-03-24 08:42:33 2026-03-24 08:44:37 0h 02m
57627 Geograph Geograph Gitlab upgrade (18.9.2) 2026-03-12 08:19:33 2026-03-12 08:34:18 0h 15m
57691 Top To Toe CRITICAL: HTTP: staging.toptotoe-gait.com on gait-prod.toptotoe.com 2026-03-18 11:47:38 2026-03-18 11:50:31 0h 03m
57693 Rotastat CRITICAL: System Health: Systemd State on gallus.rotastat.com 2026-03-18 12:47:47 2026-03-18 13:09:34 0h 22m
57665 Azura ford1/2 SSL/TLS certificate expiry (customgeophev.com) 2026-03-19 14:28:12 2026-03-19 14:29:43 0h 02m
57561 BD Group CRITICAL: HTTP: www.1stformations.co.uk HTTPS on ef-v5-nodes.c.ef-platform.internal 2026-03-07 02:40:30 2026-03-07 02:47:11 0h 07m
57700 Sucden declan and duffman decom 2026-03-19 13:14:00 2026-03-23 15:38:33 2h 25m
57526 Troika New client onboarding (Pre-Contract Checks): Troika 2026-03-04 16:47:11 2026-03-05 11:36:54 1d 8h 50m
57569 Geograph [FIRING: 1 for geograph/prod] NodeFilesystemSpaceFillingUp in tcl-monitoring 2026-03-09 14:20:48 2026-03-09 16:35:46 2h 15m
57618 Tangram CRITICAL: System Health: Systemd State on dgx1.internal.etherapeutics.co.uk 2026-03-10 17:36:40 2026-03-10 17:38:05 0h 01m
57658 Sucden patrs-gs2-p1l01.sucden.co.uk configuration failure 2026-03-16 16:07:26 2026-03-16 16:19:07 0h 12m
57451 Advensys Arpino Warehouse Site Visit 2026-03-02 20:12:27 2026-03-03 11:05:11 1d 4h 53m
57654 Tangram tangram/prod backup job failure 2026-03-17 09:24:47 2026-03-17 09:58:13 0h 33m
57646 OGT rodney.internal.ogtip.com volume usage /srv/illumina_data 2026-03-17 09:52:52 2026-03-17 10:15:53 0h 23m
57628 Tangram Tangram Gitlab upgrade (18.9.2) 2026-03-12 08:01:14 2026-03-12 08:33:42 0h 32m
57623 Tangram AWS Backup copy jobs failed 2026-03-11 15:32:25 2026-03-11 16:04:07 0h 32m
57632 Sucden duffman.sucden.co.uk mount missing /data/dl/releases 2026-03-12 10:39:25 2026-03-12 10:49:53 0h 10m
57152 Tangram Important kubernetes change - tangram 2026-03-02 11:01:36 2026-03-02 11:05:02 0h 03m
57151 Sysdoc Important kubernetes change - sysdoc 2026-03-18 14:25:56 None 9h 34m
57688 Reigate College rc-mss.reigate.ac.uk /mnt/network/SharedArea/Staff not mounted 2026-03-18 08:09:02 2026-03-18 08:25:08 0h 16m
57783 Reigate College CRITICAL: Networking: SSH on rc-mss.reigate.ac.uk 2026-04-02 10:11:31 2026-04-02 10:17:20 0h 06m
57782 Gravytrain CRITICAL: HTTP: www.bedfordinsuranceforbusiness.co.uk HTTPS on web.prod.bf2.bedfordinsurance.co.uk 2026-04-02 10:05:06 2026-04-02 10:23:11 0h 18m
57713 OGT rodney1.internal.ogtip.com failed disk 2026-03-23 11:25:58 2026-03-23 12:03:10 0h 37m
57723 OGT OOM - blaney.internal.ogtip.com 2026-03-23 17:10:42 2026-03-23 17:22:15 0h 12m
57573 DLS Plastics Setup Lets Encrypt for DLS Zimbra servers (consultancy) 2026-03-09 14:38:36 None 9h 21m
57685 Koinex Upgrade server to Debian 13 Trixie 2026-03-19 09:44:54 2026-03-19 10:03:49 0h 19m
57735 Tangram CRITICAL: AWS Site-to-Site VPN: Tunnel2 on dgx1.internal.etherapeutics.co.uk 2026-03-24 17:23:38 2026-03-24 17:24:30 0h 01m
57746 Trident Honda VM unifi 2026-03-26 09:50:32 2026-03-26 17:09:40 7h 19m
57694 Tangram GItLab error "The source branch X does not exist." 2026-03-18 16:09:14 2026-03-18 16:23:13 0h 14m
57722 Sucden iLO maintenance 2026-03-24 10:19:35 2026-03-24 10:20:40 0h 01m
57733 Tangram tangram/prod/gitlab eviction/saml errors 2026-03-25 15:33:57 2026-03-26 13:02:03 2d 1h 28m
57534 Sucden Server usage/performance statistics after move of all STAR services to Interstar07-10 2026-03-02 15:51:09 2026-03-02 16:02:14 0h 11m
57609 Gravytrain CRITICAL: System Health: Systemd State on app.prod.qs1.grvytrn.co.uk 2026-03-10 10:17:09 2026-03-10 10:23:31 0h 06m
57624 Gravytrain CRITICAL: System Health: Systemd State on app.prod.qs1.grvytrn.co.uk 2026-03-11 13:42:08 2026-03-11 14:06:18 0h 24m
57659 Sucden VMs don't auto-unlock LUKS at boot 2026-03-16 16:11:00 2026-03-16 21:28:21 5h 17m
57732 Troika Bring up VPN tunnels 2026-03-24 14:48:24 None 9h 12m
57729 Geograph geograph/prod haproxy deployments failing 2026-03-24 16:20:27 None 7h 40m
57684 Imoves Upgrade servers to Debian 13 Trixie 2026-03-18 09:55:10 2026-03-18 10:12:47 0h 18m
57542 Azura Azura systems updates failing 2026-03-03 14:45:53 2026-03-03 16:23:23 1h 38m
57553 OGT docker on mickey 2026-03-05 10:29:46 2026-03-05 10:35:08 0h 05m
57689 Midnight Hax CRITICAL: Environment: APC UPS Status on woodlands.midnighthax.com 2026-03-18 08:41:06 2026-03-18 08:44:44 0h 04m
56919 Ingeus Remove Ingeus systems from support 2026-03-26 15:16:50 2026-03-26 16:12:12 0h 55m
57718 Trident Honda smbfs1.trident.lan volume usage /srv 2026-03-24 13:00:56 2026-03-24 13:02:56 0h 02m
57740 Geograph geograph/prod/gitlab pod status 2026-03-25 08:18:40 2026-03-25 14:39:44 6h 21m
57754 Tangram tangram/prod ThanosCompactIsDown 2026-03-27 09:47:40 2026-03-27 11:06:09 1h 18m
57759 AskBio AskBio hosts down 2026-03-30 13:39:33 2026-03-30 14:39:32 0h 60m
57568 Ingeus [FIRING: 1 for ingeus/iw-prd-aks] KubeNodeUnschedulable in tcl-monitoring 2026-03-09 16:16:43 2026-03-09 17:08:23 0h 52m
57647 OGT granddad/damien winbind failure 2026-03-16 09:29:37 2026-03-16 11:26:41 1h 57m
57651 Ingeus ingeus/iw/prd-aks/tcl-monitoring loki failing 2026-03-16 12:20:12 2026-03-16 12:37:15 0h 17m
57682 ERF Electrical Upgrade servers to Debian 13 Trixie 2026-03-25 12:42:25 2026-03-25 12:55:33 0h 13m
57669 Quickline RADIUS 2026-03-17 13:03:35 2026-03-17 15:16:29 2h 13m
57715 Tangram tangram/prod issues 2026-03-23 08:48:15 2026-03-23 10:13:57 1h 26m
57546 BD Group CRITICAL: HTTP: www.1stformations.co.uk HTTPS on ef-v5-nodes.c.ef-platform.internal 2026-03-04 08:25:32 2026-03-05 15:29:49 7h 04m
57457 Geograph Loki Slow and unreliable 2026-03-03 12:33:35 2026-03-03 12:49:29 0h 16m
57438 BD Group Amend eFiling apache logging config 2026-03-03 15:53:12 2026-03-03 17:12:43 1h 20m
57555 Cirencester College Suspending Active Monitoring - March 30th 2026-03-05 13:52:07 2026-03-05 13:55:47 0h 04m
57621 OpenCorporates Help needed with us_dc_companies_fetcher deploy on aws-bot2 2026-03-11 10:00:59 2026-03-11 10:05:32 0h 05m
57771 Novadata purge leonis from everywhere. 2026-03-31 16:38:23 None 7h 22m
57545 Sucden Rebuild interstar05/06 2026-03-11 14:21:41 2026-03-11 14:45:10 0h 23m
57630 Sucden DOWN: interstar05.sucden.co.uk 2026-03-12 10:30:21 None 1d 3h 30m
57773 OpenCorporates [FIRING:1] DatasourceError Tiger Computing (ElastiCache Connections (openc-frontend-003)) 2026-03-31 10:10:01 2026-04-01 10:22:36 0h 13m
57779 TCV WARNING: NRPE Disk: /var on web4.tcv.org.uk 2026-04-02 10:37:03 2026-04-02 10:38:14 0h 01m
57150 Ingeus Important kubernetes change - ingeus 2026-03-18 12:19:49 None 1d 1h 40m
57664 JISC OpenAthens server patching 2026-03-26 08:17:58 2026-03-26 08:26:47 0h 09m
57743 Tangram Tangram Gitlab upgrade (18.10.1) 2026-03-26 08:14:05 2026-03-26 08:15:57 0h 02m
57719 Trident Honda VM migration 2026-03-23 16:37:26 2026-03-25 10:46:44 1d 8h 09m
57614 Globecast CRITICAL: Disk: multipath on gcukscheddb02.globecastne.com 2026-03-10 11:08:21 2026-03-10 11:21:53 0h 14m
57610 Tangram [FIRING: 1 for tangram/prod] KubePodCrashLooping in gitlab 2026-03-10 10:23:48 2026-03-10 14:32:21 4h 09m
57566 Tangram [FIRING: 1 for tangram/stage] ThanosCompactIsDown 2026-03-10 10:22:04 2026-03-10 11:47:37 1h 26m
57633 Geograph geograph/prod/tcl-monitoring loki failing 2026-03-12 09:05:28 2026-03-16 11:40:54 2h 35m
57528 Sucden interstar10 - ssh 2026-03-02 14:47:30 2026-03-02 15:14:51 0h 27m
57727 Quickline Zabbix Upgrade 2026-03-24 11:36:20 2026-03-24 12:55:38 1h 19m
57673 Global Brands Upgrade servers to Debian 13 Trixie 2026-03-25 16:02:32 2026-03-25 16:05:33 0h 03m
57752 Geograph Re: CRITICAL: HTTP: Geograph staging HTTPS on staging.geograph.org.uk 2026-03-27 08:54:54 2026-03-27 09:12:32 0h 18m
57751 Troika Onboard server to support: tricky.troika.lan 2026-03-26 13:05:49 2026-03-26 16:38:59 3h 33m
57750 Sucden Relocate interstar03/04 from IXN to GS2 -build out to Ubuntu 2026-03-27 15:55:31 2026-03-27 15:59:38 0h 04m
57697 Azura ocado1/2.opentrk.com storage volume usage /usr 2026-03-19 10:41:15 2026-03-19 10:42:31 0h 01m
57680 Trident Honda Upgrade servers to Debian 13 Trixie 2026-03-25 15:08:07 2026-03-25 15:42:41 0h 35m
57695 Midnight Hax fw.midnighthax.com configure graceful shutdown on power outage. 2026-03-19 15:32:40 2026-03-19 15:56:35 0h 24m
57558 Tangram Confirmation of Global Admin Access Requirement 2026-03-12 17:40:18 2026-03-12 17:41:34 0h 01m
57611 Crystal Units crylonlis2024 RHEL renewal 2026-04-29 15:04:45 None 8h 55m
57755 Trident Honda VM topaz 2026-03-27 15:03:44 2026-03-27 15:21:56 0h 18m
57786 Trident Honda VM hera - /etc/sudoers.d/azure-sudo 2026-04-02 17:56:21 2026-04-02 18:03:45 0h 07m
57683 Entrepreneurs Circle Upgrade server to Debian 13 Trixie 2026-03-18 09:11:17 2026-03-18 09:14:26 0h 03m
57716 OGT sybil.internal.ogtip.com volume usage /Data 2026-03-25 17:19:23 2026-03-25 17:22:31 0h 03m
57774 Tangram DGX H100 Maintenance – GPU Replacement 2026-03-31 12:42:41 2026-03-31 12:44:28 0h 02m
57770 TCV tcv-crm.tcv.org.uk cron php script error 2026-04-01 15:29:08 2026-04-01 16:03:12 0h 34m
57706 Tangram Tangram/prod/data-management logical backup failure 2026-03-20 10:53:44 2026-03-20 11:16:06 0h 22m
57772 Quickline Zabbix Upgrade (Consultancy) 2026-03-31 09:39:55 None 1d 4h 20m
57543 Quickline WARNING: HTTP: Zabbix HTTPS Cert on aql-svr-zabbix-app.quickline.co.uk 2026-03-03 10:58:46 2026-03-03 13:24:52 2h 26m
57529 Rotastat gallus.rotastat.com SSL/TLS certificate expiry 2026-03-04 08:38:43 2026-03-04 09:45:42 1h 07m
57717 AskBio edb-iona.askbio.com HTTP status for prompt-softdev.askbio.com 2026-03-24 15:49:45 2026-03-26 11:23:30 1d 9h 34m
57650 Sucden CrackArmor: Multiple vulnerabilities in AppArmor 2026-03-16 10:39:14 2026-03-16 10:42:39 0h 03m
57762 Reigate College rc-mss.reigate.ac.uk disk volume usage 2026-04-02 10:34:16 2026-04-02 10:36:56 0h 03m
57720 Azura UNKNOWN: Hardware: Dell OpenManage on ocado2.opentrk.com 2026-03-23 14:48:29 2026-03-27 16:21:22 1h 33m
57714 Gravytrain web.prod.bf2.bedfordinsurance.co.uk HTTP/HTTPS timeout for www.bedfordinsuranceforbusiness.co.uk 2026-03-23 10:31:52 2026-03-23 11:10:36 0h 39m
57672 Cirencester College Upgrade servers to Debian 13 Trixie 2026-03-25 15:47:14 2026-03-25 15:50:27 0h 03m
57674 Crystal Units Upgrade servers to Debian 13 Trixie 2026-03-18 10:14:54 2026-03-18 10:35:14 0h 20m
57551 Sucden Market Volatility causing re-requests and packet drops. 2026-03-04 17:31:27 2026-03-05 11:38:54 1d 8h 07m
57675 AskBio Upgrade servers to Debian 13 Trixie 2026-03-19 10:10:06 2026-03-19 10:41:08 0h 31m
57763 Sucden interstar05-10.sucden.co.uk IPMI errors 2026-03-30 14:51:22 2026-03-30 15:11:48 0h 20m
57681 TCV Upgrade servers to Debian 13 Trixie 2026-03-18 10:49:27 2026-03-18 11:09:48 0h 20m
57678 Top To Toe Upgrade server to Debian 13 Trixie 2026-03-19 09:15:21 2026-03-19 09:36:41 0h 21m
57679 Rotastat Upgrade server to Debian 13 Trixie 2026-03-30 12:23:24 2026-03-30 12:49:12 0h 26m
57677 BD Group Upgrade servers to Debian 13 Trixie | EFSRV-21666 2026-03-20 09:23:18 2026-03-20 09:29:26 0h 06m

git hash: master (6c029d07)