Difference between revisions of "Categorie:Storingen"

Current Service Interruptions and Maintainance

Horde webmail server down because of spam

  Begin        : 20121116 04:05
  End          : 20121116 09:00
  Affected     : Users of horde webmail

Last night, horde webmail has been misused for sending spam. This happened because a naive user gave the Science password to spammers. We have stopped horde, disabled the account of the naive user and restarted horde. Ben Polman 16 nov 2012 08:20 (CET)

EduroamCAT not working with Science accounts

 Begin         : 2019-02-28 00:00
 End           : ?
 Affected      : EduroamCAT users with Science accounts

EduroamCAT is the Eduroam Configuration Assistant Tool for many different devices. However, this hasn't (yet) been set up for the use of Science accounts (). C&CZ is looking for a solution. In the meantime Eduroam connections have to be configured manually (please consult www.ru.nl/wireless) or using the U/S/E number.

Recently Resolved Service Interruptions and Maintainance

To be quickly informed about service interruptions one can subscribe to the CPK mailinglist.

Science smtp service temporarily not usable

  Begin         : 2021-01-22 10:00
  End           : 2021-01-22 10:30
  Affected      : Science mail users wanting to send mail

A configuration change unwantedly made the smtp service unusable. When we noticed this, it was repaired immediately.

Very long mail aliases temporarily not usable

  Begin         : 2021-01-21 15:52
  End           : 2021-01-22 09:55
  Affected      : Science mail aliases of more than 1024 characters

A configuration change had as unwanted effect the disappearance of all very long mail aliases. When this was reported next morning, it was repaired immediately.

Switch crash; gitlab+mattermost, licenses and DHZ

  Begin         : 2021-01-07 ~14:30
  End           : 2021-01-07 ~15:00
  Affected      : GitLab and Mattermost users, Licenses, DHZ (diy)

Due to a simple management command the switch (as-ak008-04) crasht and had to be reset manually. The switch sits between the network and servers for gitlab+mattermost, licenses and the database for DHZ(diy).

Gitlab upgrade

  Begin         : 2020-11-27  04:00
  End           : 2020-11-27 ~08:00
  Affected      : GitLab and Mattermost users (including PEP)

Services will not be available for a while because of a GitLab and Mattermost upgrade.

Eduroam problem on campus

 Begin         : 2020-07-10 evenng
 End           : 2020-07-10 evening
 Affected      : Eduroam users on campus

The ISC announced: For security reasons, the certificate of the wifi server will be replaced in the evening of Friday, July 10. This has consequences for connecting your mobile device to Eduroam when you’re on campus:

• If you get the message that you have to accept the new certificate to use eduroam, choose 'yes'. You can then use eduroam again;

• If you don't get this message and can't connect to Eduroam, choose the wireless network 'eduroam-config'. Accept the terms and conditions. Follow the instructions to reinstall Eduroam.

More information can also be found at www.ru.nl/ict-uk/eduroam (you will need an internet connection for this).

If you have any questions, please contact the ICT Helpdesk (024 - 36 22222).

RU mail erroneously in Spam folder

 Begin         : 2020-03-25 17:52
 End           : 2020-07-07 13:13
 Affected      : FNWI employees with Science mail

March 25, a rule "2020 Radboud Universiteit" was added to the Science spamfilter. Recently, this matched RU-central mailings. Therefore RU-wide mailings from e.g. the RU Board and Radboud Recharge have erroneously been delivered in the Spam folder of Science emplyees. The Science spamfilter tries to fight spam and phishing, this is partly manual work in which errors can't be excluded. C&CZ apologizes for the inconvenience this has caused.

Webserver 'havik' offline

 Begin         : 2020-06-18 15:45
 End           : 2020-06-18 16:25
 Affected      : Users of various websites.

Several parts have been replaced. We assume the problem, that occurred twice, is now resolved. For dual-boot pcs, the boot menu was served by an alternative method during the repair.

Science radius disruption

 Begin : 2020-06-17 11:11
 End   : 2020-06-17 11:56

Affected : Users of Science VPN and Eduroam based on science account

The certificate of the LDAP servers has been replaced this morning, this has also changed the certificate chain. The radius server uses LDAP as authentication backend and in the radius configuration the certificate chain had to be changed too. This was initially overlooked. Radius is the authentication mechanism used by all VPN servers and Eduroam

Webserver 'havik' offline

 Begin         : 2020-06-17 03:38
 End           : 2020-06-17 08:52
 Affected      : Users of dual boot PC's (the dual-boot menu is served by a website) and various websites.

The server went down in the same way as the previous time (3rd of June 2020). The cause is most likely a system board problem. This part will be replaced Tomorrow by a support engineer.

Webserver 'havik' offline

 Begin         : 2020-06-03 06:30
 End           : 2020-06-03 10:12
 Affected      : Users of dual boot PC's (the dual-boot menu is served

by a website) and various websites.

The server couldn't be reached after the scheduled weekly reboot, not even on its management interface. Because also C&CZ employees work from home and the interruption didn't get enough urgency fast enough, the interruption lasted too long, apologies for that. The support partner has been contacted and the server has been updated, but the origin of the problem is still unclear. We will also look at making these services more redundant or more easily movable to a different server.

CN00 Slurm master ubuntu 16.04 down

 Begin         : 2020-05-18 09:50
 End           : 2020-05-19 12:15
 Affected      : slurm on ubuntu 16.04 (cn07)

Due to a failed BIOS upgrade, the hardware of the database server appears to be bricked. We transfered the disks to another machine (cn00) and all database services are now up again, at the cost of not having cn00 running. When the hardware is working well again, we will swap it all back and restore the original situation.

Sperwer Database server storing

 Begin         : 2020-05-18 06:30
 End           : 2020-05-18-10:00
 Affected      : various websites and slurm

Due to a failed BIOS upgrade, the hardware of the database server appears to be bricked. We transfered the disks to another machine (cn00) and all database services are now up again, at the cost of not having cn00 running. When the hardware is working well again, we will swap it all back and restore the original situation.

Update May 19th, 12:15 : hardware fixed, situation back to the original state.

Science VPN disruption

 Begin         : 2020-05-06 05:00
 End           : 2020-05-06 08:00
 Affected      : Users of Science VPN

Unexplained crashes starting around 5am on the host system. System offline at around 6am. After a hard reset around 08:00, all seems to be all right again.

Science datacenter network problem

 Begin         : 2020-04-30 12:08
 End           : 2020-04-30 21:44
 Affected      : Users of Ceph storage and a few new compute clusternodes

A broken transceiver caused flapping af a 100 Gb/s connection between two C&CZ datacenters. Hours later the flapping increased, which took down the complete redundant new connection between the two server rooms. When this was noticed, a workarpound was found quickly by shutting down the interface with the broken transceiver. With this the connection was restored. De broken transceiver has been replaced thanks to a swift action from our supplier. Now we have these spare parts ready to use. We asked the supplier whether a configuration change will make the connection more redundant, that just one broken transceiver will not take down the connection.

Jitsi.science.ru.nl not working properly

 Begin         : 2020-04-19 15:00
 End           : 2020-04-20 11:40
 Affected      : Users of jitsi.science.ru.nl

Due to performance tuning having gone wrong the jitsi.science.ru.nl conference rooms cannot be joined by more than one person at the moment. Solved by reinstalling server.

Mailserver certificate problem

 Begin         : 2020-04-13 14:00
 End           : 2020-04-13 14:35
 Affected      : Users Science mail

The new certificate of the Science mailserver hadn't yet been placed in the right place. The expiration of the old certificate caused a problem for Science mail users, that was resolved by replacing the old certificate.

Problems with a virtual host

 Begin         : 2020-02-18 05:30
 End           : 2020-02-18 09:08
 Affected      : Users of mx3, smtp3, crestron, gitlab (PEP), goudsmit, msql01 and labservanttestvm.

The virtual machine host 'oscar' could not boot. Again, a broken LVM snapshot caused the problem.

Archived service interruptions can be found in the service interruptions archive.

Be quickly informed via the CPK mailinglist or the RSS feed