Categorie:Storingen

Current Service Interruptions and Maintainance

Horde webmail server down because of spam

  Begin        : 20121116 04:05
  End          : 20121116 09:00
  Affected     : Users of horde webmail

Last night, horde webmail has been misused for sending spam. This happened because a naive user gave the Science password to spammers. We have stopped horde, disabled the account of the naive user and restarted horde. Ben Polman 16 nov 2012 08:20 (CET)

eduroamCAT not working with Science accounts

 Begin         : 2019-02-28 00:00
 End           : ?
 Affected      : EduroamCAT users with Science accounts

EduroamCAT is the Eduroam Configuration Assistant Tool for many different devices. However, this hasn't (yet) been set up for the use of Science accounts (). C&CZ is looking for a solution. In the meantime Eduroam connections have to be configured manually (please consult www.ru.nl/wireless) or using the U/S/E number.

Recently Resolved Service Interruptions and Maintainance

To be quickly informed about service interruptions one can subscribe to the CPK mailinglist.

Webserver 'havik' offline

 Begin         : 2020-06-03 06:30
 End           : 2020-06-03 10:12
 Affected      : Users of dual boot PC's (the dual-boot menu is served

by a website) and various websites.

The server couldn't be reached after the scheduled weekly reboot, not even on its management interface. Because also C&CZ employees work from home and the interruption didn't get enough urgency fast enough, the interruption lasted too long, apologies for that. The support partner has been contacted and the server has been updated, but the origin of the problem is still unclear. We will also look at making these services more redundant or more easily movable to a different server.

CN00 Slurm master ubuntu 16.04 down

 Begin         : 2020-05-18 09:50
 End           : 2020-05-19 12:15
 Affected      : slurm on ubuntu 16.04 (cn07)

Due to a failed BIOS upgrade, the hardware of the database server appears to be bricked. We transfered the disks to another machine (cn00) and all database services are now up again, at the cost of not having cn00 running. When the hardware is working well again, we will swap it all back and restore the original situation.

Sperwer Database server storing

 Begin         : 2020-05-18 06:30
 End           : 2020-05-18-10:00
 Affected      : various websites and slurm

Due to a failed BIOS upgrade, the hardware of the database server appears to be bricked. We transfered the disks to another machine (cn00) and all database services are now up again, at the cost of not having cn00 running. When the hardware is working well again, we will swap it all back and restore the original situation.

Update May 19th, 12:15 : hardware fixed, situation back to the original state.

Science VPN disruption

 Begin         : 2020-05-06 05:00
 End           : 2020-05-06 08:00
 Affected      : Users of Science VPN

Unexplained crashes starting around 5am on the host system. System offline at around 6am. After a hard reset around 08:00, all seems to be all right again.

Science datacenter network problem

 Begin         : 2020-04-30 12:08
 End           : 2020-04-30 21:44
 Affected      : Users of Ceph storage and a few new compute clusternodes

A broken transceiver caused flapping af a 100 Gb/s connection between two C&CZ datacenters. Hours later the flapping increased, which took down the complete redundant new connection between the two server rooms. When this was noticed, a workarpound was found quickly by shutting down the interface with the broken transceiver. With this the connection was restored. De broken transceiver has been replaced thanks to a swift action from our supplier. Now we have these spare parts ready to use. We asked the supplier whether a configuration change will make the connection more redundant, that just one broken transceiver will not take down the connection.

Jitsi.science.ru.nl not working properly

 Begin         : 2020-04-19 15:00
 End           : 2020-04-20 11:40
 Affected      : Users of jitsi.science.ru.nl

Due to performance tuning having gone wrong the jitsi.science.ru.nl conference rooms cannot be joined by more than one person at the moment. Solved by reinstalling server.

Mailserver certificate problem

 Begin         : 2020-04-13 14:00
 End           : 2020-04-13 14:35
 Affected      : Users Science mail

The new certificate of the Science mailserver hadn't yet been placed in the right place. The expiration of the old certificate caused a problem for Science mail users, that was resolved by replacing the old certificate.

Problems with a virtual host

 Begin         : 2020-02-18 05:30
 End           : 2020-02-18 09:08
 Affected      : Users of mx3, smtp3, crestron, gitlab (PEP), goudsmit, msql01 and labservanttestvm.

The virtual machine host 'oscar' could not boot. Again, a broken LVM snapshot caused the problem.

Unplanned GitLab security update

 Begin         : 2019-12-11 06:00
 End           : 2019-12-11 09:00
 Affected      : Users of gitlab.science.ru.nl, mattermost.science.ru.nl, gitlab.pep.cs.ru.nl, mattermost.pep.cs.ru.nl

A critical security update for GitLab has been released and is being installed this moment. During the update, GitLab is offline.

Problems with a virtual host

 Begin         : 2019-11-05 05:30
 End           : 2019-11-05 09:20
 Affected      : Users of mx3, smtp3, crestron, gitlab (PEP), goudsmit, msql01 and labservanttestvm.

The virtual machine host 'oscar' stopped running at 5:30am. The listed virtual machines are not running at the moment. We are working on it. Update: Resolved. Again, a broken LVM snapshot caused the problem.

Power Distribution Unit failure

 Begin         : 2019-10-23 16:00
 End           : 2019-10-23 16:55
 Affected      : Clusternode users (TCM, VSC, TheoChem) on cn71, cn72 and cn82 

A failed PDU caused the servers with a single Powersupply to die and restart after the PDU was replaced. Running jobs were not finished on these nodes.

Webserver 'havik' offline

 Begin         : 2019-10-06 10:35
 End           : 2019-10-07 08:50
 Affected      : Users of dual boot PC's (the dual-boot menu is served by a website) and various websites. 

A loose power cable caused an interruption of one of our web servers. The machine is equipped with redundant power supplies. It's not clear why a single loose cable caused a service interruption.

NFS service interruption

 Begin         : 2019-09-23 06:30
 End           : 2019-09-23 09:00
 Affected      : Users of NFS network shares from server 'peck' (/vol/* on Linux systems). Only disks that have a DNS record ({volumename}-srv.science.ru.nl) pointing to file server 'peck' are affected.

Due to a configuration error, the NFS service on file server 'peck' refused to start. After fixing the error, the NFS service started up correctly.

Boot problems of virtual host

 Begin         : 2019-09-03 04:30
 End           : 2019-09-03 10:47
 Affected      : Users of mx3, smtp3, crestron, gitlab (PEP), goudsmit, msql01, labservanttestvm and gitlabrunner

At the weekly reboot of vmhost 'oscar', it did not boot properly. The listed virtual machines are not running at the moment. We are working on it. Update: most probably, the problems where caused by full snapshots. After removal of the snapshots, the system booted properly.

Mailbox problems for a few users

  Begin         : 2019-08-23  15:10
  End           : 2019-08-23  15:57
  Affected      : A few Science mail users

Due to an administrative error the mailboxen of a few hundreds of users became inaccessible. Mail couldn't be read from or delivered to these mailboxes until this had been repaired. In this period the mailclient of 19 users tried to read the mailbox and thus possibly reported an error.

Mail problems with newly introduced incoming mail server

  Begin         : 2019-07-01 10:33
  End           : 2019-07-02 12:28
  Affected      : Science mail receivers

A new receiving mailserver was introduced July 1. This server has the most recent version of Ubuntu and other software and settings that are more conforming to the directives (in Dutch). After the introduction, problems appeared with receiving mail from Microsoft Office 365 Exchange Online (mail.protection.outlook.com). An even bigger problem was that some accepted mails were not forwarded, but bounced to the sender. Because that problem couldn't be fixed immediately, the newly introduced server has been shut down on July 2. At the end of August, C&CZ will test the probable solution. If the test succeeds, the new incoming server will be put into production.

Network interruption central RU network

  Begin         : 2019-05-22 18:55
  End           : 2019-05-22 19:37 
  Affected      : Users of network services

Due to a faulty RU core network device, several service couldn't be reached. This device is in a redundant setup. The ISC network department will start an investigation what exactly happened and how to prevent nuisance in the future.

Dozens of Science websites down, also Roundcube

  Begin         : 2019-05-14 15:28
  End           : 2019-05-14 16:20
  Affected      : Users of a website of this webserver (weber)

Because a web application used too much memory, all websites of this webserver had problems. After restarting the webserver, all websites could be reached again. We will make sure that web applications get memory limits, which will prevent this problem in the future.

DNS nameservice interruption for Science domains

  Begin         : 2019-05-14 16:13
  End           : 2019-05-15 21:22
  Affected      : Off-campus users wanting to access Science servers

Yesterday afternoon a DNS error with science.ru.nl (due to the introduction of the secure DNSSEC) made science.ru.nl disappear from the internet for many users, when viewed from outside campus. In the evening this error has been corrected. Because of the caching of DNS responses, the last problems should disappear tonight. In the meantime one can use workarounds like rebooting the home router and/or pc, using a different network (mobile provider) or using rainloop.science.ru.nl instead of roundcube.science.ru.nl for webmail access.

Network interruption Huygens wing 4

  Begin         : 2019-04-13 05:15
  End           : 2019-04-15 14:00
  Affected      : Users in Huygens wing 4

Due to a faulty network switch, approx 40 network connections in Huygens wing 4 were temporarily unavailable. The switch has been replaced.

Matlab license server down

  Begin         : 2019-03-28 07:30
  End           : 2019-03-28 09:46
  Affected      : Matlab users

This morning, the Matlab license server stopped, because yesterday an error had been made in the upgrade to version R2019a. After the error had been corrected, the license server was restarted, which resolved the problem.

Poster printer "kamerbreed" broken

  Begin         : 20190326
  End           : 20190327 16:00
  Affected      : Users of the poster printer "kamerbreed"

The poster printer has to be repaired, but we can't easily fix it. We hope to have the printer operational again in the near future. In the meantime you might use the RU copyshop.

No connections to Eduroam with science accounts

 Begin         : 2019-03-10 18:00
 End           : 2019-03-13 08:30
 Affected      : many Eduroam users with science accounts

Due to a configuration fault in the Science radius server, many connections to Eduroam for users with a science account failed. It took two days of joint investigation by ISC and C&CZ to resolve this problem.

No Eduroam end for deprecated systems (Windows XP, Android 4.1, ...)

 Begin         : 2018-12-10 05:00
 End           : 2018-12-10 12:00
 Affected      : some Eduroam users

On December 10, the support for the security protocol TLS v1 and v1.1 was discontinued within the Radboud Eduroam wireless network. From that moment on, all wireless clients such as laptops, smartphones, tablets and the like must use TLS version 1.2 or higher.

Unfortunately, the outcome of this change was not as expected. After the change, there were more clients with connection problems than the number of clients using TLSv1. After a period of testing and research, the ISC decided around 12.00 to reverse the change.

Although the underlying reasons are still valid, the termination of TLSv1 support has been suspended for the time being. In consultation with SURF it will be decided in the coming period what caused the problems of the different clients.

Disk problem milton fileserver

 Begin         : 2018-08-27 06:30
 End           : 2018-08-27 09:40
 Affected      : Users of server milton (Milieukunde), disk milkun16

One of the data-disks has a currupted file system. During the time slot menioned above, the server was unavailable. At the moment the server is running without the defective disk (milkun16). At the moment, we are trying to fix the disk.

SSH connections of older clients/servers

 Begin         : 2018-09-11 07:20
 End           : 2018-09-11 10:25
 Affected      : Ubuntu 12_04 LTS servers / Ubuntu 12 (or older) clients

A configuration change was automatically put into production Tuesday morning which had been prepared the previous day on Ubuntu 14_04 LTS, Ubuntu 16_04 LTS and Ubuntu 18_04 LTS in order to increase the level of security of SSH connections within FNWI. The change had the side effect of crashing the SSH daemon on Ubuntu 12_04 LTS and preventing older SSH clients from connecting at all. Users are encouraged to update their SSH clients so that we can gradually increase the level of security again.

Host hardware of virtual machines broken

 Begin         : 2018-08-23 01:41
 End           : 2018-08-23 11:35
 Affected      : Users of crestron (AVD) gitlab7 (PEP) goudsmit (NMR) msql01

Last night the host for a number of vm's (oscar) broke down. By using a replacement server (joule) this problem has been temporarily resolved. Tomorrow the machine will be repaired ans we will determine when the machines will be switched back.

Bug in new IMAP mailserver resolved

 Begin         : 2018-08-09 13:05
 End           : 2018-08-22 09:25
 Affected      : Users wanting to read Science mail

The new version of the IMAP mailserver software contained a bug in the handling of SSL connections, that led to connections being dropped and depending on the mailclient led to a mailserver that could hardly be used. This morning a patch for this bug has been installed.

Network problem IMAP-mailserver because of error during update

  Begin         : 2018-08-20 15:32
  End           : 2018-08-20 ca. 15:45
  Affected      : Users reading science mail

While testing a possible update to the IMAP mailserver, an error made the mailserver unavailable for ca. fifteen minutes.

Upgrade IMAP mailserver post.science.ru.nl: Thursday August 9 2018 13:00-13:05

 Begin         : 2018-08-09 13:00
 End           : 2018-08-09 13:05
 Affected      : Users wanting to read Science mail

The IMAP mailserver post.science.ru.nl is being replaced by a new server. Therefore connections to the IMAP mailserver post.science.ru.nl will not be possible during ca. 5 minutes.

Server maintenance shares: Friday 3rd of August 2018 12:00-13:00

  Begin         : 2018-08-03 12:00
  End           : 2018-08-03 13:00
  Affected      : Network-shares on C&CZ servers, shares: 45t acfiles acfiles2 acl-ru-nl$ acs4 aerochem afstudeertest afstudeerwerk akolkmandata albert amsbackup amscommon animal arb astropresentie avd bbb-priv beevee betadidactiek b-fac-backup bioalg bioexamencie bioexamencierw bioniccell bio-orgchem birds bko boltje botany-general carisma carta ccs4 ccz cczhelpdesk cczsecr cdalgemeen cdbiow cdisis cdoii cdomw cdwinst celbio1 celbio2 celbio2 celbio3 ces2017 chemprac chemstaff cncz coastoffon cudotest cursus cursusdossiers dbiwwr-ro dbiwwr-rw dcnadmin desda diafragma docent docenten ds ecogen ecologie ehef1 ehef2 ehef3 ehef4 embdata epsylon erhvedata evsf2data10 evsf2data15 evsf2data16 evsf2data17 evsf2data18 evsf2data19 evsf2data7 evsf2data8 evsf2data9 evsf2schijf1 exoarchief femtospin fsr geminstr geminstr2 gi1 gi2 gi3 giphouse gipsy gissig global global global gmi goudsmit1 goudsmit2 goudsmit3 goudsmit4 goudsmitoldusers grafting hfml-backup hfml-backup2 hfml-backup3 hfml-backup4 hfml-backup-inst hfml-data hfml-engineering hfml-secr highres honours huckgroup hybrids icis icisafstudeer icisbureau iciscop icislaquso icissec icisst ifl ifl-d1 IJSO imappdir imappsecr immbureau immstaff impuls introcie invbio iris isis isptinspec itt iwwr1 janvanhest kaartenbak kangoeroe khaj kolkteaching labservant lambiek leonardo licenses liquidhelium liquids mailmanincludes marcurkasco master mathematica mathsolcie mbaudit1 mbaudit2 mbaudit3 mbaudit4 mbaudit5 mbbioel mbbioel2 mbcortex mbonderwijs mbread mbscon mbsl mbwrite mercator mestrelab mi1 mi2 mi3 micord micro-bulk1 micro-bulk2 micro-labmembers mlf90 mlf90grp mlf90linac mlfalgemeen mlfsec molbiol molchem molchem2 molchem3 molneuro molphtec mol-secr morph mpcoldmol2 mpcoldmollab mpcoldsecr mppstaf multimedia mwstudiereis neuroinf neuroinf2 neuropi nmr500data nmrspectra olcbiocie olcbiow olccudo olcii-archief olympus onderwijskwalificatiedossiers onlyme ons orgdier orgdiermac owc owc1 parp pcb plantkunde-hgl poliepC poliepD pracnat practica project1mpp puc randomwalkmodel rotax ruversatest secrmolbiol secrthch share sigma sigmacies smol snn snn2 snn3 snn4 sofie software soheto solids sos spectra spectra-rw spibdossier spm spmdata1 spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmmagstm spmnanolab spmstaff ssi ssiguest staff-hfml stroom studiereis2013 synorgchem tece teceleiding template-ru-nl$ template-ru-nl$ template-ru-nl$ tf2devices thalia thaliacies thchem tracegas tracegastemp tzacad ucm unl vcbio_prive vergader vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc16 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin vscxray wallpaper wikis wiskalg wiskunde wiskundetoernooi wkru zeegras

Three of our main fileservers need firmware updates, this requires the servers to be rebooted several times. The shares will be unavailable for at most one hour.

RU network maintenance: Monday evening/night July 2 to 3 from 22:00 hours

Monday evening July 2, there will be maintenance on central RU network equipment. From 22:00 hours, central RU applications and network will not be available most of the time. For telephones no disturbance is expected. For the C&CZ services there is a possibility of disruptions: no Internet connection for some time. Please think which processes depend on connectivity to RU central services and prepare for this possible outage if necessary. Future maintenance windows can be seen on the ISC website.

Interruption of parts of the telephone service

 Begin         : 2018-06-09 08:00
 End           : 2018-06-09 17:00
 Affected      : Users of analog telephone equipment.

Saturday june 9th from 8:00h until approximately 12:00h analog telephone communications via AudioCodes will not be possible. At this time the existing equipment used in the Huygens- and surrounding buildings will be moved and reconnected. Only traditional analog equipment such as DECT phones and such are connected to the AudioCodes. The green (emergency) phones are not connected to these AudioCodes but to the emergency telephone exchange.

The emergency telephone exchange wil go down for ten minutes during the afternoon due to an intteruption of the power supply. The bulk (if not all) of the emergency phones and elevators are connected to this emergency telephone exchange and will therefore not be operational at that time.

No Internet (DHCP problem for leased addresses)

 Begin         : 2018-06-09 07:00
 End           : 2018-06-12 10:49
 Affected      : Network devices with leased addresses

Last week, the DHCP infrastructure at FNWI was changed, in order to move from old to new servers. An error was made when we had to move back because DHCP did not run well on a virtual interface. When the first complaints came in this morning, we quickly found and repaired the error. We try to avoid further service disruptions while improving the DHCP service.

No analog telephone communications on Saturday June 9

 Begin         : 2018-06-09 08:00
 End           : 2018-06-09 17:00
 Affected      : Users of analog telephone equipment

Saturday June 9th from 8:00h until approximately 12:00h analog telephone communications via AudioCodes will not be possible at FNWI. At that time the existing equipment used in the Huygens and surrounding buildings will be moved and reconnected. Only traditional analog equipment such as DECT phones are connected to the AudioCodes. The green emergency phones are not connected to these AudioCodes but to the emergency telephone exchange.

The emergency telephone exchange wil go down for ten minutes during the afternoon of that day due to an interruption of the power supply. The bulk (if not all) of the emergency phones and elevators are connected to this emergency telephone exchange and will therefore not be operational during those ten minutes.

FNWI DNS nameserver problem with ru.nl

 Begin         : 2018-05-22 14:24
 End           : 2018-05-22 16:00
 Affected      : Wired FNWI users wanting to access ru.nl sites or hosts

Last week, the DNS infrastructure at FNWI was changed, in order to move from old to new servers and to prepare for the use of DNSSEC. A workaround for the internal/external view of ru.nl had an unwanted side effect. This made the FNWI DNS servers lose the info about ru.nl. We had to resort to a different workaround to fix this. We try to avoid further service disruptions while improving the DNS services.

Multiple backup server issues

 Begin         : 2018-03-17
 End           : 2018-03-26
 Affected      : users wanting a restore of a recently created/changed file

Backups were incomplete since March 17th. The tape robot couldn't be used because a tapedrive failed, probably due to dry weather conditions. Therefore holding disks filled up. Only the backup of Tuesday upon Wednesday is possibly complete, but we lack backups of March 21st and 22nd, which is caused by a filesystem inconsistency of the holding disks, repairs already take two days. We're trying hard to make backups this weekend, but please be cautious with your files, since restores of recently changed files might fail.

Homedirectories onbereikbaar op windows

 Begin         : 2018-03-06  12:00
 End           : 2018-03-06  12:50
 Affected      : Windows users with homedirectory on home2 (check yours in diy.science.ru.nl)

During the process of restoring the homedirectory of the affected user of the ransomware, the samba server was restarted, however it stopped, but not start. This was noticed only 45 minutes later. We have ajusted our monitoring to help us notice this more quickly.

Gijzelsoftwware op IMAPP mappen

 Begin         : 2018-03-06  08:40
 End           : 2018-03-06  14:00
 Affected      : Users of imappsecr, imappdir and astropresentie shares

Due to an accidental opening of an e-mail attachment (called Resume.doc), which contained a macro-based ransomware, all the files and directories on the mentioned shares were either removed or encrypted and a message was left on the infected computer to contact a certain URL to get the files back. The files were restored from backup and the PC wiped clean and re-installed. NB. The f-secure software did not recognise this malware yet at the time.

Problem with outgoing mailserver (smtp)

 Begin         : 2018-02-08  02:39 + 08:57 + 
 End           : 2018-02-08  07:53 + ... + finally 15:39
 Affected      : Science users wanting to send mail through smtp.science.ru.nl

Yesterday morning, we got an alarm because the smtp service did not work properly. The cause was found to be that from various PCs from, among other countries, China, connections were built up with one of the smtp servers, without actually sending an e-mail, as a result of which all these connections were left hanging. If the number of standing connections became too high, the server became unusable for others. This service has to be be accessible from the Internet, because of authenticated SMTP. After a few PCs had been blocked, at 15:39 Fail2ban was turned on, that is also used to protect other services accessible from the Internet, such as mailservers, Linux loginservers, WWW-servers and GitLab servers.

Fileserver maintenance because of reported disc problem

 Begin         : 2018-01-22  08:30
 End           : 2018-01-22  09:45
 Affected      : Science users with network discs from this server:
       acfiles aerochem amsbackup amscommon animal astropresentie avance500
       avance600 beevee b-fac-backup bioniccell bio-orgchem boltje carisma
       celbio1 celbio2 chemprac cncz coastoffon csgi-archief desda ds ehef1
       ehef2 ehef3 ehef4 femtospin geminstr2 giphouse gissig global gmi goudsmit1
       hfml-backup hfml-backup2 hfml-backup3 hfml-backup-inst highres imappdir
       imappsecr impuls introcie isis janvanhest kaartenbak kangoeroe khaj
       linova500 linova800 marcurkasco mathsolcie mbaudit1 mbscon mbsl molbiol
       mwstudiereis neuroinf neuroinf2 nfstest olympus puc randomwalkmodel
       ruversatest secres2 secrmolbiol sigma sigmacies spectra spectra-rw
       spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmmagstm spmnanolab
       spmstaff ssi ssiguest staff-hfml stroom thalia thaliacies ucm vsc vsc1
       vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8
       vsc9 vscadmin wiskalg wiskunde wkru

This morning the fileserver reported a disc problem. To diagnose this, we have to upgrade the firmware, which makes downtime necessary. We planned this for next Monday morning because of the Next Business Day support contract. Sorry for any inconvenience.

GitLab temporarily unavailable during upgrade

 Begin         : 2018-01-16  23:00
 End           : 2018-01-17 ~11:00
 Affected      : GitLab users

Because of a planned upgrade, gitlab.science.ru.nl and gitlab.pep.cs.ru.nl will not be available. See also: [1]

Expired certificate for IMAP mailserver

 Begin         : 2017-12-11  00:59
 End           : 2017-12-11  09:47
 Affected      : Users of the Science IMAP mailserver

The new certificate for the IMAP mailserver hadn't been put into place correctly. Sorry for any inconvenience.

Incorrect Adobe Acrobat Pro DC package on install disc

 Begin         : 2017-12-02  16:20
 End           : 2017-12-05  16:14
 Affected      : Users who installed Adobe Acrobat Pro DC from the install disc

We placed an incorrectly packaged version on the install drive. After users reported this, the ISC has created a new package and we have replaced it.

Homedirectory unavailable

 Begin         : 2017-11-22  ~07:30
 End           : 2017-11-22  ~09:20
 Affected      : science users

After an update of the samba software, it appears that the smbd service didn't start up again. The problem was resolved by either rebooting (home1) or manually starting smbd (home2).

Network maintenance at Science Faculty: upgrade router

  Begin         : 2017-11-27 11:30 pm
  End           : 2017-11-28 02:00 am
  Affected      : all network users at the Science Faculty, Mercator, UBC, Childcare Heyendael

New firmware will be loaded into the Heyendaal-East distribution router. During this period the wired and wireless network will be unavailable for approx. 30 minutes.

SPIB, reviewapp, thesis submission, timetrex intermittent failures

 Begin         : 2017-11-14  ~20:50
 End           : 2017-11-16
 Affected      : students, teachers, people working with timetrex

Currently the hardware has problems, causing the machine to randomly turn off and on. We are in the process of resolving this with our supplier.

The problem doesn't appear to be hardware related, but a problem with the configuration of the OS, which we changed and it seems to be stable now.

Homeservers rebooted daily

 Begin         : 2017-10-01  ~06:28
 End           : 2017-10-06  ~11:00
 Affected      : science users

Due to a misinterpretation of the correct crontab syntax, the homeservers were rebooted daily since the beginning of this month, instead of, as we intended, once a Month on the first Monday.

DIY down due to hardware problem

 Begin         : 2017-09-27  ~15:00
 End           : 2017-09-27  ~16:45
 Affected      : science users who want to change settings of their account or aliases

The server of DHZ had a problem with the hardware, the hardware was fixed on 2017-09-29, the website DHZ could be restored more quickly on a different server, limiting the outage to a few hours.

Homedirectories unavailable

 Begin         : 2017-09-22  07:20
 End           : 2017-09-22  09:08
 Affected      : Science users on windows clients

This morning, the homeservers were automatically upgraded with normal updates, after which the windows file servicewas not restarted. The problem was resolved by restarting the machines.

Homedirectories unavailable on monday mornings

 Begin         : 2017-0?-??  ??:??
 End           : 2017-??-??  ??:??
 Affected      : mainly Ubuntu 16.04 users

Multiple Ubuntu 16.04 users are unable to access their homedirectories on machines that have been rebooted on monday morning. Keeping system clocks more in sync seems to have helped somewhat, but no definite solution has been found yet. Therefore we decided to reboot the homeservers only the *first* Monday of each month. In the meantime we still hope for a real solution.

OpenVPN not started at reboot

 Begin         : 2017-08-14  06:31
 End           : 2017-08-15  11:36
 Affected      : OpenVPN users

For yet unknown reasons the OpenVPN service did not start at the Monday morning reboot. We started the service manually.

DIY down due to hardware problem

 Begin         : 2017-08-10  ~8:00
 End           : 2017-08-11  ~11:00
 Affected      : science users who want to change settings of their account or aliases

The server of DHZ is having problems, while we are solving this with Dell support, all changes can be handled via us (tel: 53535) or postmaster@science.ru.nl

Emergency power failure Huygens building lasted hours

 Begin         : 2017-07-22 11:38
 End           : 2017-07-22 / 2017-07-24 / 2017-08-01
 Affected      : Huygens building

Saturday July 22 at 11:37, there was a power outage in the Huygens building, also for all equipment connected to emergency power. The power outage lasted long enough to leave even devices and rooms with a UPS battery backup without power. This included all network equipment (routers, switches, wireless access points) and all servers in the C&CZ large server room.

When power returned from 14:08, several devices were defective:

- Half of the fully redundant FNWI router was defective. External companies in the UBC are not doubly connected to their provider, so they had no connectivity until Monday, when their connectivity was switched. Since Monday afternoon, the FNWI router has been repaired, so is redundant again.

- Eight power distribution units (PDUs), that each supply power to ca. eight servers in the C&CZ server room were defective. Power connections of the most important servers were switched to different outlets on Saturday afternoon or these servers were restarted after replacement of their power distributors. On Monday the rest of the computer cluster nodes have been restarted.

As of August 1, the emergency power in the Huygens building is guaranteed again. The UVB investigated and solved all problems.

GitLab temporarily unavailable during upgrade

 Begin         : 2017-06-30  ~9:00
 End           : 2017-06-30 ~11:00
 Affected      : GitLab users

Because of a planned upgrade, gitlab.science.ru.nl and gitlab.pep.cs.ru.nl will not be available.

LibreOffice problem using Ubuntu 14.04

  Begin         : 2017-06-26 06:30
  End           : 2017-07-19 after reboot
  Affected      : employees and students at Faculty of Science (eg using lilo4)

LibreOffice and other software suites using Java (jvm) internally are affected by the Ubuntu 14.04 kernel update patching the Stack Clash. It seems that Linux kernel developers, Ubuntu developers and JVM developers are working hard world-wide in finding a proper solution. Work-around 'JAVA_TOOL_OPTIONS=-Xss1280k libreoffice'. Ubunti released a fix.

Fileserver needed reboot after fatal typo

  Begin         : 2017-06-13 10:45
  End           : 2017-06-13 10:51
  Affected      : Users of shared network disks:
                  acfiles beevee bioniccell bio-orgchem celbio1 celbio2
                  chemprac csgi-archief csg-projecten csg-staf desda ds ehef1
                  ehef2 ehef3 femtospin geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 hfml-backup-inst highres impuls
                  introcie isis janvanhest kaartenbak kangoeroe mathsolcie microbiology microbiology2
                  milkun2 milkun2RenD milkun3 milkun5 milkun6 milkun7 milkun7rw
                  mwstudiereis neuroinf neuroinf2 olympus puc randomwalkmodel secres2 sigma sigmacies spectra spectra-rw
                  spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmstaff ssi staff-hfml stroom thalia ucm vsc
                  vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde wkru

Apologies.

Server TCMR storing na disk-falen

  Begin         : 2017-05-29 6:30
  End           : 2017-05-30 09:30
  Affected      : TCM disk users

Sunday evening, one of the archive disks failed. The data on it was lost, as it wasn't in a redundant setup. After the weekly reboot, the server hung on this disk being broken. It took us a while to get the server up again.

Eduroam wifi behind firewall

  Begin         : 2017-05-22 22:00
  End           : 2017-05-23 02:00
  Affected      : Eduroam wifi users

Due to maintenance the wifi network on campus (Eduroam) will not be available for twenty minutes on Monday night between 10 pm and 8 pm. During those twenty minutes your mobile device appears to be connected with the wifi network but you will notice there is no internet connection. After this interruption the connection will be established automatically. During the maintenance period additional short interruptions may occur.

Monday evening the wifinetwork will be placed behind the new firewalls of the Radboud University. When this is done, users of Eduroam on our campus are no longer directly reachable from the Internet. We are doing this to protect wifi users against viruses like the Wannacry virus that was used for the worldwide ransomware attack last week.

Network failure server rack

  Begin         : 2017-05-11 09:05
  End           : 2017-05-11 09:25
  Affected      : postvak, barzan (+VMs), bioboost2, bioboost3, vos (+VMs: bfacdc04, gitlab4, smtp), heflo, lilo4, microarb, microclc, lynx (+websites: squirrel, www.cs, ...)

Switch es7-huyg-04-070 of a server rack became powerless after maintenance in the computer room of C&CZ. Restoring power and powering up the switch resolved all problems.

Maintenance network Serverroom FNWI in Huygens building

  Begin         : 2017-05-01 07:30
  End           : 2017-05-01 08:00
  Affected      : employees and students at Faculty of Science

The central network switch in the serverroom at the Faculty of Science will be restarted in connection with some software maintenance. Many servers and services are not available for approx 20 minutes.

RU network maintenance (mainly fixed phones)

  Begin         : 2017-05-08 23:30
  End           : 2017-05-09 02:00
  Affected      : network and fixed phones

Between 23:30 and 00:30 the fixed phones will be out of service for about half an hour. Between 00.30 and 02.00 hours, a few minutes of internet interruption will take place. At those moments, RU services can not be accessed from outside the campus.

Eduroam wifi maintenance

  Begin         : 2017-04-24 23:00
  End           : 2017-04-24 23:59
  Affected      : wireless network

The RU wifi network (eduroam) will be unavailable on Monday evening April 24 between 23:00 and midnight for half an hour due to maintenance work.

Poster printer "kamerbreed" broken

  Begin         : 20170403
  End           : 20170510
  Affected      : Users of the poster printer "kamerbreed"

The printer has to be repaired, but the engineer can't easily fix it. We hope to have the printer operational again in the near future. A new one with service contract has been ordered, in order to make the poster service more reliable. In the meantime you might use the RU copyshop.

Some virtual servers temporarily not available

  Begin         : 2017-04-12 07:22
  End           : 2017-04-12 09:05
  Affected      : Users of virtual machines that run on vos or wolf:
                    - Email: smtp1, smtp2, mx1 (mail servers)
                    - Web services: est2, redmine2 (webserver, redmine server of blackholecam.org)
                    - Windows gebruikers: ms3 (windows login server), bfacdc03, bfacdc04 (windows domain controllers)
                    - GitLab gebruikers: gitlab.science.ru.nl, gitlab.pep.cs.ru.nl (GitLab servers)

An unattended update of the virtual hosting software (libvirt) triggered a netwerk stack restart. This made the virtual machines unavailable on the network. After a reboot of the virtual hosts (vos and wolf), the virtual machines were available again. Action was not taken immediately, because the alarm mails didn't reach us, because the mailservers could not be reached. To improve this situation, steps are taken to route the alarms also via alternative channels (sms, whatsapp, signal, etc). Also the network configuration of the virtual machines has been changed, this will hopefully make it less vulnerable to these updates.

Crash coma00 server

  Begin         : 2017-03-12 22:17
  End           : 2017-03-13 09:08
  Affected      : cluster-astro

Coma00 serves astro0, astro1, astro3 and astro4 volumes, the server crashed on a kernel-BUG, Monday morning the server was restarted and service resumed normally.

Science and RU Internet network maintenance

  Begin         : 2017-03-06 22:00
  End           : 2017-03-07 02:00
  Affected      : network

Monday evening March 6, there will be maintenance on RU network equipment. On March 7 between ca. 01:00-01:30 no network traffic will be possible at the Faculty of Science on the wired and wireless networks. From 23:00 until March 7 02:00 there will be a period where no traffic between RU and Internet is possible. Please think which processes depend on network connectivity and prepare for this outage if necessary.

Science Radius down

  Begin         : 2017-02-28 06:30
  End           : 2017-02-28 09:30
  Affected      : Users of Eduroam who authenticate using their Science login

The Radius service didn't start after the regular reboot, restarting the Radius service solved the problem.

VPNsec upgrade

  Begin         : 2017-02-22 15:30
  End           : 2017-02-22 16:00
  Affected      : VPNsec users

Update of Strongswan server software with extra capabilities

Problem with mail delivery from RU Exchange

 Begin         : 2017-02-06 09:39am
 End           : 2017-02-06 09:58am
 Affected      : Employees with mail in RU Exchange

Monday morning several employees of FNWI were affected by a fault within the RU Exchange design whereby they could not send their mail to recipients outside of the RU Exchange (i.e. @science.ru.nl, @cs.ru.nl or elsewhere). ISC had added two additional servers to the RU Exchange service the week before, which were not yet allowed to drop their outgoing mail on designated servers for that purpose. Not all FNWI employees using RU Exchange were affected, however, since four out of six servers were still functioning properly and employees using e.g. Thunderbird use servers outside of the RU Exchange for sending their mails.

Maintenance network Serverroom FNWI in Huygens building

  Begin         : 2017-01-30 07:00
  End           : 2017-01-30 07:30
  Affected      : employees and students at Faculty of Science

The central network switch in the serverroom at the Faculty of Science will be restarted in connection with an software upgrade. Many servers and services are not available for approx 20 minutes.

VPNsec (and RU VPN) problem with new RU firewall

  Begin         : 2016-12-13 00:00
  End           : 2017-04-19 14:45
  Affected      : VPNsec (and RU VPN) users

The new RU Fortigate firewall, that is in place since December 13, manages fragmented IP packets inconsistently. Depending on the VPN client and its settings, this can cause problems for VPN users. The ISC found a solution for the cenral RU VPN in changing the configuration of the RU VPN as of 2016-12-19 15:50 hours. While waiting for the final solution, a temporary workaround has been used in the Fortigate firewall on April 19, which makes the problem disappear.

Network failure in Huygens wing 8

  Begin         : 2017-01-24 10:50
  End           : 2017-01-24 11:55
  Affected      : several users in Huygens wing 8

In Huygens wing 8 a network switchmodule crashed. Several users lost their connection to the data and telephone network for some time. The switchmodule has been replaced.

Network failure in Huygens wing 5

  Begin         : 2016-12-07 21:15
  End           : 2016-12-08 10:10
  Affected      : several users in Huygens wing 5

In Huygens wing 5 a network switchmodule crashed. Several users lost their connection to the data and telephone network for some time. The switchmodule has been replaced.

Poster printer "kamerbreed" broken

  Begin         : 20161202
  End           : 20161222
  Affected      : Users of the poster printer "kamerbreed"

The printer has to be repaired. Spare parts have been ordered. We hope to have the printer operational again this month.

Maintenance indoor Vodafone cellphone network Huygens

    Begin     : 2016-12-05 ??:??
    End       : 2016-12-05 ??:?? + 1 hour
    Affected  : Vodafone coverage in Huygens building

Due to maintenance of the indoor cellphone equipment, this equipment will be out of service for one hour max on Monday December 5th.

Some machines down due to failed power distribution unit

    Begin     : 2016-11-14 20:40
    Einde     : 2016-11-14 22:50
    Affected : CN69  - CN83, archimedes, nauta, hubel, homedcc

Due to a failed power distribution unit (PDU) a network switch powered down. As a result of that the mentioned machines lost their network connectivity. After reconnecting the switch to a different PDU, the machines were reachable again.

Servers down due to failed power distribution unit

    Begin     : 2016-09-15 00:27
    End       : 2016-09-15 08:28
    Affected  : Homeserver pile, loginserver lilo3, printerserver, mysql server

A few servers powered down due to a failed power distribution unit (PDU), because they were not redundantly connected. After switching to a different PDU, the servers were running again. The redundant connections of important servers will be checked.

RU network maintenance problem

 Begin         : 2016-09-13 01:00am
 End           : 2016-09-13 10:20am
 Affected      : Users at Molecular Biology

After the maintenance activities as mentioned below, a large number of workplaces of Molecular Biology at RIMLS did not receive an ip-address. After a reset by ISC of a table in the central RU-firewall, the problem was solved.

RU network maintenance

 Begin         : 2016-09-12 11:00pm
 End           : 2016-09-13 03:00am
 Affected      : Employees

In the night of 12 to 13 September, maintenance on the Radboud University network will be performed from 11pm to 03am. Several service interruptions may occur, like internet access via SURFnet, mail via Exchange, Bass, Osiris and Blackboard. We advise to check which systems might be affected by this and if possible shift network activity to a different time slot. You also might think about systems with alerts via Internet.

Digital exam environment not available due to maintenance

 Begin         : 2016-07-25 4:30pm
 End           : 2016-07-26 9:50am
 Affected      : Teachers

Due to maintenance, the digital exam environment will not be available for several hours.

Network drives not available after reboot

 Begin         : 2016-09-12 06:30
 End           : 2016-09-12 08:30 home servers, 10:25 departmental printers, 12:45 departmental server
 Affected      : Science users with network drives through SMB

After the Monday morning reboot, the Samba network drive service was not running on all Ubuntu 12.04 servers. Reinitializing all Samba configuration files solved the problem.

Power distribution unit failed

    Begin : 2016-07-05 06:00
    End : 2016-07-05 09:45
    Affected : Clusternodes: cn72, cn74, hinton

Three cluster nodes powered down due to a failed PDU. After replacing the PDU, the nodes came up normally.

Atlas.ti reports: license (almost) expired

 Begin         : 2016-06-17
 End           : 2016-06-24
 Affected      : Users of Atlas.ti

When starting Atlas.ti, you get the message that the license expires in a few days or already has expired. There is, however, a three-weeks grace period during which the users will see a warning message but can still use the program normally. The vendor has promised to send the license June 24, and to send it earlier next years. The new license key can be requested from C&CZ.

License server reboot problem

 Begin         : 2016-05-30 06:30
 End           : 2016-05-30 09:23
 Affected      : Users of licenses: Altera Ansys Accelrys Genesys Hdlworks Idapro
                 Xilinx IntelCompiler Ingr Maple Mathematica Matlab MentorGraphics Mestrelab
                 Topspin Orcad Originlab StateaseRlm

Due to a problem with a new license the license server didn't go down for the weekly reboot. When this came to attention, the server was rebooted and the problem was solved.

Both Science VPNs down for short maintenance

  Begin         : 2016-05-18 07:30
  End           : 2016-05-18 08:00
  Affected      : VPNsec/VPN users

The Science VPNs will be down for short maintenance, in order to switch the IP ranges. After this maintenance, there will be enough IP numbers available for the growing use of the new VPN. The old VPN has not been switched off yet, because Ubuntu 16.04 still has problems with the new VPN.

Gitlab temporarily unavailable

  Begin         : 2016-05-02 21:30
  End           : 2016-05-03 10:15
  Affected      : GitLab users

Because of a major security update that will be released tonight, GitLab will not be available. We will update GitLab on Tuesday morning. After that, gitlab.science.ru.nl will be available again.

Gitlab backup accidentially restored on production server

  Begin         : 2016-04-29 14:30
  End           : 2016-04-25 14:45
  Affected      : GitLab users

A backup file, dated "Fri Apr 29 05:07:04 CEST 2016" has accidentially been restored on our production GitLab server, where it should have happened on a test server. This caused all repositories, wiki's, issues etc. to be reset to the state of today 5.07 AM. Commits pushed between 5 am and 2:30 PM should be pushed again. Sorry for the inconvenience.

Server coma00 Monday morning reboot failed

  Begin         : 2016-04-25 06:30
  End           : 2016-04-25 07:30
  Affected      : all smb + nfs shares coma00

The server comas1 did not start, probably due to some timing problems between the RAID array and OS. Solution: reboot - hardware reset.

License server (lepelaar) reboot problem

 Begin         : 2016-05-25 06:30
 End           : 2016-05-25 09:00
 Affected      : Users of license services hosted on lepelaar.
                 Matlab (until 08:45), Origin (09:00), Mestrenova (9:20), Mathematica (9:30) and Orcad (11:20) service not started. Cause yet unknown.

RU mailserver blacklisted, for Science too

  Begin         : 2016-04-19  17:37
  End           : 2016-04-20  14:13 for Science mailservers
  Affected      : Users of RU mail sending mail out of the RU mailserver

Yesterday the RU outgoing mailserver smtpout3.ru.nl got listed on the SORBS blacklist of confirmed senders of spam and phishing mails. The reason probably is that recently many students fell for phishing mails, which made their username and password known to Internet criminals. These criminals use that to send spam and phishing mails. Because also the Science mailservers use that blacklist, we did not accept mail from the RU mailserver anymore. The senders received an error message: "Your message wasn't delivered due to a permission or security issue." We solved this for our Science users by putting the RU mailserver on our own whitelist.

Power failure

  Begin         : 2016-03-29  10:43
  End           : 2016-03-29  11:20
  Affected      : Various services, including: home directories, printing, authentication services, licenses, vpn,
                  websites, databases, mailman and dns resolvers.

Several services were inaccessible due to power loss of a rack switch caused by a failing PDU (power distribution unit). The PDU has been temporarily replaced.

Hardware replacement webserver

  Begin         : 2016-03-24  12:19
  End           : 2016-03-24  12:30
  Affected      : users of the following web services that run on 'oehoe':
                  oehoe,aerochem,akn,aqua,bgard,boekingen,bomen,cac,cafe,celbi,docenten,exo,filosofie,hfml
                  iconic,ifl,irmacrash,itp2011,iwwr-boookings,kanboard,kndv,labservant,math,mbfys,mbp,mealab,
                  nieuwescheikunde,nlt,pilab,printeger,probiomed,puff,pz,qpl,quantumcs,ra,refbase,rrl,
                  scheikundeolympiade,siam,sigma-lustrum,smartech4health,sws,theorphys,tk,toernooi,tredd,
                  trooster,vsc,w4kangoeroe,wortel,zwaartekrachtsgolven

Memory modules and or the motherboard of webserver 'oehoe' will be replaced. The mentioned web sites will not be available for a while.

Hardware failure webserver

  Begin         : 2016-03-21  14:00
  End           : 2016-03-21  17:00
  Affected      : users of the following web services that run on 'oehoe':
                  oehoe,aerochem,akn,aqua,bgard,boekingen,bomen,cac,cafe,celbi,docenten,exo,filosofie,hfml
                  iconic,ifl,irmacrash,itp2011,iwwr-boookings,kanboard,kndv,labservant,math,mbfys,mbp,mealab,
                  nieuwescheikunde,nlt,pilab,printeger,probiomed,puff,pz,qpl,quantumcs,ra,refbase,rrl,
                  scheikundeolympiade,siam,sigma-lustrum,smartech4health,sws,theorphys,tk,toernooi,tredd,
                  trooster,vsc,w4kangoeroe,wortel,zwaartekrachtsgolven

A defective memory module caused spontaneous reboots. Since two modules where taken out, these reboots have not occurred anymore. Additional down time will be needed to resolve the problem.

Gitlab upgrade

  Begin         : 2016-03-16  21:40
  End           : 2016-03-16  22:05
  Affected      : gitlab users

Because of an upgrade of gitlab to version 8.5.7, the GitLab service was offline for a while.

Server isolator lacking private class networks

  Begin         : 2016-03-07 06:30
  End           : 2016-03-07 09:00
  Affected      : Users of shares/network isolator
  Problem       : For yet unknown reasons the private class network interfaces where incorrectly
                  numbered and activated
  Solution      : Reboot

emoe stuck at reboot

  Begin         : 2016-03-07 06:30
  End           : 2016-03-07 07:45
  Affected      : CnCZ
  Problem       : At reboot time some file system (/opt) was not recognized, resulting in a kernel panic.
  Solution      : Reboot

Gitlab upgrade

  Begin         : 2016-02-26  09:00
  End           : 2016-02-26  10:45
  Affected      : gitlab users

Because of an upgrade of gitlab to version 8.5.1, the GitLab service will be offline for a while.

Virtual servers problem

  Begin         : 2016-02-25  07:25
  End           : 2016-02-25  08:30
  Affected      : medewerkers, studenten, mail versturen

Because of automated updates, Several virtual host had become unreachable. Server vos: gitlab3, bfacdc04, smtp2, redmine2 Server wolf: smtp1, bfacdc03, est2

Rebooted vos and wolf.

miii server problem

  Begin         : 20160223  ??:??
  End           : 20160223  08:50
  Affected      : all smb users miii

For unknown reasons, the miii mail server was unavailable. Rebooted.

IMAP mail server problem

  Begin         : 20160222  06:30
  End           : 20160222  08:03
  Affected      : all users wanting to read Science mail 

For unknown reasons, the IMAP mail server was unavailable. The service cyrus-imapd was not correctly started. Starting this service service cyrus-imapd start solved the problem.

IMAP mail server problem

  Begin         : 20160215  11:20
  End           : 20160215  13:45
  Affected      : all users wanting to read Science mail 

For unknown reasons, the IMAP mail server was unavailable. Mail services were halted after some time because of logging activity. As a temporary workaround logging is now performed using UDP instead of TCP.

Fileserver problems

  Begin         : 20160209 after 09:00
  End           : 20160209 15:37
  Affected      : Users of shared network disks

The Ubuntu 14.04 fileservers "flock" and "muckle" had problems a few times. After we reverted to the previous kernel version, the service seems to be stable again.

Matlab license server crashed

  Begin1         : 2016-02-08 20:04
  End2           : 2016-02-08 08:58
  Begin2         : 2016-02-09 10:41
  End2           : 2016-02-09 10:51
  Affected       : Matlab users

Last night the Matlab license server crashed, because the number of open file handles reached the maximum configured, because a user started a lot of Matlab processes. Because we do not yet monitor this properly and it was not reported to us, we could only repair it this morning. We increased the number of file handles by a factor of four. We assume that this will be sufficient to wait till the reboot of next Monday, when it will again be enlarged, this time by a factor of sixteen. We will also on short notice start to monitor the Matlab license server properly.

Server lilo4 Monday morning reboot slowed

  Begin         : 2016-02-01 06:30
  End           : 2016-02-01 09:40
  Affected      : fnwi login server users

The server lilo4 stalled at shutdown during the monday morning reboot. Solution: it continued its reboot after server wielewaal became available.

Webserver down: tftp & egw

  Begin         : 2016-02-01 06:30
  End           : 2016-02-01 09:20
  Affected      : websites of this server (wielewaal): blackgem brainhealth croatia14 croatia15 cs eduspec eduspecorg egw
                  ethergids fbkr kickstart mhealth oni snn

Quite a number of websites of ICIS were not available amongst which was eGroupWare. Furthermore, the boot process of workstations was impaired (because of the missing tftp). A reboot of the server seems to be sufficient to solve the issue.

No free IP numbers in Vlan136

  Begin         : 2016-01-21 11:44
  End           : 2016-01-21 16:23
  Affected      : 6 laptops of staff departments of the Faculty of Science

The DHCP server didn't have free leases on Vlan136, so new devices on Vlan136 didn't get an IP address, so didn't have a working network connection. After we freed some unused numbers for PCs with a fixed address, we were able to drastically enlarge the number of leases on Vlan136.

Peter van Campen 22 jan 2016 17:46 (CET)

Network failure in part of study area Huygens

  Begin         : 2016-01-20 12:57
  End           : 2016-01-20 13:25
  Affected      : 30 pc's at the Study Area Huygens

In Huygens wing 2 a network switchmodule crashed twice. About 30 pc's in the Study Area lost their connection to the network for some time. The switchmodule will be replaced on 2016-01-21 at 07:30 am.

Marcel Kuppens 20 jan 2016 15:14 (CET)

Login servers down to activate safer kernel

  Begin         : 2016-01-20 09:30
  End           : 2016-01-20 10:00
  Affected      : lilo3 en lilo4 login servers.

simon 20 jan 2016 10:48 (CET)

Webserver down to add hard disks

  Begin         : 2016-01-15 17:00
  End           : 2016-01-15 17:30
  Affected      : websites of this server: aerochem akn aqua bb2syllabus bgard boekingen boekwur bomengids cac
                  cafe celbi dossiers est2b exo filosofie fnwi hfml iconic ifl irmacrash
                  itp2011 iwwrbook kanboard kndv labservant leonardo math mbfys mbp
                  nieuwescheikunde nlt notebooks olympus pilab piwik printeger projecten
                  puff pz qpl quantumcs refbase resourceanalysis rrl scheiolymp setup
                  siam sigma-lustrum sito smartech4health studiereis sws tango taskboard
                  tk toernooi tredd trooster ubuntumirror vsc w4kangoeroe wortel

Virtual servers network interfaces broken after updates

  Begin         : 2016-01-13 07:16
  End           : 2016-01-13 09:14
  Affected      : bfacdc03 bfacdc04 drukker est2 gitlab3 ms2
                  nwidc01 redmine2 smtp1 smtp2 ts2

The virtual hosts fret, wolf and vos received updates to libvirt packages, causing the network interfaces of the virtual machines running on them to fail. Solved after reboot (fret) or restart of the libvirt daemons and the virtual machines.

Server lilo3 and lilo4 Monday morning reboot failed

  Begin         : 2016-01-11 06:30
  End           : 2016-01-11 07:30
  Affected      : fnwi login server users

The server lilo3/4 stalled at shutdown during the monday morning reboot. Solution: power cycle. Cause currently unknown.

Server comas1 Monday morning reboot failed

  Begin         : 2016-01-11 06:30
  End           : 2016-01-11 07:20
  Affected      : all smb + nfs shares comas1

The server comas1 did not start due to some possible timing problems between the raid array and OS. Solution: skipping the mount of /vd0 and /vd1 during boot (key S), mounts fixed manually later on followed by exportfs -uaf; exportfs -raf.

  Begin        : 2016-01-05 06:30
  End          : 2016-01-05 14:30
  Affected     : Users FTP
  Problem      : Machine failed for unknown reason at boot. Solved by rebooting.

RU Internet connection down

  Begin         : 2015-11-23 16:52
  End           : 2015-11-23 16:09
  Affected      : All users of the RU Internet connection

Yesterday at ca. 15:52 a SURFnet core router crashed, taking down the RU Internet connection. At 16:09 the connection was restored. This was reported yesterday on a temporary SURFnet service interruptions page and on the ISC/CIM service interruptions page. Details can be found on the SURFnet Network Tickets mailing list.

Print server printto problems

  Begin         : 20151119 ???
  End           : 20151119 13:00
  Affected      : Users wanting to attach printers through \\printto.science.ru.nl

After a reboot of the print server (and waiting ca. 10 minutes) printers could be attached again.

Problems availability wireless network eduroam

  Begin         : 20151116 08:00
  End           : 20151116 09:30
  Affected      : All eduroam users at the RU

As a result of a short network service interruption last night, there were some problems with obtaining ip addresses on the wireless network eduroam.

Fileserver goudsmit problems

  Begin         : 20151104 23:20
  End           : 20151105 10:30
  Affected      : Users with home directories / data on the goudsmit fileserver

System crashed possibly caused by an erroneous harddisk causing all CPUs to lockup.

RU Internet connection down for maintenance

  Begin         : 2015-10-20 00:00
  End           : 2015-10-20 03:00
  Affected      : All users of the RU Internet connection

The ISC has announced that on Tuesday morning, October 20, 2015 between 00:00 and 03:00 SURFnet and the ISC will perform maintenance on the connection of the RU with SURFnet/Internet. From 00:00 to 01:30 there is no connection. Between 01:30 and 03:00 short interruptions may occur. We advise to check which systems can be affected by this and if possible shift network activity to a different time slot. You also might think about systems with alerts via Internet.

Konica Minolta MFPs switched

  Begin         : 2015-10-08 08:00
  End           : 2015-10-08 12:00
  Affected      : Users of the KM MFPs:
                  pr-hg-00-011 pr-hg-00-089 pr-hg-00-627 pr-hg-00-825
                  pr-hg-02-038 pr-hg-02-556 pr-hg-02-832

In the morning of October 8, some KM MFPs will be switched, in order to have the faster/bigger machines at the locations where the print volume is te largest. The replacement of each machine will not take much time.

Server lilo4 Monday morning reboot failed

  Begin         : 2015-10-05 06:30
  End           : 2015-10-05 07:45
  Affected      : fnwi login server users

The server lilo4 stalled at shutdown during the monday morning reboot. Solution: power cycle. Cause currently unknown.

Radboudumc buildings power switched off for test

  Begin         : 2015-09-30 ca. 16:00
  End           : 2015-09-30 ca. 16:45
  Affected      : Radboudumc buildings

Wednesday September 30, around 4:00 p.m., Radboudumc will carry out a test by switching off the power to almost all their buildings. For the RU buildings this test has no direct effect. Liander states that switching off and on again will have no noticeable effect on the rest of the power net in Nijmegen including Radboud University. However, switching high power always has the possibility of introducing a power problem elsewhere. If research, education or any activity will be seriously hindered by a power problem, you might consider ending this activity in time or taking measures to guarantee continuity with emergency power and UPS.

Brief network interruptions at FNWI

  Begin         : 2015-09-12 09:00
  End           : 2015-09-12 13:00
  Affected      : Network Huygens building and peripheral buildings

Saturday September 12, between 9:00 a.m. and 1:00 p.m., maintenance work will be conducted on the network router at the Faculty of Science and a number of server switches in C&CZ data centers. This means that between these points in time, a few brief network interruptions with a duration of approximately 1 minute will occur. Many FNWI servers will be unreachable at those moments. The Mitel IP phones will also go out of service momentarily several times. All other phones, however, will continue to operate normally. The maintenance work is necessary to enable resumption of the IP telephony rollout at the Faculty of Science. For further information, please contact .

Mailman mailing list problem

  Begin         : 2015-09-07 06:30
  End           : 2015-09-07 12:24
  Affected      : Users of mailing lists

This morning we came to know that mails to mailing lists were not accepted by the mailing list server. The reason was a change that was made last Friday because of the new smtp server mid July. After correcting the change, mails were accepted again.

Network shares unavailable after reboot

  Begin         : 2015-09-07 06:30
  End           : 2015-09-07 09:25
  Affected      : Users of the following network disks: acs4 afric afstud afstudeerwerk afstudTest albert
                  bin bioalg bioexamencie bko BKO-TEMPLATE ccz cczhelpdesk cczsecr celbio2 celbio3 CuDoTest CURSUS
                  cursusdossiers diafragma ecogen ecologie embdata evsf2data10 evsf2data15 evsf2data16 evsf2data17
                  evsf2data18 evsf2data19 evsf2data7 evsf2data8 evsf2data9 fsr geminstr grafting honours huckgroup
                  hybrids ifl-d1 imappdir imappsecr immbureau iobw kolkteaching localdist logs mbaudit3 mbbioel2
                  mbonderwijs mbsynid micro-bulk1 micro-bulk2 micro-labmembers milkun8 milkun9 mlf90 mlfalgemeen
                  mlfsec molchem3 morph mpcoldmollab mppstaf nmr500data nmrspectra olcbiocie olcbiowetenschappen
                  orgdier owc1 parp petseq project1mpp seq snn3 snn4 soheto studiereis2013 sucudo-transfer tf2devices
                  tracegas unl vcbio_prive vergader vsc16

During the move this weekend of network shares to a new server, an administrative error was made, that prevented the server from rebooting ok. After correcting this, the server booted ok.

Water damage for pr-hg-03-825

  Begin         : 2015-08-31
  End           : 2015-09-03
  Affected      : Users of the KM MFP pr-hg-03-825

A leak above the KM MFP hg PR-03-825 made this fail beyond repair. We hope to resolve this by replacing it by a little-used KM MFP of the same type.

Interruption public telephone network

  Begin         : 2015-08-31 14:00
  End           : 2015-08-31 22:00
  Affected      : Entire Campus

Since 14:00 this afternoon it's not possible to reach phones outside the campus. Also, only external phones on the KPN network and cell phones from all providers can reach the campus. For central RU service interruptions and planned maintenance, see the combined ISC/CIM interruptions page.

Network shares unavailable during migration

  Begin         : 2015-08-06  8:45
  End           : 2015-08-06 ~9:45
  Affected      : Users of the following network disks: 45t, acfiles2, acs4, afric, afstud, afstudeerwerk,
                  afstudTest, albert, arb, bbb-priv, bin, bioalg, bioexamencie, birds, bko, botany-general,
                  bsweet, carta, ccs4, ccz, cczhelpdesk, cczsecr, celbio2, celbio3, CuDoTest, cursus,
                  cursusdossiers, dcnadmin, diafragma, ecogen, ecologie, embdata, evsf2data10, evsf2data15,
                  evsf2data16, evsf2data17, evsf2data18, evsf2data19, evsf2data7, evsf2data8, evsf2data9,
                  exoarchief, flaredisc, fsr, geminstr, gi1, gi3, gipsy, grafting, hfml-data, hfml-engineering,
                  hfml-secr, honours, huckgroup, hybrids, icissec, ifl, ifl-d1, imappdir, imappdir, imappsecr,
                  immbureau, iobw, iris, itt, iwwr1, kolkteaching, lambiek, localdist, logs, mailmanincludes,
                  mbaudit3, mbbioel, mbbioel2, mbcns, mbcortex, mbonderwijs, mb-read-write, mbsynid, mercator,
                  mestrelab, mi1, mi3, micro-bulk1, micro-bulk2, micro-labmembers, milkun4, milkun4rw, milkun8,
                  milkun9, mlf90, mlfalgemeen, mlfsec, molchem, molchem2, molchem3, molphtec, mol-secr, morph,
                  mpcoldmollab, mppstaf, multimedia, neuropi, nmr500data, nmrspectra, nskpromo, olcbiocie,
                  olcbiowetenschappen, ons, orgdier, owc, owc1, parp, pcb, petseq, plantkunde-hgl, practica,
                  project1mpp, rotax, seq, smol, snn, snn2, snn3, snn4, soheto, sos, spmdata1, SSNMR1,
                  studiereis2013, sucudo-transfer, tcbackup, tcscratch, tece, teceleiding, tf2devices, thchem,
                  tracegas, tracegastemp, tracelab, tracelabdb, unl, vcbio_prive, vergader, vsc16, vscxray,
                  wikis, wiskundetoernooi and zeegras

Starting from 8:45h, the network disks will not be available during a migration to a new server.

Server vos (gitlab3/redmine2/smtp2) preventive maintenance

  Begin         : 20150731 09:00
  End           : 20150731 16:00
  Affected      : gitlab/remine users

The server vos will receive extra disks and need to be reconfigured. Virtual clients gitlab3, redmine2 and smtp2 will be offline during maintenance.

SMTP server problem because of too short keys

  Begin         : 2015-06-13
  End           : 2015-07-15
  Affected      : Employees sending autheticated mail through smtp.science.ru.nl

Starting a month ago, mail programs could get an update due to the vulnerability for a LogJam attack. After this update these programs refused connections with a server with a too short key. The old C&CZ authenticated SMTP server had such a short key. Users who reported this problem (with the latest versions of Pine, Thunderbird and recently also Apple Mail), were advised to temporarily use an alternative server. As of today using this alternative server is no longer necessary, because the server smtp.science.ru.nl has been replaced by a new server.

Network shares temporary not available during migration

  Begin         : 2015-06-26 ~8:45
  End           : 2015-06-26 ~9:45
  Affected      : Users of the following network disks: acfiles, aerochem, amsbackup, amscommon,
                  beevee, bfac-backup, bio-orgchem, bioniccell, carisma, chemprac, csg-staf,
                  csgi-archief, desda, ds, ehef1, ehef2, ehef3, geminstr2, giphouse, gissig,
                  gmi, hfml-backup2, highres, impuls, introcie, isis, janvanhest, kaartenbak,
                  kangoeroe, mbaudit1, microbiology, microbiology2, milkun2, milkun3, milkun5,
                  milkun6, milkun7, milkun7rw, molbiol, mwstudiereis, neuroinf, neuroinf2, nfstest,
                  nwibackup, olympus, puc, ruversatest, secres2, secrmolbiol, sigma, sigmacies,
                  spectra, spm, spmdata3, spmdata4, spmdata5, spmdata6, spmdata7, spmmagstm,
                  spmnanolab, spmstaff, ssi, staff-hfml, stroom, thalia, ucm, vsc, vsc1, vsc10,
                  vsc11, vsc12, vsc13, vsc14, vsc15, vsc2, vsc3, vsc4, vsc5, vsc6, vsc7, vsc8,
                  vsc9, vscadmin, wiskalg, wiskunde and wkru.

Starting from 8:45h, the network disks will not be available during a migration to a new server.

Server homedcc had old homes after reboot

  Begin         : 2015-06-15 06:30
  End           : 2015-06-15 15:30
  Affected      : Employees of DCC

After the server reboot on Monday morning, the changes to the home directories of the last two weeks had disappeared. It is still not clear how this could have happened. The backups of Saturday June 13 22:55 have been restored.

Services unavailable due to power outage

  Begin         : 2015-06-11 20:00
  End           : 2015-06-11 20:30
  Affected      : Computer systems and network switches that don't have a backup power supply (UPS). Among others FELIX.

A short power dip interrupted desktop computers and some network switches without UPS backup power on the RU campus. It took approx. 20 minutes before the affected switches became fully operational again. A network switch at the Transitorium reported a failed module until it was reset around 8:30 am on Friday morning. For ca. 46 outlets/users the network didn't function until 8:40. The UVB reported as cause a cable cut at the Kapittelweg. Liander reports a power disturbance an hour later at that location.

Fileserver tcmr problems

  Begin         : 2015-06-05 16:05
  End           : 2015-06-05 16:55
  Affected      : Employees of TCM

Resolved by a server reboot.

Ubuntu 10.04 cluster / cn99 server problem

 Begin         : 2015-06-03 12:10
 End           : n/a
 Affected      : Users of Ubuntu 10 clusternodes

The server (cn99) which serviced SUN Grid Engine on Ubuntu 10.04 is no more. It could not find its disks while booting. This will not be solved because the machine is already more than five years old. Clusternodes cn67, cn74, cn75 and cn81 are influenced. They can still be used by logging into them directly.

GitLab server problems

 Begin         : 2015-05-30 07:34
 End           : 2015-05-31 00:36
 Affected      : Users of the GitLab server gitlab.science.ru.nl

A configuration error caused the automatic installation of an older version of GitLab (7.7.1) over the the most recent version (7.11.4). Because the GitLab version did not correspond to the state of the underlying database, what caused multiple problems. Among which the gitlab shell for ssh access and inaccessibility of some parts of the web interface. The problems were solved after upgrading to version 7.11.4.

License server reboot problem

 Begin         : 2015-05-11 06:30
 End           : 2015-05-11 09:11
 Affected      : Users of licenses: Altera Ansys Accelrys Genesys Hdlworks Idapro
                 Xilinx IntelCompiler Ingr Maple Mathematica Matlab MentorGraphics Mestrelab
                 Topspin Orcad Originlab

Today the license server booted without configuring the network. The cause of the problem was an error in a configuration file. This has been corrected.

IMAP mail server upgrade next monday night

  Begin         : 20150216 23:00
  End           : 20150217 07:45
  Affected      : all users wanting to read Science mail 

The IMAP server will be replaced by a new server next Monday night. During the night, no Science mail can be read from the server, because then the most recent mails will be synchronized to the new server. Mail can be sent in this period, but new mail will start being delivered on the new server from 07:30. A word of warning, one could login on the old server with the first 8 characters of the password, the new server only accepts the full password.

Network maintenance in Huygens wing 8

  Begin         : 20150211     18:00
  End           : 20150211     20:00
  Affected      : network in Huygens wing 8 and in the central street between wings 6 and 8 (east side), all floors.

The configurations in the network switches in the buildings/floors/locations mentioned above, will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible.

Network switch problem

  Begin         : 20150204  16:42
  End           : 20150204  16:59
  Affected      : all users at Faculty af Science

Due to a configuration fault in a C&CZ server switch it became unreachable. Therefore some servers and services were unreachable as well (nameserver, mail etc). After a quick fix the systems were up and running again.

Power dip Huygens building Saturday, January 17

  Begin         : 20150117  07:54
  End           : 20150117  09:11
  Affected      : network Huygens building, incl wireless and IP telephony

Saturday morning, during maintenance by UVB and Liander, there were a few short unintended power dips in the Huygens building. A large part of the network switches does not have a UPS and thus restarted. After 09:11 all switches were operational again.

Background: January 10 and 17, maintenance has been carried out on the power transformers in the Huygens building. This work should be carried out once every 5 years. Because a power failure of 6-8 hours (duration of maintenance) is unacceptable for the Faculty of Science, last year bypasses have been constructed, to make the transformers able to replace each other. Beforehand, the chance that a power failure should occur, was thought to be particularly small. The next time such a service is planned, in 2020, everyone will be informed in advance.

Upgrade of license server

  Begin         : 2015-01-08 17:30
  End           : 2015-01-08 17:45
  Affected      : Users of licenses of: Ansys Accelrys Genesys Idapro IntelCompiler Mathematica Matlab Orcad Originlab

The license server still runs Ubuntu 10.04. This means that in a few months it would not receive security updates. Furthermore we want to use this server for DNS/DNSSEC/DHCP-service, which makes Ubuntu 14.04 necessary. Because this is a license server, we cannot switch to a new server, but we will place disks with the new OS in the existing server. Because this only means a small amount of downtime, users will hardly notice.

Fileserver bulk problems

  Begin         : 2015-01-07 14:35
  End           : 2015-01-07 14:46
  Affected      : Users of shared network disks:
                  acfiles beevee bioniccell bio-orgchem celbio1 celbio2
                  chemprac csgi-archief csg-projecten csg-staf desda ds ehef1
                  ehef2 ehef3 femtospin geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 hfml-backup-inst highres impuls
                  introcie isis janvanhest kaartenbak kangoeroe mathsolcie microbiology microbiology2
                  milkun2 milkun2RenD milkun3 milkun5 milkun6 milkun7 milkun7rw
                  molbiol mwstudiereis neuroinf neuroinf2 olympus puc randomwalkmodel secres2 sigma sigmacies spectra spectra-rw
                  spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmstaff ssi staff-hfml stroom thalia ucm vsc
                  vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde wkru

Resolved by a server reboot.

Again IMAP mail server problem

  Begin         : 20141224  17:46
  End           : 20141225  09:47
  Affected      : all users wanting to read Science mail 

There was again a problem with another harddisk of the IMAP server. This made the server unusable. A reboot solved the problem.

IMAP mail server problem

  Begin         : 20141224   9:20
  End           : 20141224  11:05
  Affected      : all users wanting to read Science mail 

The IMAP server got overloaded again which could not be due to user usage. The server was rebooted since all I/O processing was stopped and we could not kill any processes. Snapshots have now been completely disabled. It took quite some time before the server had all disks and filesystems in working order.

IMAP mail server problem

  Begin         : 20141215   6:30
  End           : 20141215  11:15
  Affected      : all users wanting to read Science mail 

There was a problem with a failed harddisk of the IMAP server. This slowed the machine down so that it was practically unusable. The problem was solved by replacing the disk.

Network problem RU

  Begin         : 20141126     09:45
  End           : 20141126     10:00
  Affected      : all users of the network at RU

As reported by the ISC, the RU was victim of a DDOS attack. As a consequence of this is the connection to the Internet was interrupted. Measures have been taken to prevent a similar attack having this result in the future.

Network problems FNWI

  Begin1        : 20141111     06:00
  Begin2        : 20141119     13:34
  End           : 20141120     15:32
  Affected      : all users of the wired network at FNWI

After the network maintenance of 20141111 a single user reported later a problem with the wired network. These problems intensified and grew faculty wide on 20141119 at 13:34 hours. The network switches in the Faculty of Science received so many topology changes of the Spanning Tree Protocol (STP) that network traffic often stuttered. By curbing the STP traffic on 20141120 at 15:32 the stuttering disappeared. We still search for the exact cause, probably it is a housing move.

Network maintenance in Mercator III south and low-rise buildings

  Begin         : 20141119     18:30
  End           : 20141119     23:00
  Affected      : network in Mercator III north

The network switches in the buildings/floors/locations mentioned above will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible. This includes the network for access control, climate control etc.

Server nachtwacht preventive maintenance

  Begin         : 20141117 16:30
  End           : 20141117 17:00
  Affected      : n/a (no disturbance for users expected)

The server nachtwacht reports having a hard drive that will soon fail. We will replace the disk as soon as possible.

DNS resolver problem

  Begin1         : 20141115     11:29
  End1           : 20141115     14:40
  Begin2         : 20141117     04:30 / 06:30
  End2           : 20141117     08:37 for DNS, 09:34 for network boot and websites
  Affected       : almost all computers within FNWI

The server that acts within FNWI as first DNS resolver crashed, probably because of a disk becoming defective. This made the network slow or even virtually unusable for a lot of computers in FNWI. Only after a reboot of the server, the problem1 was solved. Monday morning, at the weekly reboot of this server, it would only boot after manually acknowledging the defective disk (problem2), When other servers rebooted at 06:30, a lot of them had problems due to the missing DNS resolver. Further measures to reduce the inconvenience of such a crash have been taken. The move to new DNS servers is in preparation.

Fileserver bulk problems

  Begin         : 2014-11-12 14:45
  End           : 2014-11-12 15:05
  Affected      : Users of shared network disks: acfiles ARK-Backup Bargerveen Bargerveen-Gebruikers beevee bioniccell bio-orgchem B-Ware-Arnica B-Ware-Dotter B-Ware-Erica B-Ware-Knolrus B-Ware-Lobelia B-Ware-Stratiotes celbio1 celbio2 csgi-archief csg-staf cvi desda ds ehef1 evsf2dataoud evsf2schijf1 femtospin Floron geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 highres impuls introcie isis janvanhest kaartenbak kangoeroe microbiology microbiology2 milkun2 milkun3 milkun5 milkun6 milkun7 milkun7rw molbiol mwstudiereis neuroinf neuroinf2 nfstest nwibackup olympus puc RandomWalkModel Ravon-Algemeen Ravon-Foto Ravon-Projecten secres2 sigma sigmaalmanak sigmacies sigmaexchange sigmasymposium splex spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde

Resolved by a server reboot.

Network maintenance in Mercator III north

  Begin         : 20141111     18:30
  End           : 20141111     23:00
  Affected      : network in Mercator III north

The network switches in the buildings/floors/locations mentioned above will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible. This includes the network for access control, climate control etc.

Network maintenance in server room Huygens

  Begin         : 20141111     06:00
  End           : 20141111     06:15
  Affected      : network in server room Huygens. Connection to many servers may be impossible.

Because of a software upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be rebooted at 06:00h (AM) exactly. This means that both wired en wireless networks at these locations will be interrupted for 10 to 15 minutes and network traffic will not be possible. This includes the network for access control, climate control etc.

Network maintenance in part of Huygens

  Begin         : 20141106     06:00
  End           : 20141106     06:15
  Affected      : network in Huygens wing 3 all floors
                             Huygens wing 4 all floors
                             Huygens wing 8 all floors
                             Huygens central street between wing 6 and 8 all floors

Because of a software upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be rebooted at 06:00h (AM) exactly. This means that both wired en wireless networks at these locations will be interrupted for 10 to 15 minutes and network traffic will not be possible. This includes the network for access control, climate control etc.

Network maintenance in peripheral buildings

  Begin         : 20141104     06:00
  End           : 20141104     06:15
  Affected      : network in peripheral buildings: FELIX, HFML, Nanolab, Goudsmit pavillion (NMR),
                  Lab DCN in A-2.002, Kinderopvang Heyendael (both buildings),
                  Logistiek Centrum FNWI, Linnaeus building, Mercator 1, 6th floor

Because of a software upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be rebooted on Tuesday morning at 06:00h (AM) exactly. This means that both wired en wireless networks at these locations will be interrupted for 10 to 15 minutes and network traffic will not be possible. This includes the network for access control, climate control etc.

Network maintenance in Huygens wing 7

  Begin         : 20141029     19:00
  End           : 20141029     23:00
  Affected      : network in Huygens wing 7 and in the central street between wings 5 and 7, all floors.

Because of an upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible. This includes the network for access control, climate control etc.

RU network problem

  Begin         : 20141029     09:38
  End           : 20141029     15:30
  Affected      : everything connected to the network within RU until 9:45, after that Eduroam users.

Due to a short DDOS-attack the RU was virtually disconnected from the Internet during ca 7 minutes. All network connected systems may have had problems during that time. Until 15:30 some Eduroam users still had problems acquiring an IP address.

FNWI network problem

  Begin         : 20141015     03:44
  End           : 20141015     ca. 11:36 
  Affected      : almost everything connected to the network within FNWI (Heyendaal East)

The router that regulates all IP traffic between subnets in the Huygens building and surrounding buildings, became too busy. All network traffic, especially UDP had problems. ISC network admins reduced the load by simplifying the ACLs and reducing logging. At about 12:00 the problem was temporarily solved. C&CZ will do a further inspection of all ACLs, with the aim to reduce the load on the router without reducing security for connected systems.

DNS resolver problem

  Begin         : 20140920     10:45
  End           : 20140920     14:04 
  Affected      : almost all computers within FNWI

The server that acts within FNWI as first DNS resolver crashed. This made the network virtually unusable for a lot of computers in FNWI. Only after a reboot of the server, the problem was solved. Measures to reduce the inconvenience of such a crash and to make such a crash less likely, have been partly taken and are partly in preparation.

Disturbance of Vodafone VWO data traffic

  Begin         : 20140924 08:30
  End           : 20140924 17:00
  Affected      : Employees with a RU Vodafone VWO mobile phone

On the ISC website we read:

Radboud University agreed on a new mobile telephone contract with - again - provider Vodafone. For this purpose, on Wednesday, September 24th during daytime the Radboud subscriptions will be adjusted. Employees with a Vodafone mobile wille have no disturbance in calling or being called that day. But the Vodafone data network might be temporarily unavailable. To restore the data connection it is necessary to turn the mobile phone off and then on again. Of course employees on campus can always use the eduroam Wi-Fi network.

Webserver (klopjacht) disk problem

  Begin         : 20140909 10:46
  End           : 20140909 11:50
  Affected      : Dozens of websites on this webserver

A defective disk made the server stop serving web requests. A reboot fixed the problem. The defective disk will be replaced soon.

Surfnet uplink offline

  Begin         : 20140908 22:00
  End           : 20140909  0:03
  Affected      : All network traffic in and outbound the RU.

Due to an error during the planned maintenance by Surfnet of the RU network interface, the uplink of Radboud University to Surfnet was down.

Fileserver goudsmit problems

  Begin         : 20140908  6:30
  End           : 20140908 10:43
  Affected      : Users with home directories / data on the goudsmit fileserver

One of the filesystems contained errors and prevented the system from starting up.

SMTP server offline for half an hour

  Begin         : 20140821 16:00
  End           : 20140821 16:30
  Affected      : Users of the science mail service

Both processor fans have failed and need to be replaced.

Fileserver bulk problems

  Begin         : 20140815 12:04
  End           : 20140815 14:25
  Affected      : Users of shared network disks: acfiles ARK-Backup Bargerveen Bargerveen-Gebruikers beevee bioniccell bio-orgchem B-Ware-Arnica B-Ware-Dotter B-Ware-Erica B-Ware-Knolrus B-Ware-Lobelia B-Ware-Stratiotes celbio1 celbio2 csgi-archief csg-staf cvi desda ds ehef1 evsf2dataoud evsf2schijf1 femtospin Floron geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 highres impuls introcie isis janvanhest kaartenbak kangoeroe microbiology microbiology2 milkun2 milkun3 milkun5 milkun6 milkun7 milkun7rw molbiol mwstudiereis neuroinf neuroinf2 nfstest nwibackup olympus puc RandomWalkModel Ravon-Algemeen Ravon-Foto Ravon-Projecten secres2 sigma sigmaalmanak sigmacies sigmaexchange sigmasymposium splex spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde

The fileserver had problems, which were resolved after a server reboot.

M1.04 no network

  Begin         : 20140814  11:32 partly - 16:09 complete
  End           : 20140815  09:28 partly - 11:46 complete
  Affected      : Users in Mercator1 4th floor

While administering the network switchports for the move of ISIS to Mercator1 floor 0 to 3, ISC network management erroneously also changed the configuration of the network switchports of floor 4. When this was reported this morning, it could be readily remedied.

Failure cn56 disk

  Begin         : 20140707   8:00
  End           : 20140707  11:00
  Affected      : Users of cluster node cn56 and Sun Grid Engine on Ubuntu 12.04 systems

One of the mirrored disks failed. We didn't manage to rebuild the raid set with a new disk. The system is reinstalled with two new disks. After that, data from /scratch could be recovered from one of the old disks.

Reboot of login servers after network drive migration

  Begin         : 20140605  16:00
  End           : 20140605  ~17:00
  Affected      : Users of e.g. the Linux login servers (lilo2, lilo3, lilo4)

The planned move of network drives from an old to a new server caused a lot of "stale NFS file handles", which made it necessary to reboot some login servers.

Network drive migration

  Begin         : 20140605  16:00
  End           : 20140605  ~17:00
  Affected      : Users of the following network drives:
                  acfiles, bioniccell, bio-orgchem, celbio1, CSGI-Archief,
                  Csg-Staf, ds, evsf2schijf1, giphouse, gissig, gmi, hfml-backup,
                  hfml-backup2, highres, impuls, introcie, isis, kaartenbak,
                  microbiology, microbiology2, microbiologyftp, milkun2, milkun3,
                  milkun5, milkun6, milkun7, milkun7rw, mwstudiereis, neuroinf,
                  olympus, permissies, puc, secres2, sigma, sigmaalmanak, sigmacies,
                  sigmaexchange, sigmasymposium, Soheto, splex, spmdata3, spmdata4, 
                  spmdata5, spmdata6, spmdata7, stroom, thalia, ucm, vsc, vsc1, vsc2,
                  vsc3, vsc4, vsc5, vsc6, vsc7, vsc8, vsc9, vsc10, vsc11, vsc12,
                  vsc13, vsc14, vsc15,wiskalg, wiskunde

The network drives will be migrated to a new server. During the migration, the network drives will be unavailable.

IMAP mail server problem

  Begin         : 20140605  14:05
  End           : 20140605  14:15
  Affected      : all users wanting to read Science mail 

The IMAP server got overloaded again. Restarting the IMAP service was required to bring it back in operation. We think it is related to snapshots and therefore regularly remove invalid snapshots from now on.

IMAP mail server problem

  Begin         : 20140527  14:00
  End           : 20140527  14:10
  Affected      : all users wanting to read Science mail 

The IMAP server got overloaded. Restarting the IMAP service was required to bring it back in operation.

Data/file/vol server (heap) problem

  Begin         : 20140526 06:45
  End           : 20140526 13:45
  Affected      : Users of this data/file/vol server

The server had a problem, which had started at the Monday morning reboot. A reboot of the machine solved the problem. Soon all data will be moved to a new server. This will not cause much problems for users, because everybody uses the aliases something-srv.science.ru.nl.

Poster printer "kamerbreed" broken

  Begin         : 20140421
  End           : 20140516
  Affected      : Users of the poster printer "kamerbreed"

The printer has to be repaired. Spare parts have been ordered. We hope to have the printer operational again mid-May.

Maintenance wireless infrastructure May 19

  Begin         : 20140519 8:00 pm
  End           : 20140519 12:00 pm
  Affected      : Users of Eduroam and Ru-guest

At Monday May 19, a major upgrade of the wireless infrastructure and its maintenance tool (called Prime) will take place by the ISC. Therefore the wireless service (i.e. Eduroam) will be interrupted several times from 8:00 pm. Existing wireless connections will be lost and new connections will not be possible for some time. Due to the complexity of this operation this maintenance will possibly be continued at Monday, May 26, again with interruptions from 8:00 pm.

Maintenance Goudsmit

  Begin         : 20140512 13:00
  End           : 20140512 14:21
  Affected      : Users of the goudsmit

One of the systemdisks reported SMART-errors and will be replaced.

lilo.science.ru.nl offline

  Begin         : 20140512  6:30
  End           : 20140512  9:36
  Affected      : Users of the linux loginserver lilo

Lilo did not shut down correctly during the planned weekly reboot. After a hardware reset, the machine booted just fine

And yet again Print/phpMyAdmin server problem

  Begin         : 20140508 16:24
  End           : 20140508 16:33
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Just like yesterday, a reboot of the machine was necessary to restore the functionality. Tomorrow morning, the server will restart with a new kernel. It this doesn't solve the problem, we will look into a new version of Samba.

Yet again Print/phpMyAdmin server problem

  Begin         : 20140507 12:49
  End           : 20140507 13:14
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Just like yesterday, a reboot of the machine was necessary to restore the functionality. We will investigate if we can remediate this problem with newer software versions (Samba, Ubuntu, ...)

Eduroam problem

  Begin        : 20140506 13:40
  End          : 20140506 18:29
  Affected     : Eduroam users

Some Eduroam users didn't get an IP number, because one of the IP ranges had no free leases. This ranges contained all @science.ru.nl users. The ISC networking department first decreased the lease time. Now they also make this range free for use by @science.ru.nl users. This will prevent the problem from occuring again soon.

U-number authentication problem Eduroam

  Begin        : 20140430 ca 13:00
  End          : 20140501 16:25
  Affected     : Employees using  for Eduroam authentication, primarily within FNWI

The new RU IdM system erroneously rewrote the RU password hash when data about a person changed. This primariliy affected employees of FNWI, because for this group the Science email address had been incorporated in IdM. This will make it possible to use the Science address as an external mail address when resetting RU passwords. When the problem was recognized, it was fixed swiftly by restoring the RU password hashes from backup.

And again Print/phpMyAdmin server problem

  Begin         : 20140506 12:14
  End           : 20140506 12:35
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Again a reboot of the machine was necessary to restore the functionality. As a patch we will reboot this server each night.

Peage MFP Huygens building in maintenance April 28/29/30

  Begin         : 20140428 ca. 12:00
  End           : 20140430 ca. 12:00
  Affected      : Users of Peage in the Huygens building

April 28/29/30, the Peage MFP and the Peage POS near the restaurant in the Huygens building cannot be used. The ISC will then investigate whether Peage can also be made available for employees instead of only for students. April 28, the MFP will be moved to the test location, April 30 it will be moved back.

After replacement still Print/phpMyAdmin server problem

  Begin         : 20140417 13:35
  End           : 20140417 13:50
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Although the hardware of the server was replaced, the server didn't react for unknown reasons. A reboot of the machine solved the problem. We will think about ways to tackle this problem.

Print/phpMyAdmin server replacement Thursday April 17 08:30-09:20

  Begin         : 20140417 08:30
  End           : 20140417 09:20
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Because of the recent problems with this server, we are going to replace the server, in order to prevent further service interruptions.

U:/ home server pile problem

  Begin         : 20140416 17:42 (and earlier for some users)
  End           : 20140416 ca. 18:00
  Affected      : Users of this U:/ home server

The server had a problem with one partition, which had started during the creation of new snapshots. We waited with the reboot untill after working hours. A reboot of the machine solved the problem. To prevent these problems in the future, we will no longer make local snapshots of homeservers, but of course the daily backups of the homeservers by the backup server will be continued.

And yet again Print/phpMyAdmin server problem

  Begin         : 20140414 13:05
  End           : 20140414 13:12
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Just like four days ago, the server didn't react for unknown reasons. A reboot of the machine solved the problem. We plan to replace the server next Thursday 08:30-09:00 hours, in order to prevent further service interruptions.

Again Print/phpMyAdmin server problem

  Begin         : 20140410 12:54
  End           : 20140410 13:15
  Affected      : Users of the printto/printsmb/phpMyAdmin service

Just like two weeks ago, the server didn't react for unknown reasons. A reboot of the machine solved the problem. Together with the supplier we will try to find out what to replace to prevent this in the future.

U: / home server bundle problems

  Begin         : 20140404 ca. 11:35
  End           : 20140404 12:25
  Affected      : Users of this U: / home server

From the moment of yesterday's snapshots (13:00 hours), more and more processes were hung at the server. The first complaints arrived at C&CZ ca. 11:15 hours this morning. Therefore we decided ca. 11:35 to restart the server. The reboot resolved the problem. The number of snapshots will be reduced in order to try to prevent problems due to snapshots in the future.

Printer pr-hg-00-002 did not print from Windows/Mac

 Begin         : 20140402 11:32
 End           : 20140404 10:27
 Getroffen     : students of the Faculty of Science

Probably due to a too large printjob, the printer queue on the server (printto/printsmb/ooievaar) for the pr-hg-00-002 printer got stuck onWednesday morning. C&CZ was only informed of this problem at the end of Thursday. Friday morning this has been fixed by emptying the printqueue with old jobs.

Print/phpMyAdmin server problem

  Begin         : 20140327 10:40
  End           : 20140327 11:00
  Affected      : Users of the printto/printsmb/phpMyAdmin service

The server didn't react for unknown reasons. A reboot of the machine solved the problem.

Linux login server lilo3 reboot due to NFS-problems

  Begin         : 20140310 12:31
  End           : 20140310 13:53
  Affected      : Users of lilo3

The Linux login server lilo3 didn't have the homedirectories of some users. It took a reboot to fix this.

U:/ home server pile problem

  Begin         : 20140324 17:25
  End           : 20140324 17:35
  Affected      : Users of this U:/ home server

The server had a problem with one partition. A reboot of the machine solved the problem.

File server heap problems

  Begin         : 20140310 06:30
  End           : 20140310 10:10
  Affected      : Users of shared network disks

The fileserver failed to start after the Monday morning reboot, because of filesystem errors (LVmd2-7). Manually fixing the errors solved the problem and the machine started successfully.

U: / home server pile and Linux login server lilo/stitch problems

  Begin         : 20140305 ca. 13:00
  End           : 20140305 16:40
  Affected      : Users of this U: / home server or thee Linux loginservers

The server had a high load and stopped servicing users, probably because of problems with the creation of new snapshots at 13:00 hours. A reboot solved the problem.

Scan to email sometimes doesn't work for Konica Minolta MFPs

  Begin         : 2013???? ??:??
  End           : 20140226  
  Affected      : Users of a KM MFP C364e as a scanner through e-mail

Update: we changed the configuration of the network switchports of the KM's, which resolves the problem. This was necessary, because the KM's do not try hard enough to make a connection with the SMTP-server.

Update: an alternative is scanning to a USB stick. Log in at the MFP with the scan-pin, put a document in the feeder or on the document glass, select Scan and then plug the USB stick into the USB port on the right side of the MFP , on the top side. After a few seconds the MFP recognizes the USB stick and presents the choice "Save a document to external memory". Choose OK and press Start. See also [RU http://www.ru.nl/publish/pages/687597/uitrol-mf-scan.pdf manual] on the RU-page about the MFP's.

Users report to us that the scanning to e-mail sometimes doesn't work, with an error like "Server not found. Scan deleted". The problem has been reported to KM. As far as we know, the problem occurs only seldom. It can temporarily be resolved by switching the machine off and on again.

BASS Java 7 problem

  Begin        : 20131201
  End          : 20140210
  Affected     : Users of BASS with Oracle forms (Java)

The ISC let us know that after the installation of patches installed in BASS during the weekend of December 1, the possibility to use a recent version of Java (version 7) had erroneously disappeared. As of February 11, 2014 this issue has been resolved. The function in BASS to work with all versions of Java SE (JRE) on the client (PC) has been reactivated. Furthermore all JAR-files in BASS have received a security certificate (they have been signed), with which it possible for BASS to work with the latest Java SE versions (Java SE 7.51 en hoger). BASS users who work with Forms, will see pop-up windows asking "Do you want to run this application?". Of this a description has been made on the RU Intranet.

Network interruptions in Huygens wing 5

  Begin        : 20140203 19:00 hrs (7:00 pm)
  End          : 20140203 24:00 hrs (12:00 pm)
  Affected     : Users in Huygens wing 5

On Monday evening, February 3rd between 19:00 - 24:00 (7:00 - 12:00 pm) maintanance will be carried out on the network devices in Huygens Wing 5. Therefore the wired and wireless networks will not be available at some moments at all locations in this Wing, on all floors.

Again network interruptions in Huygens wing 2

  Begin        : 20140128 7:00 pm
  End          : 20140128 12:00 pm
  Affected     : Users in Huygens wing 2 and corridors between wings 2 and 4

The announced proceedings on Tuesday, January 21st on the network equipment unfortunately did not take place due to a sudden instability of the network in Wing 1 the same day. In order to prevent further instability issues in Wing 2, it was necessary to solve the problems in Wing 1 first. These problems have been identified and corrected. The network maintenance in Wing 2 will now take place Tuesday, January 28th between 19:00 and 24:00 ( 7:00 pm to 12:00 ) during which the wireless and cabled networks will be off line for some time. This will affect all floors in Wing 2 and rooms along the corridors between wings 2 and 4 (all floors).

Network interruptions in Huygens wing 2

  Begin        : 20140121 7:00 pm
  End          : 20140121 12:00 pm
  Affected     : Users in Huygens wing 2 and corridors between wings 2 and 4

On Tuesday evening January 21 between 7:00 pm and 12:00 pm maintanance will be carried out on the network devices in Huygens wing 2. Therefore the wired and wireless networks will not be available at some moments at all locations in wing 2 and corridors between wings 2 and 4, on all floors.

Network interruptions in Huygens wing 1

  Begin        : 20140113 7:00 pm
  End          : 20140113 12:00 pm
  Affected     : Users in Huygens wing 1 and corridors between wings 1 and 3

On Monday evening January 13 between 7:00 pm and 12:00 pm maintanance will be carried out on the network devices in Huygens wing 1. Therefore the wired and wireless networks will not be available at some moments in many locations in wing 1 and corridors between wings 1 and 3, on all floors.

Network problems

  Begin        : 20140106 11:29
  End          : 20140106 11:40

Due to an error by an ISC network administrator, all internal and external RU network traffic was blocked for a short period.

Moving network shares to new servers

  Begin         : 20131223
  End           : 20131224
  Affected      : Users of the following network shares:
                  acfiles2 botany botany-general carta comsol digicd encapson encapson2
                  exoarchief felix gi2 gi3 hfml-45t hfml-data hfml-engineering ifl iris
                  mailmanincludes mbaudit1 mbaudit2 mbbioel mbcns mbcortex mbdata mbread
                  mbwrite mestrelab mi2 mi3 milkun4 milkun4rw molchem molchem2 molphtec
                  mol-secr multimedia neuropi ns3 nwi-backup onlyme owc pcb planthgl
                  plantkunde-hgl sdisk/software share snn sofie spmdata1 tdisk/cursus tece teceleiding
                  temp tracegastemp wallpaper xpcursus xpsoftware

In the course of these two days, above network shares will be moved to new servers. During the move action, a share will not be available. If you encounter any problems connecting to a network share using Windows, always connect using the "share-name minus srv" naming scheme: \\sharenaam-srv.science.ru.nl\sharenaam. See also Diskruimte#Naming.

Postponed replacement of home-server "bundle"

  Begin        : 20131223 07:30
  End          : 20131223 09:00 (expected)
  Affected     : Users with homedirectory server "bundle" (as can be seen on http://DIY.science.ru.nl)

The replacement of the old home server "bundle" has been postponed and will now guaranteed take place on Monday morning December 23. Because the data have been synchronized with the new server, there will not be much downtime. The new server should be very dependable: hardware RAID-6, double processors and power supplies and a 5-year support contract from the supplier. The performance has improved, e.g. by using hardware RAID with a 1 GB write cache with battery backup.

Interruption of network in Huygens wing 1

  Begin        : 20131217 19:00
  End          : 20131217 24:00
  Affected     : Users of the network in Huygens wing 1, especially the ground floor.

Tuesday night December 17 19:00-24:00 hours, maintenance work will be done affecting the data network in Huygens wing 1. The wired and wireless network will suffer service interruptions a few times during that period. The main inconvenience will be on the ground floor.

Service interruption Konica Minolta MFP's outside of Huygens building

  Begin         : 20131209 12:48
  End           : 20131209 13:40 after reset of a KM MFP
  Affected      : Users of the KM MFP's outside of the Huygens building

C&CZ changed the DHCP-configuration of the KM MFP's, because KM had told us all KM's had an identical configuration after the upgrade of the night of December 4. Soon thereafter the KM's outside of the Huygens building showed problems. After changing the DHCP-configuration back to the previous version and restarting the MFP's, the problem was resolved. The firmware upgrade of these machines still has to take place.

Konica Minolta MFP's firmware upgrade

  Begin         : 20131204 02:00
  End           : 20131204 06:00
  Affected      : Nightly users of the KM MFP's

Wednesday night, the firmware of the Konica Minolta multifunctionals will be upgraded. This should resolve existing problems, like the not waking up from sleep mode. Please report all remaining problems, for MFP-hardware and paper to KM via phone: 55955 option 4.

Power dip December 3

  Begin         : 20131203  ca 09:36
  End           : 20131203  ca 10:00
  Affected      : all RU/UMCN users

Tuesday morning around half past nine, there was a short power dip for RU/UMCN, probably due to a switch error at Liander. This power dip, that is not listed on the power interruption website, made all systems restart that were not on UPS power. Because only the network switches in Huygens wing 1 and 7 are on UPS power, a lot of users lost their connection to the network, including wireless and IP-telephony, for about 20 minutes. Apparatus that restarted faster than the network, might have needed an extra restart to restore the connection to the network.

Poster printer "kamerbreed" broken

  Begin         : 20130925 17:00
  End           : 20131120 14:00
  Affected      : Users of the poster printer "kamerbreed"

The motherboard of the printer had to be replaced. Because there is no maintenance contract for this old printer and spare parts were hard to get, repair has taken a long time.

Gipphoenix network problem

  Begin         : 20131118 06:33
  End           : 20131029 09:16
  Affected      : Users of virtual machines hosted by gipphoenix

For unknown reasons, the network interface of the server was not online. Running ifdown / ifup eth2 resolved the problem.

Web lectures unavailable for 20 minutes

  Begin         : 20131115 10:35
  End           : 20131029 17:17
  Affected      : Users of Blackboard / Web lectures

For unknown reasons one of the servers had a high load and stopped servicing users. A reboot solved the problem.

U: / home server bundle problems

  Begin         : 20131029 ca. 16:00
  End           : 20131029 17:17
  Affected      : Users of this U: / home server

For unknown reasons the server had a high load and stopped servicing users. A reboot solved the problem.

Again IMAP mail server problem

  Begin         : 20131028 13:35 
  End           : 20131028 14:08
  Affected      : all users wanting to read Science mail 

Just like 3 days ago, the IMAP server got overloaded after noon. It took a reboot to bring the service back in operation. We suspect that our making of a backup-snapshot triggers this and now have disabled the snapshot during working hours.

IMAP mail server problem

  Begin         : 20131025 13:23 
  End           : 20131025 14:18
  Affected      : all users wanting to read Science mail 

During an extra backup the IMAP server got overloaded. It took a reboot to bring the service back in operation.

print_mail_to_link('Wireless_', 'ru.nl_authentication_problem')">Wireless @ru.nl authentication problem

  Begin         : 20131023  ca 14:30
  End           : 20131023  ca 16:15
  Affected      : all wireless @ru.nl users 

Yesterday afternoon around 14:30 the ISC conducted a seemingly innocent maintenance on the LDAP-server, but immediately after that auth-requests from Radius were no longer serviced. This made it impossible for wireless users to authenticate with their u/s/e number. Users in the realm @science.ru.nl were not affected by this.

Power dip October 22, ca. 11:00

  Begin         : 20131022  ca 10:55
  End           : 20131022  ca 11:00
  Affected      : all RU/UMCN users

Tuesday morning around 11 o'clock, there was a short power dip for RU/UMCN. This power dip, that is not listed on the power interruption website, made all systems restart that were not on emergency power. Because only the network switches in Huygens wing 1 and 7 are on emergency power, a lot of users lost their connection to the network, including wireless and IP-telephony, for about 5 minutes. Apparatus that restarted faster than the network, might have needed an extra restart to restore the connection to the network. A department reported that a departmental printer did not survive the power dip.

Again mail problems after supplying password to phishers

 Begin         : 20131010 00:04
 End           : 20131010 00:33
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Printto/printsmb/phpmyadmin server problems again

  Begin         : 20131001 15:18
  End           : 20131001 15:31
  Affected      : Users of phpmyadmin, printto and printsmb printer servers

The printserver crashed again, for yet unknown reasons. A reboot of the machine solved the problem.

U: / home server bundle problems

  Begin         : 20130930 13:46
  End           : 20130930 14:02
  Affected      : Users of this U: / home server

The server crashed. A reboot of the machine solved the problem.

Printto/printsmb/phpmyadmin server problems

  Begin         : 20130927 11:02
  End           : 20130927 11:10
  Affected      : Users of phpmyadmin, printto and printsmb printer servers

The printserver crashed. A reboot of the machine solved the problem.

Sound on Linux dual-boot PC's disabled

  Begin         : 20130923
  End           : 20170901
  Affected      : Users of sound in Linux on dual boot PC's

The availability of sound in Windows appeared to be depending on the state in which Linux had left it. Therefore the sound in Linux on dual boot PC's has been disabled, in order to have it available on Windows. Sound under Linux will be available again once the HP Compaq Elite 8300 AiO harware has been phased out.

BASS RU problem

  Begin         : 20130603     00:00
  End           : 20130926     08:00
  Affected      : all FNWI users using the Port Forwarder to connect to BASS RU.

The central BASS RU environment has been updated last weekend. Part of this upgrade was a change of the web address of the second logon screen, as announced on our BASS page. This morning it became clear that access to the second logon screen via the Port Forwarder didn't work anymore. Therefore the UCI rolled back the change of the second logon screen. This means that AFTER LOGGING ON TO THE PROXY at https://admin.ru.nl/ BASS is available again via https://bassruap01.uci.ru.nl:8010/OA_HTML/AppsLocalLogin.jsp

This problem will be finally resolved when the Port Forwarder will be stopped on October 1, 2013.

Printto/printsmb server problems

  Begin         : 20130924 11:41
  End           : 20130924 12:22
  Affected      : Users of the printto and printsmb printer servers

The printserver crashed. A reboot of the machine solved the problem.

File server heap problems

  Begin         : 20130917 16:25
  End           : 20130917 17:15
  Affected      : Users of 75 shared network disks

The fileserver failed to see its disks. A reboot of the machine solved the problem.

File server chunk failed reboot

  Begin         : 20130916 06:30
  End           : 20130916 13:05
  Affected      : Users of the following disks:
                arb botgarden ccs4 excienwi exoarchief FIT gi1 gipsy isis-dhz
                itt iwwr1 leonardo mercator micord molspec olcwis ons pvs
                ratio tzacad vb vscxray WiskundeToernooi zeegras

The fileserver failed the regular Monday morning reboot. Only after a rescue boot and a removal of all snapshots rebooting the machine worked.

Mail problems after supplying password to phishers

 Begin         : 20130906 10:13
 End           : 20130906 10:20
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 6 sep 2013 11:17 (CEST)

B/W KM printer pr-hg-00-002 printed in color

 Begin         : 20130902 13:33
 End           : 20130903 15:02
 Getroffen     : students of the Faculty of Science

The b/w tuned Konica Minolta printer/MFP pr-hg-00-002 suddenly printed in color. C&CZ refunded the student's budget and stopped the printer until KM had corrected the settings.

Peter van Campen 3 sep 2013 17:49 (CEST)

And again mail problems after supplying password to phishers

 Begin         : 20130904 12:22
 End           : 20130904 12:30
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 4 sep 2013 12:38 (CEST)

And yet again mail problems after supplying password to phishers

 Begin         : 20130903 13:12
 End           : 20130903 13:30
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 3 sep 2013 13:33 (CEST)

DNS nameserver problem

  Begin         : 20130902     04:30
  End           : 20130902     08:30
  Affected      : DNS clients

The DNS server on ns1.science.ru.nl didn't start after the reboot, due to a syntax error in one of the zone files. When this had been corrected, it started without problems.

Peter van Campen 2 sep 2013 18:27 (CEST)

Wireless access to BASS problem

  Begin         : 20130812     00:00
  End           : 20130823     ca. 15:00
  Affected      : all FNWI users using the wireless network to connect to bass.ru.nl.

From August 12 the wireless network in the Science buildings is being replaced. The new IP numbers appear to not be able to connect directly to BASS. We expect that this will be changed soon.

Peter van Campen 20 aug 2013 11:56 (CEST)

Yet again mail problems after supplying password to phishers

 Begin         : 20130829 05:30
 End           : 20130829 06:30
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 29 aug 2013 06:36 (CEST)

Fileserver chunk needs reboot

 Begin         : 20130829 07:30
 End           : 20130829 08:00 (expected)
 Affected      : Users of file services:
                 arb botgarden ccs4 excienwi exoarchief FIT gi1 gipsy isis-dhz
                 itt iwwr1 leonardo mercator micord molspec olcwis ons pvs
                 ratio tzacad vb vscxray WiskundeToernooi zeegras

Wim Janssen 28 aug 2013 25:37 (CEST)

Again mail problems after supplying password to phishers

 Begin         : 20130822 22:38
 End           : 20130822 23:18
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 22 aug 2013 23:27 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130822 01:45
 End           : 20130822 06:30
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 22 aug 2013 06:34 (CEST)

Windows Roaming Profile problem

 Begin         : 20130801 00:00
 End           : 20130807 12:00
 Affected      : Windows B-Fac Users with roaming profiles

An error in distributing a Group Policy Object (GPO) caused roaming profiles to fail for the last week. The start date/time of the problem is unknown, This is an estimate.

Wim Janssen 7 aug 2013 12:23 (CEST)

Lilo3 reboot for different IP number

 Begin         : 20130715 11:00
 End           : 20130715 11:15
 Affected      : Users of lilo (lilo3)

Today lilo3 appeared to have an incorrectly chosen IP number. A reboot fixed this.

Peter van Campen 15 jul 2013 12:56 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130701 23:50
 End           : 20130702 00:35
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 2 jul 2013 00:42 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130627 17:49
 End           : 20130627 18:19
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Wim Janssen 28 jun 2013 23:20 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130627 17:49
 End           : 20130627 18:19
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Peter van Campen 27 jun 2013 18:23 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130624 09:00
 End           : 20130624 10:05
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Wim Janssen 24 juni 2013 11:06 (CEST)

Mail problems after supplying password to phishers

 Begin         : 20130623 15:54
 End           : 20130623 16:30
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

DNS nameserver problem

  Begin         : 20130617     04:30
  End           : 20130617     08:45
  Affected      : DNS clients

The DNS server on ns1.science.ru.nl didn't start after the reboot, due to a syntax error in one of the zone files. When this had been coreected, it started without problems.

Peter van Campen 17 jun 2013 11:19 (CEST)

Mail problem (IMAP server)

  Begin         : 20130612     13:45
  End           : 20130612     14:27
  Affected      : all FNWI users wanting to read mail on the IMAP-server.

The IMAP server had a problem, which made a reboot necessary.

Peter van Campen 12 jun 2013 14:29 (CEST)

Maintenance Wireless@RU

  Begin         : 20130606     10:00 pm
  End           : 20130606     12:00 pm
  Affected      : all users using the wireless networks ru-wlan and eduroam.

On Thursday June 6th from 10:00 pm the wireless networks ru-wlan and eduroam will be unavailable for at least 2 hours. All existing connections will be cut off. The wireless network Science however will not be effected and will be kept available.

Marcel Kuppens 3 jun 2013 13:51 (CEST)

OpenID server problem

  Begin         : 20130524 ca. 03:30
  End           : 20130529     14:15
  Affected      : all FNWI users wanting to login on a wiki using OpenID.

During the renewal of the OpenID server a configuration error was made. When users reported problems logging in, we started searching for the origin of problem. When it was found, it could easily be corrected and the server could be restarted with the correct configuration.

Peter van Campen 29 mei 2013 16:49 (CEST)

Homeserver bundle crashed

  Begin         : 20130521 ca. 17:00
  End           : 20130521     17:27
  Affected      : all FNWI users with a homedirectory on fileserver bundle

The fileserver crashed. After a reboot everything was back to normal.

Peter van Campen 22 mei 2013 13:58 (CEST)

Fileserver stack crashed due to failed hard disk

  Begin         : 20130429 04:08
  End           : 20130501 09:15
  Affected      : Users of disk volumes/network shares on file server Stack.
  Problem       : Stack: Crash due to defective disk
  Solution      : Deactivated disk using hot spare and reboot

Wim Janssen 02 may 2013 10:00 (CET)

Homeserver bundle failed reboot

  Begin         : 20130429 06:30
  End           : 20130429 11:30
  Affected      : all FNWI users with a homedirectory on fileserver bundle

The fileserver failed to reboot during the regular Monday morning shutdown schedule. It was possible to gain access to the system console only after having removed all power from the chassis. Snapshots were removed using the rescue reboot but rebooting the machine resulted in a faulty filesystem. We were able to boot the system after all filesystems had been checked offline. These actions resulted in a unusual long downtime.

Erik Joost Visser 29 apr 2013 12:00 (CET)

Homeserver bundle failed reboot

  Begin         : 20130422 06:30
  End           : 20130422 09:50
  Affected      : all FNWI users with a homedirectory on fileserver bundle

The fileserver failed the regular Monday morning reboot. Only after a rescue boot and a manual removal of all snapshots rebooting the machine worked.

Disk server stack offline

  Begin        : 20130408 08:55
  End          : 20130408 09:30
  Affected     : Users of disk volumes/network shares on file server Stack.
  Problem      : Stack:  Crash due to failing disk
  Solution     : Deactivated disk using hot spare

Disk server pile offline

  Begin        : 20130408 06:30
  End          : 20130408 08:15
  Affected     : Users of disk volumes on file server Pile (userhomes).
  Problem      : Pile: Did not shutdown properly during weekly reboot due to a kernel panic which was 
  Solution     : Executed power-cycle of the system

Erik Joost Visser 8 apr 2013 9:30 (CET)

Mail problems after supplying password to phishers

 Begin         : 20130319 11:45
 End           : 20130319 12:14
 Affected      : Users of Science mail

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

File server miii no private network

  Begin        : 20130315 17:30
  End          : 20130318 09:00
  Affected     : Users of network disks of the server miii.
  Problem      : While performing maintenance (removing a network card) a network cable was not inserted correctly.

Wim Janssen 18 mrt 2013 11:11 (CET)

Disk server pile offline

  Begin        : 20130318 06:30
  End          : 20130318 07:36
  Affected     : Users of disk volumes on file server Pile.
  Problem      : Pile: Was waiting for interactive input after reporting a warning (^d)

Network problems due to installation of Matlab R2013a

Begin         : 20130313 13:00
End           : 20130313 13:40
Affected      : users of the network

Yesterday Matlab R2013a has been installed. Today at 13:00 hours many servers started to automatically copy this 5.4 GB to their local disc. Some parts of the network were overloaded by all these copying, which made accessing the network slow for many users. The distribution of this software will now be scheduled to happen over a longer period, primarily outside of working hours.

Peter van Campen 13 mrt 2013 18:31 (CET)

More IP-numbers for ru-wlan and Science (wireless)

Monday, March 4th 2013 at 18:00 hours, the number of IP numbers that is available in the FNWI buildings for ru-wlan and Science will be doubled. Because ru-wlan moves to a new range, users of ru-wlan will lose connectivity for at most 15 minutes. There was already a plan to replace ru-wlan and Science within the FNWI buildings by the RU-wide Eduroam and ru-wlan, but the wireless network usage has grown so fast that we can not wait for this plan to be realized. Last week some wireless users at times could not even get an IP address, although the lease time had been brought down to 30 minutes. Therefore this temporary measure became necessary on such short notice.

Marcel Kuppens 4 mar 2013 12:20 (CET)

Short interval in wireless network service

On Monday feb 18 at 6:00 pm there will be some maintenance at the wireless network which will effect the following locations at Toernooiveld:

FNWI cellar A1
FEL
Huygens: Library of Science, terrace behind Huygens, cantine, room HG-1.132
KDV1 en KDV2
Linnaeus building
Logistic Centre
Mercator I
Mercator II, ground floor and 7nd floor
Mercator III, 2nd floor
Transitorium FNWI (ACSW and FELIX)
UBC

We expect the service will be completely available again within 30 minutes.

Marcel Kuppens 18 feb 2013 10:53 (CET)

Mail problems after supplying password to phishers

 Begin         : 20130212
 End           : 20130214 (for now)
 Affected      : Users of Science mail, specifically of horde webmail

The last few days three Science users have supplied their Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam. This time they even used a fake copy of the horde Science webmail website. The big differences with the real horde Science webmail website are:

• the URL is not within science.ru.nl
• the connection is not a secure https connection, there is no lock
• the username and password do not arrive at C&CZ servers, but in the hands of Internet criminals.

PLEASE: do not naively click on a link in an e-mail!

New Radius server for ru-wlan and eduroam (wireless)

On Monday, January 28th 2013 at 8:00 am, one of the servers that is being used by the wireless network of the RU, will be replaced. This replacement will affect you as a user of the wireless networks ru-wlan and eduroam: There will appear a new certificate when connecting. You can just accept this, after which the connection should work. If this appears not to be the case, then it’s best that you remove your old Eduroam- respectively your old RU-WLAN settings first to activate the new connection .

Specifically for iPhone / iPad users: We recommend that you first remove your old Eduroam- respectively your old RU-WLAN profile before activating the new connection without a profile. If that unexpectedly fails, please review the information on www.ru.nl/wireless for iPhone/iPads. If necessary, you can also download a new profile from that site.

Marcel Kuppens 17 jan 2013 10:53 (CET)

Homeserver bundle crashed

  Begin         : 2013-01-16 ~ 13:30
  Einde         : 2013-01-16 ~ 14:00
  Affected      : all FNWI users with a homedirectory on fileserver bundle

Because the file server crashed, it had to be rebooted.

LDAP server vernieuwd

  Date         : 20121214
  Affected     : Users with a Fedora based desktop PC

Older Fedora desktop PC's may experience startup problems after an upgrade of one of our LDAP servers. A fix is available and has been applied. If you still encounter this problem, please contact C&CZ.

Mail problems after supplying password to phishers

  Begin        : 20121116 04:45
  End          : 20121117 ca 12:00
  Affected     : Users of horde webmail and users wanting to send mail to e.g. hotmail.com

Horde webmail again appeared to be misused for sending spam. This could happen because a naive user gave the Science password to phishers/spammers. After first stopping horde, early Friday morning we disabled the account of the naive user and restarted horde. Saturday morning it appeared that this short spam-outbreak had caused administrators of hotmail.com to add our mail server to their blacklist. Therefore we switched the IP-number of this mail server Saturday morning.

Homeserver bundle will be rebooted

  Begin         : 2012-10-24 ~ 12:45
  Einde         : 2012-10-24 ~ 13:00
  Affected      : all FNWI users with a homedirectory on fileserver bunlde

Because the file server refuses to accept a spare disk, it needs a reboot.

Homeserver bundle unavailable

  Begin         : 2012-10-22 12:15
  End           : 2012-10-22 13:00
  Affected      : all FNWI users with a homedirectory on fileserver bunlde

At the moment, we are solving the problem.

Services unavailable due to power and network outage

  Begin         : 20121018 03:00
  End           : 20121018 10:00
  Affected      : all users until 09:30; afterwards: "bundle" home directories, wireless, "plus" network shares and several websites

During the night of wednesday on thursday a power outage resulted in a network outage in the basement computing facilities. The power was restored to the network equipment using a bypass thus circumventing the UPS at about 09:15. Further checks implied that most servers had not become powerless so that most services became automatically available again. Network drivers on "bundle" had to be restarted in order to get access to home directories for a large number of users. Furthermore, several websites had to be restarted which made it possible for PC's to boot properly. During the day, an unrelated issue with the RAID storage of "plus" has been fixed as well granting access to the following network shares: sofie, ams*, molchem, mb*, encapson, milkun4, snn, neuropi, digicd. carta, ... Since wireless devices were unable to acquire IP addresses, i.e. gain access to the network, a split-brain situation was diagnosed within the DHCP service which was resolved around 13:00.

Announced downtime: home server "pile" down for reboot

  Begin        : 20121012 07:00
  End          : 20121012 09:00
  Affected     : Users with homedirectory server "pile" (as can be seen on http://DIY.science.ru.nl)

Next Friday morning, the home server "pile" will be rebooted. There are problems with the snapshots, which could make a reboot take more time. Therefore we schedule the reboot for early next Friday.

Peage top-up unit near Huygens restaurant in maintenance

In order to test new software, the Peage top-up unit near the Huygens restaurant was switched to maintenance mode. This unit is not used often yet, therefore this wil not have caused problems. Students that wanted to top-up their Peage account, could do that only elsewhere on campus. See the http://www.ru.nl/peage Peage website], locations are the halls of the Erasmus, Spinoza and Library buildings.

Eduroam incoming doesn't work for iPhone/iPad/iPod

  Begin         : spring 2012 (?)
  End           : 20121005
  Affected      : incoming Eduroam users with an iPhone/iPad/iPod

The UCI network management reports that at this moment the incoming version of Eduroam doesn't work for iPhone/iPad/iPod. A solution is being worked upon. Eduroam incoming means that one uses the wireless network of a remote institute, with authentication (login/password) being checked by RU or Science.

Horde webmail server down because of spam

  Begin        : 20120925 23:05
  End          : 20120926 10:20
  Affected     : Users of horde webmail

Yesterday evening, horde webmail appeared to be misused for sending spam. This could happen because a naive user gave the Science password to spammers. First we stopped horde. This morning we disabled the account of the naive user and restarted horde.

Disk server "Stack" offline

  Begin        : 20120924 06:30
  End          : 20120924 09:35
  Affected     : Users of disk volumes on file server Stack.

Disk server "Plenty" offline

  Begin        : 20120924 06:30
  End          : 20120924 09:00
  Affected     : Users of disk volumes on file server Plenty. The S and T disks that are used in the PC rooms.

During the weekly reboot (monday mornings), the server got stuck in the BIOS.

Announced downtime: home server "pile" down for replacement

  Begin        : 20120724 07:00
  End          : 20120724 09:00 (ca)
  Affected     : Users with homedirectory server "pile" (as can be seen on http://DIY.science.ru.nl)

Next Tuesday morning, the home server "pile" will be replaced by a new, more powerful server. Because the data have been synchronized with the new server, there will not be much downtime.

Postponed: Announced downtime: home server "pile" down for replacement

The downtime below has been postponed, because we had a few questions on the new server, that could not be answered in time. To be continued...

  Begin        : 20120724 07:00
  End          : 20120724 09:00 (ca)
  Affected     : Users with homedirectory server "pile" (as can be seen on http://DIY.science.ru.nl)

Next Tuesday morning, the home server "pile" will be replaced by a new, more powerful server. Because the data have been synchronized with the new server, there will not be much downtime. The new server should be very dependable: hardware RAID-6, double processors and power supplies and a 5-year support contract from the supplier. The performance has improved, e.g. by using hardware RAID with a 1 GB write cache with battery backup.

Partly announced downtime for mailman + horde webmail server

  Begin        : 20120712 09:09
  End          : 20120712 14:00 (ca)
  Affected     : Users of horde webmail and/or mailman mailing lists

This morning, horde webmail appeared to be misused for sending spam. This could happen because naive users gave their Science password to spammers. After we found out who the users were and had them change their password, we decide to also replace a defective cpu fan. Therefore also Mailman mailing lists will be down from 13:00 to 14:00 hours.

SMTP server blacklisted by MS Live Hotmail

  Begin        : 20120711 03:08
  End          : 20120711 14:55
  Affected     : Science mail users trying to send mail to MS-domains: hotmail.com, live.com, ...

This morning, users reported that mail from smtp.science.ru.nl to hotmail users was being bounced by hotmail. We have tried to let the hotmail administrators change this fast, but when this took too long, we changed the IP-number of our smtp-server.

Planned service interruption: file server with problems

  Begin        : 20120622 17:03
  End          : 20120624 19:30
  Affected     : stack fileservices

A hardware failure of a boot disk of the fileserver stack was reported Friday morning June 22. We decided to repair this after working hours. Thus at approximately 17:00 the defective boot disk was removed from the machine and replaced by a spare one. Enabling the disk, making it bootable, restoring file systems and rebooting the machine (after removing all snapshots) took a lot of time. When this was resolved Friday evening, the NFS/SMB fileservice was not active on the mounted filesystems. It took a reboot Sunday evening to resolve all problems.

Tracelab server poly defective

  Begin       : 20120621 14:12
  End         : 20120621 17:15
  Affected    : Tracelab for users. For administrators also Prism&Deploy and the WDS-service

A hardware failure of the server poly was reported at 2012-06-21 14:12. After a restart of the machine, it stopped working again. No more recoveries were attempted and an identical spare machine was outfitted with the disks from the defective server. Disks had to be synchronized before making the machine available again.

Servers without electric power

  Begin       : 20120607 13:45
  End         : 20120607 15:30
  Affected    : e-mail and users of the fileservers bundle, heap and stack

A power failure in a rack in a server room brought some C&CZ servers down. After less than two hours all problems were dealt with. Affected systems ware mainly: postvak (Science mail server), bundle (user homedisk), heap/stack (network discs), resser/kookpunt/brievenbus/rustug (mail transport smtp servers)

Planned Service: website-databases and maybe Linux clients

20 Apr 2012 17:00 - 17:15

A defective hard disc has been replaced in a server, but the server needs to be rebooted to ensure that this is reboot proof. The MySQL database of roughly 70 websites will therefore be down for a short time. Since this server also provides the Kerberos authentication for Linux clients, Linux clients might encounter service interruptions during a short period.

Windows server "plenty" with xpsoftware unavailable

Thursday July 7, around 13.00 hours the server "plenty" could not be reached. Because this server serves the "xpsoftware" share for the Managed Windows PC's, all these PC's had a problem. After the server was restarted and the disks had been checked, it was available again at 14:26.

Downtime Science servers: Sunday July 3, 09:00 - 12:00 hours

In order to improve the cooling of a server room, we plan to move three racks of Science servers a few meters on Sunday morning, July 3. We will have to switch off a lot of servers temporarily. Therefore several services will be unavailable some time starting July 3, 09:00 hours. We expect the downtime will last until 10:00 hours for servers with a lot of different users. The cn compute cluster will probably be fully operational again at 12:00 hours.

The servers/services affected are:

fileservers: plenty/pile/bundle with shares like:
             amsbackup2 bbb-priv botany bsweet comsol exoarchief gi3 hfml-data ifl iris
             lambiek mestrelab mi1/2/3 molchem2 molphtec morph multimedia olsen pcb planthgl
             sdisk share snn2 spmdata1 tdisk tece temp wallpaper xpcursus xpsoftware
potkast: films via Blackboard
ts2: Windows Terminal Server
lilo1: Linux Login Server, alternative: lilo/lilo2
cn compute cluster
horde webmail
License server for: Comsol

With apologies for the inconvenience
C&CZ

Peter van Campen 22 jun 2011 09:57 (UTC)

Network outage June 22, 10:55-11:30

This morning, in the network hub for Huygens South a UPS (battery power supply) went down, which made a set of network switches loose power. Because of this, users in Huygens wing 1 and spin-off companies lost their connection to the network. After bypassing the UPS, everything was up and running again at 11:30. We are still searching for the exact origin of this outage.

New SSH keys for new login servers

The LInux LOgin server lilo has been replaced. The name now points to the new machine lilo2, because that one is faster than the other login server lilo1. Therefore it is quite normal to accept once the new SSH-key.

Planned Service: Limited computer services

12 Feb 2011 7:00 - 11:00

A backup cooling system will be installed in our main computer room. Therefore the air conditioning system must be switched off, which means that most of the computer facilities in this room must be shut down. This includes the cluster nodes cn00 through cn53 and many of the web- and file- (network share) servers. It is advised to expect a very limited service level. We will try to keep all home directories and the mail system available. For detailed information about the impact please contact C&CZ.

Printer lp5

24 Jan 2011 - 11 Mar 2011

Printer lp5 has been moved to HG00.089. You can't use this printer at the moment, there's a problem with the power supply unit.

Fixed phone problem

7 Mrt 2011

You can't reach certain fixed phones at the university right now, mobile phones and Skype do work ok though.

Mailserver blacklisted

4 Feb 2011 9:00 - 12:00

One of our mail servers has been sending loads of spam after a successful phishing attack. Since then, our server has been blacklisted on several domains. Currently this affects the delivery of email to @hotmail and @live addresses.

---

Be quickly informed via the CPK mailinglist or the RSS feed