Difference between revisions of "Digitaal toetsen"

From Cncz
Jump to: navigation, search
m ([Voor- en nadelen][Advantages and disadvantages])
m ([Voor- en nadelen][Advantages and disadvantages])
 
(43 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= [Digitale toetsomgeving][Digital testing environment] =
+
== [RU digitaal toetsen en Chromebooks][RU digital assessment and Chromebooks] ==
 +
[nl]
 +
Om digitaal toetsen op grote schaal aan te kunnen bieden, heeft de RU ca. 1000 [https://en.wikipedia.org/wiki/Chromebook Chromebooks] aangeschaft. Ook is gekozen voor de gebruikersvriendelijke toetssoftware [http://www.cirrusassessment.com/nl/ Cirrus Assessment]. In de toekomst zal de facultaire digitale toetsomgeving (TAO, zie onder) alleen nodig zijn voor toetsen die Cirrus niet aankan. Ook is het draadloze netwerk van het Gymnasion/RSC geschikt gemaakt om dit grote aantal Chromebooks van een goed draadloos netwerk te voorzien. Meer informatie hierover is te vinden op de [https://www.ru.nl/docenten/virtuele-map/onderwijs/toetsen-en-beoordelen/digitaal-toetsen/ RU-pagina over digitaal toetsen].
 +
 
 +
Op Chromebooks kunnen alleen webgebaseerde applicaties worden aangeboden, in een browser. In juni 2018 is succesvol een toets afgenomen bij meer dan 300 studenten Informatica, met gebruik van een Windows-applicatie (een moderne programmeeromgeving) die via [https://aws.amazon.com/appstream2/ Amazon Appstream 2.0] in een apart tabblad van de Safe Exam Browser op deze Chromebooks werkte.
 +
[/nl]
 +
[en]
 +
In order to facilitate large scale digital assessment, the university purchased ca. 1000 [https://en.wikipedia.org/wiki/Chromebook Chromebooks]. For these, [http://www.cirrusassessment.com/ Cirrus Assessment] was chosen as user friendly assessment software. In the future, the faculty digital assessment software (TAO, see below) will only be necessary for assessments that Cirrus can't handle. Also the wireless network in the Gymnasion/RSC was upgraded to give this large number of Chromebooks a good wireless network connection. More information on this cam be found on the [https://www.ru.nl/lecturers/education/assessment-appraisal/digital-assessment/ RU page on digital assessments].
 +
 
 +
Chromebooks only run webbased applications, in a browser. In June 2018, more than 300 students Computing Science were successfully graded in a large scale digital assessment with these Chromebooks using a Windows application (a modern computer program development environment) that ran in a tab of the Safe Exam Browser and was delivered through [https://aws.amazon.com/appstream2/ Amazon Appstream 2.0].
 +
[/en]
 +
 
 +
== [Facultaire digitale toetsomgeving: TAO][Faculty digital testing environment: TAO] ==
  
 
[nl]
 
[nl]
 +
Er is een digitale toetsomgeving beschikbaar, gebaseerd op het open source product [http://www.taotesting.com/ TAO Testing], die voor zowel summatief als formatief toetsen gebruikt kan worden. Hoewel TAO een groot aantal vraagtypes kent, wordt slechts een selectie hiervan ondersteund bij de operationele omgeving van FNWI:
 +
* [https://userguide.taotesting.com/3.2/interactions/extended-text-interaction.html extendedTextInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/choice-interaction.html choiceInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/hotspot-interaction.html hotspotInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/associate-interaction.html associateInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/match-interaction.html matchInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/order-interaction.html orderInteraction]
 +
 +
De TAO-omgeving bevat zelf b.v. geen mogelijkheid om wiskundige of chemische formules in te voeren in de antwoorden. Zulke formules kunnen op redelijke schaal wel gebruikt worden in de toetsvragen. Het is wel mogelijk om in TAO computer broncode in te voeren maar alleen in een vrij tekstveld, niet in een vorm die in de buurt komt van de programmeeromgevingen die we tegenwoordig gewend zijn. In dit soort gevallen kan er een programmeeromgeving, naast de browser waarin TAO wordt gebruikt, worden gedraaid. Er zijn toetsen afgenomen waarbij naast de toetssoftware ook Windows Paint, Microsoft Word, [https://clean.cs.ru.nl/Clean Clean] en [http://www.virtualnmr.org Virtual NMR] zijn gebruikt.
 
[/nl]
 
[/nl]
 +
 
[en]
 
[en]
We offer a digital testing environment, based on the open source product TAO Testing, which can be used for both summative and formative testing. The testing environment supports several types of questions, but there is no support for entering mathematical or chemical formulae in answers, although formulae can be used in questions. Entering computer source code is possible in a free text field, but not in a manner which resembles the computer program development environments people are used to nowadays. So, in general, questions which require free text or multiple choice answers are currently supported. Images can be used in questions and it is possible to let the student make and upload a drawing (using a basic tool available on the desktop such as Windows Paint) as (part of) their answer. This feature was used in a real life digital test and functioned properly, although it is not ideal. All in all this means that at the moment the environment can only be used for a limited set of our courses. Additional features must be implemented for the environment to become a general purpose digital testing facility for the Faculty of Science as a whole.  
+
We offer a digital testing/assessment environment, based on the open source product [http://www.taotesting.com/ TAO Testing], that can be used for both summative and formative testing. Although TAO supports many question types, only a limited selection is supported in the operational setup at the Faculty of Science:
 +
* [https://userguide.taotesting.com/3.2/interactions/extended-text-interaction.html extendedTextInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/choice-interaction.html choiceInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/hotspot-interaction.html hotspotInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/associate-interaction.html associateInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/match-interaction.html matchInteraction]
 +
* [https://userguide.taotesting.com/3.2/interactions/order-interaction.html orderInteraction]
 +
 
 +
In TAO itself, there is no support for entering mathematical or chemical formulae in answers, although formulae can be used in questions. Entering computer source code is possible in a free text field, but not in a manner which resembles the computer program development environments people are used to nowadays. In these cases, a programming environment can used, next to the browser with TAO. Digital tests have been done with Windows Paint, Microsoft Word, [https://clean.cs.ru.nl/Clean Clean] and [http://www.virtualnmr.org Virtual NMR]) next to the test software TAO.
 +
[/en]
 +
 
 +
== [Onderdelen][Parts] ==
 +
 
 +
[nl]
 +
De facultaire toetsomgeving bestaat uit vijf delen, allemaal web-gebaseerd:
 +
# Een "database" omgeving waarin docenten vragen kunnen samenstellen en hun verzameling toetsvragen kunnen beheren.
 +
# Een "authoring" omgeving waarin docenten tentamens kunnen samenstellen met de vragen uit de database.
 +
# Een "testing" (toets)omgeving waarin tentamens worden afgenomen. The toetsen vinden plaats op de PC's in de [[Terminalkamers|pc onderwijsruimtes]] die door C&CZ worden beheerd, met [https://www.safeexambrowser.org/ Safe Exam Browser].
 +
# Een "grading" omgeving waarin de antwoorden kunnen worden beoordeeld.
 +
# Een "review" omgeving waarmee deelnemers inzage in de gemaakte toets kunnen krijgen.
  
The environment consists of four parts, all of which are web-based.
+
Al deze omgevingen bestaan en zijn operationeel.
 +
[/nl]
 +
[en]
 +
The environment consists of five parts, all are web-based:
 
# A “database” environment in which lecturers can create and manage their own collection of exam questions and answers.
 
# A “database” environment in which lecturers can create and manage their own collection of exam questions and answers.
 
# An “authoring” environment in which lecturers can create exams from questions from the database.
 
# An “authoring” environment in which lecturers can create exams from questions from the database.
# A “testing” environment in which tests can be performed. The tests take place on the computer lab PCs managed by C&CZ in a so called Secure Exam Browser environment.
+
# A “testing” (exam) environment in which tests can be performed. The tests take place on the [[Terminalkamers|computer lab]] PCs managed by C&CZ with [https://www.safeexambrowser.org/ Safe Exam Browser].
 
# A “grading” environment in which the answers that were handed in can be reviewed and graded.
 
# A “grading” environment in which the answers that were handed in can be reviewed and graded.
 +
# A "review" environment in which participants can view their graded test.
  
At this moment the grading environment does not exist yet. Instead the results of the test are exported from the testing environment into a spreadsheet for the lecturer to review and grade the answers. For multiple choice type questions automatic grading is already supported. The other three parts of the environment do exist and are operational.
+
At this moment, all parts of the environment exist and are operational.
 
[/en]
 
[/en]
  
= [Voor- en nadelen][Advantages and disadvantages] =
+
== [Voor- en nadelen van digitaal toetsen][Advantages and disadvantages of digital assessments] ==
  
 
[nl]
 
[nl]
De nieuwe digitale toetsomgeving is al met succes gebruikt voor een tiental tentamens.  De voordelen zijn groot.
+
De voordelen zijn groot.
 
# Correctie en beoordeling, vooral voor een groot aantal deelnemers, kan sneller worden uitgevoerd.
 
# Correctie en beoordeling, vooral voor een groot aantal deelnemers, kan sneller worden uitgevoerd.
## Antwoorden op open vragen worden ingetypt i.p.v. geschreven en daardoor veel beter leesbaar.
+
## Antwoorden op open vragen worden ingetypt i.p.v. geschreven en zijn daardoor veel beter leesbaar.
 
## Multiple choice vragen kunnen automatisch worden beoordeeld.
 
## Multiple choice vragen kunnen automatisch worden beoordeeld.
# Naar mate de database met toetsvragen groeit, wordt het eenvoudiger voor docenten om toetsen samen te stellen.
+
# Naarmate de database met toetsvragen groeit, wordt het eenvoudiger voor docenten om toetsen samen te stellen.
 
# Voor de meeste studenten is typen en klikken handiger dan schrijven.
 
# Voor de meeste studenten is typen en klikken handiger dan schrijven.
# Voor Informatica cursussen, vooral op het gebied van programmeren, is de mogelijkheid om code te typen en verbeteren een enorme verbetering t.o.v. het schrijven van code op papier.
+
# Voor Informatica-cursussen, vooral op het gebied van programmeren, is de mogelijkheid om code te typen en verbeteren een enorme verbetering t.o.v. het schrijven van code op papier.
# De cursus-enquete (Alice) is geintegreerd na afloop van de digitale toets, waardoor de response op de enquete enorm is gestegen.
+
# De cursus-enquete (Alice) is geïntegreerd aan het einde van de digitale toets, waardoor de response op de enquête enorm is gestegen.
 
[/nl]
 
[/nl]
 
[en]
 
[en]
Our digital testing environment was used successfully for about a dozen exams.  The advantages of digital testing are big.
+
The advantages of digital testing are many.
 
# Correcting/grading, especially for large numbers of participants, can be done much faster.
 
# Correcting/grading, especially for large numbers of participants, can be done much faster.
 
## Answers to open questions are typed in, solving the poor handwriting problem.
 
## Answers to open questions are typed in, solving the poor handwriting problem.
 
## Automatic grading of multiple-choice questions is possible.
 
## Automatic grading of multiple-choice questions is possible.
# As the database of exam questions grows it becomes easier for lecturers to create tests.
+
# As the database of exam questions grows, it becomes easier for lecturers to create tests.
 
# For most students typing and/or clicking is much easier than writing.
 
# For most students typing and/or clicking is much easier than writing.
 
# For courses in Computing Science, especially computer programming, the possibility to type and modify code instead of writing code on paper is a major improvement.
 
# For courses in Computing Science, especially computer programming, the possibility to type and modify code instead of writing code on paper is a major improvement.
# Functionality like what is offered by Versatest (digital microscopy) is only possible in a digital environment (but it has not yet been integrated in TAO Testing).
+
# Functionality as offered by Versatest (digital microscopy) is only possible in a digital environment (but it has not yet been integrated in TAO Testing).
 
# We have integrated the course survey (Alice) at the end of the digital test which has led to a vast increase of the survey response percentage.  
 
# We have integrated the course survey (Alice) at the end of the digital test which has led to a vast increase of the survey response percentage.  
 
[/en]
 
[/en]
Line 41: Line 86:
 
[nl]
 
[nl]
 
Er zijn ook nadelen:
 
Er zijn ook nadelen:
# Digitaal toetsen introduceert nieuwe fraude-risico's. Het kan lastig zijn om deze allemaal ui te sluiten.
+
# Digitaal toetsen introduceert nieuwe fraude-risico's. Het kan lastig zijn om deze allemaal uit te sluiten.
# Digitaal toetsen legt beslag op de beschikbare PC's voor studenten.  Dit probleem neemt toe naarmate het digitaal toetsen toeneemt.
+
# Digitaal toetsen legt beslag op de beschikbare PC's voor studenten.  Dit probleem neemt toe naarmate het gebruik van digitaal toetsen toeneemt. Zie [https://wiki.science.ru.nl/cncz/Digitaal_toetsen#RU_digitaal_toetsen_en_Chromebooks Chromebooks].
# Mede in verband met het vorige punt kan het nodig zijn om digitale toetsen in de avonduren te roosteren. Dit is momenteel een probleem omdat IT ondersteuning buiten kantooruren niet geregeld is.
 
 
# De PC's in de computerruimtes zijn niet beschermd tegen stroomuitval. Bij een langdurige uitval kan een digitaal tentamen niet doorgaan.
 
# De PC's in de computerruimtes zijn niet beschermd tegen stroomuitval. Bij een langdurige uitval kan een digitaal tentamen niet doorgaan.
# Op dit moment is het niet mogelijk om te werken met wiskundige of chemische formules, reactievergelijkingen, diagrammen, of welke andere vorm van input anders dan gewone tekst.
+
# Op dit moment is het niet eenvoudig mogelijk om te werken met wiskundige of chemische formules, reactievergelijkingen, diagrammen, of welke andere vorm van input anders dan gewone tekst.
 
[/nl]
 
[/nl]
 
[en]
 
[en]
 
There are also disadvantages:
 
There are also disadvantages:
 
# Digital testing introduces new cheating risks. It may be difficult to eliminate all of them.
 
# Digital testing introduces new cheating risks. It may be difficult to eliminate all of them.
# Digital testing places a burden on the available PCs for students. This problem increases as the use of digital testing grows.  
+
# Digital testing places a burden on the available PCs for students. This problem increases as the use of digital testing grows. See [https://wiki.science.ru.nl/cncz/Digitaal_toetsen#RU_digitaal_toetsen_en_Chromebooks Chromebooks].
# It may be necessary to schedule digital tests in the evening due to the previous issue. Currently this is not possible because IT support cannot be guaranteed outside office hours.
 
 
# The PCs in the computer labs are not shielded from a major power outage. If this happens the exam can only be cancelled.
 
# The PCs in the computer labs are not shielded from a major power outage. If this happens the exam can only be cancelled.
# The current solution can not be used for exam questions where the answer requires the use of mathematical or chemical formulae, reaction equations, diagrams, in general any substantial amount of free format “drawing” by the student.
+
# The current solution can not easily be used for exam questions where the answer requires the use of mathematical or chemical formulae, reaction equations, diagrams, in general any substantial amount of free format “drawing” by the student.
 
[/en]
 
[/en]
  
 +
[[Bestand:lijn.png]]
 
[[Category:Studenten]]
 
[[Category:Studenten]]
 
[[Category:Docenten]]
 
[[Category:Docenten]]
 
[[Category:Onderwijs]]
 
[[Category:Onderwijs]]

Latest revision as of 17:56, 16 August 2019

RU digital assessment and Chromebooks

In order to facilitate large scale digital assessment, the university purchased ca. 1000 Chromebooks. For these, Cirrus Assessment was chosen as user friendly assessment software. In the future, the faculty digital assessment software (TAO, see below) will only be necessary for assessments that Cirrus can't handle. Also the wireless network in the Gymnasion/RSC was upgraded to give this large number of Chromebooks a good wireless network connection. More information on this cam be found on the RU page on digital assessments.

Chromebooks only run webbased applications, in a browser. In June 2018, more than 300 students Computing Science were successfully graded in a large scale digital assessment with these Chromebooks using a Windows application (a modern computer program development environment) that ran in a tab of the Safe Exam Browser and was delivered through Amazon Appstream 2.0.

Faculty digital testing environment: TAO

We offer a digital testing/assessment environment, based on the open source product TAO Testing, that can be used for both summative and formative testing. Although TAO supports many question types, only a limited selection is supported in the operational setup at the Faculty of Science:

In TAO itself, there is no support for entering mathematical or chemical formulae in answers, although formulae can be used in questions. Entering computer source code is possible in a free text field, but not in a manner which resembles the computer program development environments people are used to nowadays. In these cases, a programming environment can used, next to the browser with TAO. Digital tests have been done with Windows Paint, Microsoft Word, Clean and Virtual NMR) next to the test software TAO.

Parts

The environment consists of five parts, all are web-based:

  1. A “database” environment in which lecturers can create and manage their own collection of exam questions and answers.
  2. An “authoring” environment in which lecturers can create exams from questions from the database.
  3. A “testing” (exam) environment in which tests can be performed. The tests take place on the computer lab PCs managed by C&CZ with Safe Exam Browser.
  4. A “grading” environment in which the answers that were handed in can be reviewed and graded.
  5. A "review" environment in which participants can view their graded test.

At this moment, all parts of the environment exist and are operational.

Advantages and disadvantages of digital assessments

The advantages of digital testing are many.

  1. Correcting/grading, especially for large numbers of participants, can be done much faster.
    1. Answers to open questions are typed in, solving the poor handwriting problem.
    2. Automatic grading of multiple-choice questions is possible.
  2. As the database of exam questions grows, it becomes easier for lecturers to create tests.
  3. For most students typing and/or clicking is much easier than writing.
  4. For courses in Computing Science, especially computer programming, the possibility to type and modify code instead of writing code on paper is a major improvement.
  5. Functionality as offered by Versatest (digital microscopy) is only possible in a digital environment (but it has not yet been integrated in TAO Testing).
  6. We have integrated the course survey (Alice) at the end of the digital test which has led to a vast increase of the survey response percentage.

There are also disadvantages:

  1. Digital testing introduces new cheating risks. It may be difficult to eliminate all of them.
  2. Digital testing places a burden on the available PCs for students. This problem increases as the use of digital testing grows. See Chromebooks.
  3. The PCs in the computer labs are not shielded from a major power outage. If this happens the exam can only be cancelled.
  4. The current solution can not easily be used for exam questions where the answer requires the use of mathematical or chemical formulae, reaction equations, diagrams, in general any substantial amount of free format “drawing” by the student.

Lijn.png