Selecting an online exam platform

If you are looking around for an online exam platform, you have certainly discovered there are many out there. How do you choose? Well, for starters, it is important to understand that the range of quality and sophistication is incredible. This post will help you identify some salient features and benefits that you should consider in first establishing your requirements and then selecting online exam software that best meets the needs of your organization.

Types of online exam platforms

There are many, many software systems in existence that can perform some sort of online assessment delivery. From a high level, they differ substantially in terms of sophistication.

There are, of course, many survey engines that are designed for just that: unproctored, unscored surveys. No stakes involved, no quality required.
At a slightly higher level are platforms for simple or formative assessment. For example, a learning management system will typically include a component to deliver multiple choice questions. However, this is obviously not their strength – you should use an LMS for assessment no more than you should use an assessment platform as an LMS. There are also a lot of basic assessment platforms for multiple choice questions and the like.
At the top end of the spectrum are exam platforms that are designed for real assessment. That is, they implement best practices in the field, like Angoff studies, item response theory, and adaptive testing. The type of assessment effort being done by school teachers is quite different from that being done by a company producing high-stakes international tests. FastTest is an example of such a platform.

This post describes some of the aspects that separate the third level from the lower two levels. If you need these aspects, then you likely need a “Level 3” system. Of course, there are many testing situations where such a high quality system is complete overkill.

Another consideration is that many online exam platforms are closed-content. That is, they are 100% proprietary and used only within the organization that built it. You likely need an open-content system, which allows anyone to build and deliver tests.

Test development aspects

Reusable items: If you write an item for this semester’s test, you should be able to easily reuse it next semester. Or let another instructor use it. Surprisingly, many systems do not have this basic functionality. This is a core aspect of item banking.

Item metadata: All items should have extensive metadata fields, such as author, content area, depth of knowledge, etc. It should also store item statistics or IRT parameter, and more importantly, actually use them. For example, test form assembly should utilize item statistics to evaluate form difficulty and reliability.

Form assembly: The system needs advanced functionality for form assembly, including extensive search functionality, automation, support for parallel forms, etc.

Publication options: The system needs to support all the publication situations that your organization might use: paper vs. online, implementation of time limits, control of widgets like calculators, etc.

Item types: The field of assessment is moving excitedly towards technology enhanced items (TEIs) in an effort to assess more deeply and authentically. While this brings challenges, online testing platforms obviously need to support these.

Best practices in online exams

Supports standards: The workflow and reports of the system should facilitate the following of industry standards like APA/AERA/NCME, NCCA, ANSI, and ITC.

Psychometrics: Psychometrics is the Science of Assessment. The system needs to implement real psychometrics like item response theory, computerized adaptive testing, distractor analysis, and test fraud analysis. Note: just looking at P values and point-biserials is extremely outdated.

Logs: The online testing platform should log user and examinee activity, and have it available for queries and reports.

Reporting: You need reports on various aspects of the system, including item banks, users, tests, examinees, and domains.

Security: Be able to limit availability windows, require a lockdown browser, enable remote proctoring, and so many other options.

Essay scoring: If you deliver open-response items, the platform should have a module for scoring them.

System aspects

Security: Security is obviously essential to high stakes testing organizations. There are many aspects though: user roles, content access control, browser lockdown, options for virtual proctoring, examinee access, proctor management, and more.

Reliability: The online exam software needs to have very little downtime. Does the provider have a system status page? Best-in-class hosting like Amazon Web Services?

Scalability: The online exam system needs to be able to scale up to large volumes.

Configurability: The functionality throughout the system needs to be configurable to meet the needs of your organization and individual users.

Request a demo with ASC’s online exam platform

Bio
Latest Posts

Nathan Thompson, PhD

Nathan Thompson earned his PhD in Psychometrics from the University of Minnesota, with a focus on computerized adaptive testing. His undergraduate degree was from Luther College with a triple major of Mathematics, Psychology, and Latin. He is primarily interested in the use of AI and software automation to augment and replace the work done by psychometricians, which has provided extensive experience in software design and programming. Dr. Thompson has published over 100 journal articles and conference presentations, but his favorite remains https://pareonline.net/getvn.asp?v=16&n=1.

Latest posts by Nathan Thompson, PhD (see all)

Content Validity in Assessment - November 3, 2023
Predictive Validity of Test Scores - October 3, 2023
What is Psychometrics? Definition: Improve exams with science. - July 5, 2023

Selecting an Online Exam Platform