Entries by Nathan Thompson, PhD

Content Validity in Assessment

Content validity is an aspect of validity, a term that psychometricians use to refer to evidence that interpretations of test scores are supported.  For example, predictive validity provides evidence that a pre-employment test will predict job performance, tenure, and other important criteria.  Content validity, on the other hand, focuses on evidence that the content of […]

Predictive Validity of Test Scores

Predictive Validity is a type of test score validity which evaluates how well a test predicts something in the future, usually with a goal of making more effective decisions about people.  For instance, it is often used in the world of pre-employment testing, where we want a test to predict things like job performance or […]

,

What is Psychometrics? Definition: Improve exams with science.

Psychometrics is the science of educational and psychological assessment.  It scientifically studies how tests are developed, delivered, and scored, regardless of the test topic.    The goal is to provide validity: evidence to support that interpretations of scores from the test are trustworthy. Psychometricians tackle fundamental questions around assessment, such as how to determine if a […]

Certification Management System: Streamline Credential Management

A Certification Management System (CMS) or Credential Management System (CMS) plays a pivotal role in streamlining the key processes surrounding the certification or credentialing of people, namely that they have certain knowledge or skills in a profession.  It helps with ensuring compliance, reducing business operation costs, and maximizing the value of certifications. In this article, […]

ANSI/ISO 17024 Accreditation

ANSI/ISO 17024 accreditation is an internationally recognized standard for the accreditation of personnel certification bodies. ANSI stands for the American National Standards Institute, while ISO refers to the International Organization for Standardization. The portion of ANSI which carries out the accreditation process is the ANSI National Accreditation Board (ANAB). What does ANSI/ISO 17024 cover? ANSI […]

Microcredentials: What are they?

Microcredentials are short, focused, and targeted educational or assessment-based certificate programs that offer learners a way to acquire specific skills or knowledge in a particular field.  In today’s fast-paced and rapidly evolving job market, traditional degrees may not always be enough to stand out among the competition, and they often take too long to achieve. […]

,

What is automated essay scoring?

Automated essay scoring (AES) is an important application of machine learning and artificial intelligence to the field of psychometrics and assessment.  In fact, it’s been around far longer than “machine learning” and “artificial intelligence” have been buzzwords in the general public!  The field of psychometrics has been doing such groundbreaking work for decades. So how […]

,

Coefficient alpha reliability index

Coefficient alpha reliability, sometimes called Cronbach’s alpha, is a statistical index that is used to evaluate the internal consistency or reliability of an assessment. That is, it quantifies how consistent we can expect scores to be, by analyzing the item statistics. A high value indicates that the test is of high reliability, and a low […]

Differential item functioning (DIF)

Differential item functioning (DIF) is a term in psychometrics for the statistical analysis of assessment data to determine if items are performing in a biased manner against some group of examinees. Most often, this is based on a demographic variable such as gender, ethnicity, or first language. For example, you might analyze a test to […]

, ,

Multistage Testing

Multistage testing (MST) is a type of computerized adaptive testing (CAT).  This means it is an exam delivered on computers which dynamically personalize it for each examinee or student.  Typically, this is done with respect to the difficulty of the questions, by making the exam easier for lower-ability students and harder for high-ability students.  Doing […]