CLEAR Exam Review Summer 2020

CLEAR Exam Review Summer 2020, Volume 30, No. 1

• Abstracts and Updates, by George Gray- George Gray begins Abstracts and Updates with a brief description of recent additions to the National Council on Measurement in Education’s instructional modules (ITEMS) and a link to access these training opportunities focused on measurement concepts and techniques. Next, he summarizes an article of general interest related to understanding what is meant by “stakes” in testing and another on validity evidence for licensing exams. The remainder of the Abstracts column covers recent articles describing specific exams and programs in licensure and certification. While most of these represent health care fields, a variety of topics are addressed that generalize to many regulated professions—including test development, validation, purposes of certification, justification for/against certification, and costs and benefits of certification.
• Legal Beat, by Dale Atkinson- As society reacts to the COVID-19 pandemic, there has been increased scrutiny of the government’s role in regulating professions. Adjustments to licensing requirements, such as the suspension of educational and/or examination requirements to allow for provisional licenses, may be needed to ensure sufficient numbers of essential workers. In Legal Beat, Dale Atkinson reminds us why government involvement in regulating professions is important and why regulation should be done well. The column addresses the need for legislators and other decision makers to understand the purpose of licensure examinations, the distinction between educational requirements and the demonstration of competence, and the role of credentialing examinations in protecting the public. Legislators, especially, are cautioned against too much specificity in regulations.
• Recent CLEAR Quick Poll Results, by Carla Caro- Carla Caro summarizes the results of Quick Poll surveys conducted by CLEAR over the past year. In an open-ended query, regulators were asked to identify questions they commonly receive; their responses are grouped by general topic area. A separate Quick Poll addressed the use and utility of digital badges in credentialing programs. Another requested information on low-volume exams and measures taken to evaluate reliability and enhance defensibly when the candidate population is small. Finally, testing programs were asked about the use of open-book exams and practices related to open-book testing. Responses to each Quick Poll are reported by groups, where applicable. Results of these polls provide food for thought and may identify areas where the development of pre-prepared responses and additional research would be beneficial.
• Leveraging Scientific Advances to Inform and Enhance Item Development, by Matthew J. Burke, Naomi E. B. Pitkin, Steven J. Durning, and Jason Carr- Automated item generation is the focus of our first feature article, as Matthew Burke and his colleagues present a model-based approach to the creation of test content. This methodology is designed not only to produce larger quantities of high-quality test items, but also to provide strong documentation of the connections between test items and the knowledge and theory underlying the construct(s) the examination aims to measure. A step-by-step description of the item modeling process is provided and illustrated with an example originating in a realistic clinical diagnostic task. The authors go on to discuss practical considerations that credentialing organizations will find helpful in considering the utility of a model-based approach for their own testing programs.
• Conflicts of Interest in Credentialing, by Peter Mackey- Credentialing programs rely heavily on the contribution of subject matter experts (SMEs) to support the validity of their examinations as measures of relevant competencies. In the final article of this issue, Peter Mackey discusses conflicts of interest that can arise, often unintentionally, among professionals serving as SMEs in support of licensure and certification exams. These individuals are often active in a variety of roles related to their professions and do not always recognize when their overlapping roles may create a potential conflict. The author provides examples of such roles and the reasons that conflicts of interest may not always be recognized, along with insights for preventing, managing, and resolving such conflicts.