Methodology
The following databases provide information about tests, including validity and reliability. Only APA PsycTests provides actual text of tests. Some dissertations will also provide the the text of tests.
SAGE Research Methods Online is a methods library with books, reference works, journal articles, and instructional videos from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. The site is designed to guide users to the content they need to learn a little or a lot about their method. The Methods Map can help those less familiar with research methods to find the best technique to use in their research.
SAGE Research Methods Cases is a collection of case studies of real social research, specially commissioned and designed to help you understand abstract methodological concepts in practice.
Sherman Library Video: Finding Tests and Measurements
Buros Center for Testing Videos
The Buros Center for Testing offers seven videos in their Evaluating Test Quality: Advice From Mental Measurements Yearbook Reviewers series. The links below should take you to the respective video hosted in the Alexander Street Press database.
- The Argument-based Approach to Validation In his intermediate-level lecture, Dr. Michael Kane will discuss how explicit recognition of the inferences and assumptions underlying the interpretation and use of test scores can guide the process of validation by identifying which kinds of evidence are needed to evaluate the claims being made.
- Evaluating Clinical Screening Measures This is an intermediate-level presentation. Screening measures have historically been used for the early identification of those persons who are potentially at high risk for a specific disorder or condition. A multitude of clinical screening instruments are available commercially and in the public domain and provide screens for conditions such as anxiety and depression, dyslexia, posttraumatic stress, suicide, and so forth. Many screening measures have proven to be valuable tools for the clinician due to their ease of use, practical utility, and purported value in clinical practice. However, despite claims to the contrary, many clinical screening measures fail to meet acceptable standards. This webinar will provide an overview of clinical screening measures and will differentiate them from standard diagnostic measures. It will also provide information on how to critically evaluate clinical screening measures by applying the parameters identified in Standards for Educational and Psychological Testing (2014). Critical reviews of recently reviewed clinical screening measures will provide practical examples on how clinicians, researchers, and test developers may approach the development and subsequent use of clinical screening measures.
- Evaluating Tests of Intelligence This is an intermediate-level presentation. Surveys of test usage consistently indicate that tests of intelligence are among the most frequently utilized assessment tools in educational, occupational, and clinical settings. These tests provide standardized samples of behavior useful for norm-referenced comparisons that facilitate inferences about how individuals are likely to perform and participate within social, academic, and occupational settings. Tests of intelligence are utilized for a variety of purposes, perhaps most notably as part of the following processes: (a) eligibility determination for special education and related services, (b) eligibility determination for Social Security income benefits, (c) selection of employees in industrial and business settings, (d) problem-solving activities in educational settings, and (e) diagnostic formulation and development of treatment goals and strategies in clinical practice. No test is universally valid for all purposes. Thus, test users must evaluate tests relative to their intended interpretations and uses of test scores. As tests of intelligence are routinely used to inform important decisions, it is imperative that test users be familiar with evaluation criteria and empirical research so that they can better evaluate the adequacy of these tests for specific purposes.
- Evaluating the Definition of a Test Domain Psychological processing is often addressed in theories that give rise to assessments, but careful domain specification and sampling plans for processing is often lacking in commercially available tests. In achievement testing more specifically and educational testing more generally the definition of a test domain, as well as its blueprint (the sampling plan of the domain) often delineates only content, if even that. The cognitive processing dimension, crucial to a full understanding of what an examinee can do, is often not specified, except perhaps after the fact in evaluating the scope of a given assessment. A similar dimension is also relevant in psychological assessments designed to discover in what way and/or how people process material. The material itself is independent of the person, but is nevertheless important because the processing is only meaningful in relation to it. Yet only with an understanding of processing can it be ensured that test stimuli elicit the range of internal activities intended.This intermediate-level presentation will discuss this prevalent deficiency in many, if not most, published tests. Possible reasons why this deficiency exists, the effects of the deficiency on interpretability, and how domain specifications can improve test documentation for users and development of new forms by test publishers and authors will be discussed.
- Guiding Principles for Evaluating an Educational or Psychological Assessment Because there is no simple standard checklist for evaluating an educational or psychological assessment, this intermediate-level webinar will focus on several important and timely principles for guiding an evaluation of assessment quality. Key issues to be discussed include judging the adequacy of validity evidence, score reliability/precision, and cross-cultural translation and adaptation.
- How to Evaluate and Review a Test: Reflections of a Long-Time Mental Measurements Yearbook Reviewer This introductory-level webinar outlines thought processes that might be involved in completing a Mental Measurements Yearbook review of a test. Some parts of the review process present little difficulty. Other parts entail considerable challenge. Identifying the test's purpose and describing the test's materials and processes usually present little difficulty. Perhaps surprisingly, evaluating the standardization (the norms) is often straightforward. Four aspects of the review process frequently present special challenges. First, many test manuals present an overwhelming amount of data on reliability and validity. The reviewer must sort through all this information and formulate overall conclusions that can be conveyed in relatively succinct language. Second, many tests provide a host of scores. An important, and sometimes tricky, part of the review process is spotting all the scores (and suggested interpretations) and determining if adequate technical information is available for all of them. Third, fairness has emerged as a major concern, now ranking with the traditional "big three" of norms, reliability, and validity. Many tests have not yet adequately treated this subject. Some have covered it, but only in the most cursory manner. Fourth, in formulating an overall conclusion about the test, the reviewer faces the difficult task of deciding whether to comment exclusively about the test under review or to place it in the wider context of other tests in the same domain.
- Screening and Progress Monitoring Tools: Issues and Implications for Clinical and School Psychologists Settings in which screening and progress measures are used vary as does the purpose of the assessment. School psychologists may use measures to determine the need for additional assessment, or in the case of an omnibus instrument, as an evaluation of social-emotional functioning that is required as part of eligibility evaluations. They also monitor the progress of interventions. Clinical psychologists use these same tools, although the emphasis is typically on diagnosis and intervention planning/evaluation. In this intermediate-level presentation, issues to consider when selecting a test such as sensitivity, reliability and validity will be examined across the different types of measures and settings/purposes. Measures such as the BASC-3 and the BASC-PM can be used for these varied purposes and will be used as cases in point. The examination of data and its interpretation for purposes such as special education eligibility, diagnosis, selection and planning treatment, and monitoring the progress of interventions will be addressed.