Psychometric Services

Test data directly affect not only the lives of students and their families, but the allocation of millions of public tax dollars to educational programs. When a test score is presented to a parent, teacher, or student, there is a presumption that the score has meaning for a student’s achievement. In public education, test data are routinely used by teachers, school administrators, policy makers and institutional staff to inform decision making for students for purposes such as formative assessment at the classroom level, high-stakes summative testing for grade promotions and high school graduation, and for college admissions decisions.

These uses of test data imply a professional responsibility on the part of a testing company to comply with the highest professional standards governing the development, administration, and use of tests. Psychometricians are the most highly trained professionals in the testing industry and are essential for ensuring the quality and responsible use of tests. Practically, the activities of psychometricians are oriented toward gauging the accuracy of a score and advising as to the uses and inferences that can be drawn from students’ test scores.

Pacific Metrics is committed to offering high levels of psychometric expertise in conjunction with all of its product and service offerings. Psychometricians at Pacific Metrics play leading roles in all aspects of test development, administration, scoring, and reporting of test results, and they are available for consulting with policy making bodies on these activities. To this end, we are proud to have an experienced team of measurement professionals on staff. Our research team includes:

• Dr. Howard C. Mitzel, President and Research Director
• Dr. W. Alan Nicewander, Chief Psychometrician
• Dr. E. Matthew Schulz, Sr. Psychometrician
• Dr. Susan Lottridge, Psychometrician
• Mr. Donald Murray, Director of Statistical Services

Dr. Howard C. Mitzel, President and Research Director

Dr. Howard C. Mitzel is co-founder and President of Pacific Metrics. Dr. Mitzel holds a Ph.D. in Research Methodology and Quantitative Psychology from the University of Chicago and is widely known in the assessment community for his innovative work in large-scale testing programs. Dr. Mitzel is a co-developer of the Bookmark standard-setting procedure, which is the most widely-used tool for setting defensible standards in large-scale K-12 assessment programs. He is the architect of Pacific Metrics psychometric software for equating and technical form construction. He has served as Louisiana’s lead psychometrician for the Graduation Exit Exam since its inception. Dr. Mitzel serves on several technical advisory committees and is a frequent contributor to CCSSO’s Large-Scale Assessment conference. His specializations include:

• Item response theory (IRT) -based calibration, test equating
• Standard setting
• Psychometric software development
• Assessment policy

Dr. W. Alan Nicewander, Chief Psychometrician

Dr. Nicewander is Pacific Metrics Chief Psychometrician and a practicing expert in educational and psychological measurement for more than 30 years. An innovator of many contributions, he is best known for using IRT for solving applied testing problems. Dr. Nicewander has worked in the academic, commercial and government sectors. Highlights of his distinguished career include Professor and Chairman of the Department of Psychology at the University of Oklahoma, Director of Statistical Research for ACT, Inc., and Head of the Personnel Testing Division, Department of Defense. In this last position, Dr. Nicewander oversaw the highly successful conversion of the Armed Forces Vocational Aptitude Test from paper-and-pencil assessment to an adaptive, computer-administered test. This experience positions him as one of the foremost experts in computer-based testing in the nation. Dr. Nicewander has written dozens of journal articles dealing with both theoretical and applied measurement. His three most recent articles address accounting for measurement error in selection tests, analysis of data from extreme groups, and model-based methods for equating test forms. Most recently, Dr. Nicewander has been working on growth measurement and vertical scaling methodology. His specializations include:

• IRT-based calibration, vertical scaling
• Computer-based testing
• Computerized adaptive testing
• Simulation studies
• Applied Statistics

Dr. E. Matthew Schulz, Sr. Psychometrician

Dr. Schulz joined Pacific Metrics in 2007 as Senior Psychometrician. Dr. Schulz comes to Pacific Metrics from ACT, Inc., where he was a Principal Research Statistician. Over the past 15 years he developed techniques based on IRT and domain scores for assigning students to achievement levels and for quantifying the consistency and accuracy of achievement level scores. Dr. Schulz’s work in this area earned him national recognition as a leader in domain score theory and led to contracts with the U.S. Department of Education for setting achievement levels on the National Assessment of Educational Progress (NAEP). His role as Project Director led to the development of a Bookmark-based standard-setting method called Mapmark and the subsequent adoption of Mapmark for setting NAEP achievement levels. Prior to his tenure at ACT, Dr. Schulz served as Director of Testing at the National Council of State Boards of Nursing, where he supervised research and development for transitioning the nursing licensure examination to a computer-adaptive format. As coordinator for the development of criterion referenced tests at Chicago Public Schools from 1987 to 1989, Dr. Schulz developed vertical scales and promoted their use for measuring student growth.

Dr. Schulz has published widely in the area of educational measurement, and his articles are frequently referenced by others in the field. He regularly serves as a consultant to state boards of education, school districts and licensure and certification boards on topics of equating, vertical scaling, job analysis and computerized testing. His research specializations include:

• IRT-based calibration, test form equating
• Growth measurement and vertical scaling
• Standard setting
• Domain score theory
• Computer-based testing

Dr. Susan Lottridge, Psychometrician

Dr. Lottridge joined the Pacific Metrics team in 2006. Her current responsibilities include conducting the psychometric and statistical analyses involved in test development, administration and scoring. She is also investigating and incorporating automated essay scoring into current Pacific Metrics software applications. For three years prior to joining Pacific Metrics, Dr. Lottridge was an intern and graduate student at James Madison University. Through this work, she participated in the development, administration and analyses for both large- and small-scale assessment plans involving pencil-and-paper as well as computer-based tests. Prior to her graduate work in psychometrics, Dr. Lottridge worked as an educational evaluator for six years at the University of Wisconsin at Madison. Much of this work centered on the impact of educational innovation in science classrooms using qualitative and survey methods.

Dr. Lottridge holds a Ph.D. from James Madison University’s Assessment and Measurement Program, a Master’s Degree in Computer Science and a Master’s Degree in Mathematics from the University of Wisconsin at Madison. Her specializations include:

• IRT-based calibration and vertical scaling
• Computer-based testing
• Computer-based scoring
• Simulation studies

Mr. Donald Murray, Director of Statistical Services

Mr. Murray joined the Pacific Metrics team in 2005 as Director of Statistical Services. His current responsibilities include supporting contract obligations with specified data analysis, investigating the applicability of other technologies to assessment and instruction, assisting with the management of Pacific Metrics Item Management System (IMS) and ORCA™ Engine, maintaining Pacific Metrics psychometric software and managing a number of other technical projects.

Mr. Murray’s background of experience in education and assessment includes over six years in large-scale assessment data analysis at CTB/McGraw-Hill, as well as eight years in secondary mathematics instruction. His combined work experience and additional scientific training make him an accomplished and valued member of the company’s technical staff. He holds a B.S. in Mathematics from Colorado State University at Fort Collins. Mr. Murray’s areas of specialization include:

• Statistical programming and data analysis
• Database design and data management
• IRT-based calibration