Psychometric Software
I-Vault™
Web-based item management system for large-scale assessment needs
I-Vault is a secure, web-based, test-item management system specifically designed to store items and statistical data for large-scale assessment needs. This flexible solution is available in a full-service model hosted and managed by Pacific Metrics, or as an internal solution managed by a state education agency or assessment provider.
In the state-level application, data-based item images and their attributes, such as item statistics, are shared among state education department working groups, vendors, and review committees. I-Vault establishes a standard interface for information exchange used by vendors to build test forms, by reviewers to edit test items, and by education department personnel to review and manage the material that is central to all assessment programs. The base system can be easily customized to meet client-specific needs.
FormBuilder
Software for test construction, item selection, and pre-equating
FormBuilder is an assessment tool that can be used “as is” or customized for a client’s specific use. Assessment organizations use this software to create statistically equivalent test forms for high-stakes assessments. Test development staff can create pre‐equated test forms by utilizing a user-friendly graphical interface coupled with item-data and test-characteristic curves. FormBuilder can be customized to establish raw-score-to-scale-score relationships.
ParTrans Software
Software for creating, editing, and transforming parameter files
In educational measurement, it is often desirable to have the scale of measurement linked to the number-correct (raw) score and to know how accurately a test measures in various regions of the score scale. Pacific Metrics developed the ParTrans program to accomplish this psychometric function.
The ParTrans program computes number-correct-to-scale-score (NCSS) tables along with conditional standard errors of measurement (SEMs) for each score. Both multiple-choice and constructed-response items may be used. Two methods for computing NCSS tables are included: (1) the inverse test characteristic curve (TCC) method with exact SEMs, and (2) expected a-posteriori (EAP) theta estimates with SEMs computed as posterior standard deviations. This program handles both multiple-choice and constructed-response items. The input is a set of equated IRT item parameter estimates.
TranScale
Software to equate parameter files linked by common item designs
TranScale performs equating to put multiple forms of a test on the same scale of measurement. TranScale equates test forms using Item Response Theory (IRT) methods and will handle both multiple-choice and constructed-response items. Three types of equating are possible using this software:
1) Mean/Sigma IRT equating
2) Stocking-Lord IRT equating
3) Nicewander IRT equating (equating by parameterized test characteristic curves)
The goodness of the equatings are summarized by (a) the average, weighted distance between anchor and equated TCCs, (b) the correlation between p-values predicted by the anchor and equated item parameters, (c) the bias in predicted item p-values (the difference in the means of the two sets of predicted p-values, (d) the root mean square difference in the two sets of p-values, and (e) a statistical test (F-ratio) that the best line relating the two sets of predicted p-values is the identity line (slope=1, intercept=0). The inputs to this program are IRT parameter estimates for a set of “anchor” data and a set of “target” data.
