Overview of Careers
Training provided by the M.S. and Ph.D. degrees in the Department of Educational Research Methodology prepares individuals for a range of careers in the measurement sciences, program evaluation, and quantitative methodology. Individuals with training in these areas are in very high demand in a variety of professional settings. With an increasing emphasis being placed on data-driven decisions in organizations and institutions associated with education, health outcomes, and other public policies, high demand for individuals with advanced training in the measurement sciences, program evaluation, and quantitative methods is expected to continue for the foreseeable future.
Careers in Measurement and Assessment
The use of assessments to measure knowledge, skills, abilities, cognitive traits, and health outcomes is pervasive in our culture. Such assessments are countless, and include large-scale academic assessments (e.g., SAT®, ACT®, MCAT®), assessments used in K-12 educational settings for monitoring student achievement and growth, licensure and certification exams, psychological inventories, and measurement instruments used in the health sciences, human resources, and organizational management. The field of measurement sciences encompasses the methodologies and statistical models used in developing assessments, assigning scores to individuals on assessments, and evaluating the validity and reliability of scores generated by assessments. These methodologies and models include classical test theory, item response theory (IRT), linking and equating, scaling, computer adaptive testing, language testing and assessment, diagnostic modeling, and dimensionality analysis.
Specialists in the measurement sciences are employed in a range of non-profit and for-profit organizations including testing organizations (e.g., ETS, ACT), school districts, state boards of education, research and evaluation centers, colleges and universities, and also serve as private measurement consultants. The particular careers in these areas range from highly technical positions that concentrate on conducting rigorous statistical analyses to project management and/or director positions focusing on the development, administration, scoring, and score reporting associated with assessments.
View Current Job Postings
Examples of professional positions related to measurement and assessment can be found in several job posting websites, including:
- The Career Listings of the National Council on Measurement in Education
- Job Postings of professional recruiting firms specializing in measruement and assessment, such as Horizon Search, Inc.
- The Online Job Board of the American Educational Research Association
Item Response Theory
Item response theory, or IRT, is a statistical modeling framework that is a fundamental component to the development and scoring of most large-scale standardized tests. A description of IRT can be found at the OAERS page for Item Response Theory. Because IRT is a specialized analysis framework that is needed throughout the testing and research industry, individuals with solid training in IRT are in very high demand with testing and research organizations. Recognizing the importance of IRT to the testing and research organizations, ERM offers one of the broadest set of courses related to IRT of any graduate training program in the world. Our curriculum spans a series of seven courses related to IRT: ERM 669 Item Response Theory, ERM 729 Advanced Item Response Theory, ERM 735 Multidimensional Item Response Theory, ERM 727 Computer-Based Testing, ERM 734 Equating, and advanced topics courses in Cognitive Diagnostic Modeling and Assessment Engineering (ERM 726). Syllabi for these courses can be found on the ERM Course Syllabi Page.
Careers in Program Evaluation
Programs aimed at supporting and enhancing the well-being of individuals, communities, and organizations are prevalent throughout the fields of education, business, and the social and health services. These programs represent a wide range of sizes, scopes, and settings, including educational supports and interventions, public policies, and health services. In order to determine how a given program can be improved to better meet its goals, the program or organization is evaluated using a systematic assessment of the various program phases like the operations, planning, implementation, objectives, and impact. To meet that end, evaluators must possess working knowledge of methodologies used in conducting systematic assessment and inquiry, including statistical methods, experimental and quasi-experimental research designs, qualitative approaches, and mixed methodologies.
Individuals specializing in program evaluation are employed in non-profit and for-profit organizations involved in evaluation and research, research agencies, colleges and universities, school districts, federal organizations and state boards of education, and also serve as private evaluation consultants.
View Current Job Postings
Examples of professional positions related to program evaluation can be found in several job posting websites, including:
Careers in Quantitative Methodology
Quantitative research in education and the behavioral and health sciences makes use of many different methodologies. These methods include descriptive and inferential statistical methods associated with making statements about individual population parameters, and extend to rigorous statistical modeling methods that include general linear models, structural equation modeling, hierarchical linear modeling, and other forms of latent trait and latent class modeling. All of these statistical methods play a fundamental role in determining the effectiveness of interventions and treatments, shedding light on the relationships between important outcome variables, and establishing the requisite properties of a research study (e.g., sample size) in order to appropriately answer the desired research questions.
Individuals specializing in quantitative methodology are employed in non-profit and for-profit organizations involved in quantitative research, research agencies, colleges and universities, school districts, federal organizations and state boards of education, and also serve as private statistical consultants. Careers in applied statistics span a wide range of opportunities, from highly technical positions that concentrate on conducting rigorous statistical analyses to project management and/or director positions responsible for overseeing particular data-based and research initiatives.