The University of North Carolina at Greensboro

School of Education - Educational Research Methodology

Graduate Programs in Research Methodology, Educational Measurement, and Program Evaluation

The Department of Educational Research Methodology (ERM) offers Masters and Doctoral degree programs in the areas of research methodology, educational measurement, and program evaluation. With eight full-time faculty, over 50 graduate students, and approximately 30 graduate-level courses in methods-related topics offered on a regular basis, ERM graduate programs benefit from one of the largest concentrations of research methodology training in the nation. The graduate programs offered by ERM include the M.S. Program, the Ph.D. Program, and the joint M.S./Ph.D. Program.

This page presents an overview of the ERM graduate degree programs, their curriculums, and the nature of the training provided in each. The information on this page is organized according to the five sections listed below. You can jump to a particular section by clicking on the section name in the list below.

1. Overview of Training and Resources
2. Overview of the M.S. Program
3. Overview of the Ph.D. Program
4. Overview of the M.S./Ph.D. Program
5. Overview of the Doctoral Minor in ERM

Click here to view our informational video →

 

1. Overview of Training and Resources

The training provided by ERM graduate programs consists of a combination of carefully crafted course sequences, hands-on experiences, and exceptional student mentoring by the ERM faculty. Each of the ERM degree programs has been designed to meet the needs of the career opportunities associated with the degree. Students in ERM benefit from an deep curriculum of courses in the areas of research methodology, applied statistics, educational measurement, program evaluation and psychometrics. A complete list of ERM courses with accompanying syllabi can be found on the Courses page of this website.  In addition, students in ERM have extensive opportunity to gain hands-on, applied experiences through internships, practicums, and field experiences in the areas of research methodology, educational measurement, and program evaluation. These experiences are organized and supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS). More information about practical learning experiences offered to ERM students can be found on the Practical Experience page of this website.

Graduate Training in Research Methodology

Graduate programs in ERM provide extensive training in research methodology. Coursework pertaining to research methodology spans applied statistics, experimental and quasi-experimental design, analysis of variance, regression, general linear models, multivariate statistics, structural equation modeling (SEM), hierarchical linear modeling (HLM), mixed methods, qualitative methods, survey design and analysis, measurement, and data presentation. Through this course work and applied field experiences students gain broad experience using software for data management and analysis (e.g., Excel, SPSS, SAS, R, LISREL, Mplus, RQDA).

Graduate Training in Educational Measurement

ERM graduate programs provide comprehensive training in educational measurement. Relevant coursework includes foundations of educational measurement, classical test theory, item analysis, item response theory (IRT), multidimensional IRT, polytomous IRT models, IRT parameter estimation, computer adaptive testing, linking and equating, scale design, dimensionality assessment, validity and validation, language testing, cognitive diagnostic modeling, differential item functioning, assessment engineering, and test design. Students are introduced to the most contemporary methods in educational measurement and gain valuable experience using a wide range of measurement computer programs, including IRTPRO, Winsteps, BILOG, MPLUS, PARSCALE, NOHARM, and SPSS. Members of the ERM faculty are internationally recognized experts in educational measurement, serving in numerous leadership positions for the National Council on Measurement in Education (NCME), and are frequent contributors to educational measurement journals such as the Journal of Educational Measurement, Educational Measurement: Issues and Practice, Applied Measurement in Education, Educational and Psychological Measurement, and Applied Psychological Measurement. ERM students benefit from the extensive expertise of ERM faculty in the area of educational measurement, and work collaboratively with ERM faculty on research advancing theory and methods related to educational measurement. Students also gain hands-on experience in educational measurement through working on data analysis contracts supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS).

Graduate Training in Program Evaluation

The training in program evaluation provided by ERM graduate programs draws from a combination of coursework and applied field experiences. Relevant coursework includes training in the theory of program evaluation, direct application of program evaluation methods, case study methods, mixed methods, qualitative analysis, and relevant training in research methodology and measurement (as described above). The comprehensive set of methodological courses offered by ERM (listed on the Courses page of this website) provides our students exceptionally broad training in methods used to evaluate the implmentation and effectiveness of programs. This breadth of methodological coursework is a halmark of ERM training in program evaluation; the training provided by ERM is among the broadest of any training offered in the nation. In addition to methodological coursework, ERM students also deepen their knowledge of program evaluation through working on real program evaluation projects contracted through ERM’s Office of Assessment, Evaluation, and Research Services (OAERS). Much of the hands-on training in program evaluation offered through OAERS consists of serving on program evaluation teams lead by ERM faculty members whereby students are involved in actual program evaluations from the initial planning stages of the evaluation through data collection, data anlaysis, and the final report writing. Methods and results of program evaluations conducted by ERM students are often presented at the annual conferences of the American Evaluation Association and the American Educational Research Association.

Graduate Training in Psychometrics

ERM graduate programs provide extensive training in psychometrics. ERM offers courses in measurement, structural equation modeling (SEM), exploratory and confirmatory factor analysis, scale design, multidimensional scaling, diagnostic classificaiton models, and numerous courses in item response theory. These courses provide foundational training in psychometrics that is further supported by opportunities for students to engage in reserach advancing the theory and application of psychometric methodologies. Members of the ERM faculty are internationally recognized experts in psychometric theory and quantitative psychology, specializing in areas of latent variable modeling, item response theory, cognitive diagnostic modeling, mixture models, multivariate classification methods, latent class analysis, and measurement invariance. These faculty are frequent contributors to psychometric research journals such as Psychometrika, Applied Psychological Measurement, Psychological Methods, and Educational and Psychological Measurement. Students in ERM have extensive opportunity to conduct research in a wide range of areas in psychometrics under the tutelage of ERM faculty.

Hands-On Experience

Across research methodology, educational measurement, program evaluation, and psychometrics students in ERM have ample opportunity to gain hands-on, applied experiences through collaboration on research, participation in funded data analysis and program evaluation contracts, structured field experiences, practicums, and internships. An overview of practical, hands-on learning experiences offered in ERM can be found on the Practical Experience page of this website. Many of these experiences are organized and supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS). For more information about OAERS and the relevant experiences facilitated by OAERS, visit the OAERS site.

 

2. Overview of the M.S. Program

The Master of Science (M.S.) program in ERM provides training in the foundations of research methodology, educational measurement, and program evaluation. Students in the M.S. program receive comprehensive training through a core set of required courses, as well as hands-on, practical experience obtained through field experiences, practicums, and internships coordinated and supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS). Students are able to tailor portions of the curriculum to meet their professional and academic goals through elective courses that allow students to gain a concentration in a particular methodological area of interest, such as program evaluation, educational measurement and assessment, or statistical modeling.

All required courses of the M.S. program are offered in the early evening to accommodate individuals with other commitments during the day. The courses offered in the M.S. program place and emphasis on application to real-world problems and the use of computer software in conducting relevant analyses. While it is assumed that students entering the M.S. program have taken an introductory course in statistics, required courses for the M.S. do not assume extensive previous training in statistics or methodology.

Curriculum of the M.S. Program

The M.S. program consists of 30 required credit hours (10 courses). Of the 30 required credit hours, 24 credit hours correspond to the following eight core required courses:

  • ERM 600: Validity and Validation
  • ERM 604: Methods of Educational Research
  • ERM 642: Evaluation of Educational Programs
  • ERM 667: Foundations of Educational Measurement Theory
  • ERM 668: Survey Research Methods in Education
  • ERM 675: Data Presentation and Reporting
  • ERM 680: Intermediate Statistical Methods in Education
  • ERM 681: Design and Analysis of Educational Experiments

Descriptions of these courses, along with their syllabi,  can be found on the Courses Page of the ERM website.

In addition to the eight core required courses listed above, students are required to take a minimum of six credit hours (e.g., 2 courses) in one or more elective areas of concentration. The elective courses chosen by the student in coordination with the student’s advisor, and are intended to provide the student with an opportunity to gain expertise in one or more particular areas of research methodology that aligns with the students professional and academic goals. For example, students seeking broader training in program evaluation may opt to take ERM 643 (Applied Educational Evaluation), where they are part of a real program evaluation team that is supervised by an ERM faculty member. As another example, students seeking broader training measurement and assessment may opt to take ERM 669 (Item Response Theory) or ERM 633 (Language Assessment and Testing). Students can also take elective courses in Departments outside of ERM in areas such as statistics, computer programming, psychology, educational leadership, and public health. Lastly, students may fulfill elective credits to complete an internship, practicum, or field experience in research methodology, educational measurement, or program evaluation. Many of these opportunities are supported through OAERS, and more information about applied experiences offered to students in ERM graduate programs can be found on the Practical Experiences Page of the ERM website.

Who Should Apply to the M.S. Program?

The M.S. program is intended for any individual with a professional or intellectual interest in how we can use data to inform: (a) the answers to research questions; (b) the mechanisms underlying educational and social programs; and (c) the assessment of knowledge, skills, abilities, and cognitive traits of individuals. The training provided by the program can be applied across a wide range of professional areas including education, psychology, the health sciences, and public policy. The M.S. degree is particularly well suited for individuals having one of the following professional goals:

Career in Methodology and Evaluation: The M.S. program offers the requisite training for a career providing methodological support to organizations involved in research, program evaluation, and assessment. Students completing the M.S. program have a range of career options in the areas of research study design, data collection and analysis, program evaluation, and assessment. Students graduating from the M.S. program are employed in a variety of professional settings, including school districts, state boards of education, assessment and testing organizations, organizations conducting research, organizations conducting program evaluations, school districts, and other organizations that make data-driven decisions.

Professional Advancement: The M.S. program is often attractive to individuals who are currently employed, but seek professional advancement through gaining valuable skills in research methodology. Most of the coursework of the M.S. program is offered during evening hours to accommodate individuals who are working during the day.

Pursuing a Ph.D. in Methodology: The M.S. program provides appropriate training for individuals looking to complete a Ph.D. in research methodology, quantitative methods, measurement sciences, and program evaluation.

Pursuing a Ph.D. in Other Research-Intensive Fields: The M.S. program provides fantastic training for individuals ultimately interested in doctoral training in substantive fields that employ quantitative research methods; examples of such fields include education, psychology, the health sciences, and public policy. Doctoral programs in these substantive fields often seek applicants having strong skills in research methodology and applied statistics, and thus the quantitative methodology training provided by M.S. program can be a strong asset to students applying to such programs.

 

3. Overview of the Ph.D. Program

The Doctoral (Ph.D.) program aims to develop students into expert methodologists who are appropriately prepared to pursue influential careers in the fields of research methodology, educational measurement, and program evaluation. To this end, the training offered by the Ph.D. program has the joint focus of providing students with a comprehensive training across the major domains of research methodology, while also having students gain extensive knowledge in one or more areas of concentration that align with the student’s professional and intellectual interests. In ERM, areas of concentration typically correspond to one of three broad methodological fields:

1. Educational Measurement and Psychometrics
2. Program Evaluation Methods
3. Statistical Modeling

Extended study in the student’s area(s) of concentration comes from the completion of advanced coursework in the area, the development of a line of research in the area, applied field experiences, and mentored learning in the area under the supervision of the student’s graduate research advisor. Research experience is obtained by working collaboratively with ERM faculty on research studies. As students progress through the doctoral program they develop their own research topics that ultimately lead to their dissertation topic. Many of our students present their research at national and international conferences, such as the annual meetings of the American Educational Research Association (AERA), the American Evaluation Association (AEA), and the National Council on Measurement in Education (NCME).

Applied Field Experiences: The Ph.D. program provides its students a range of opportunities to be involved in evaluation, measurement, and data analysis projects. Some of these opportunities are internships in organizations specializing in research, large-scale testing, and program evaluation. Other opportunities stem from funded evaluation and data analysis contracts overseen by ERM faculty. Many of these hands-on, practical learning experiences are coordinated and supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS).

Curriculum of the Ph.D. Program

The Ph.D. program consists of 66 required credit hours. Of the 66 required credit hours, 33 credit hours correspond to the following 11 core required courses:

  • ERM 633: Language Assessment and Testing
  • ERM 642: Evaluation of Educational Programs
  • ERM 643: Applied Educational Evaluation
  • ERM 668: Survey Research in Education
  • ERM 669: Item Response Theory
  • ERM 675: Data Presentation and Reporting
  • ERM 682: Multivariate Analysis
  • ERM 727: Computer-Based Testing: Methods and Applications
  • ERM 728: Exploratory and Confirmatory Factor Analytic Methods for Scale Construction
  • ERM 729: Advanced Item Response Theory
  • ERM 731: Structural Equation Modeling

Descriptions of these courses, along with their syllabi, can be found on the Courses Page of the ERM website.

In addition to the 11 required courses, students must complete 21 credit hours in elective courses. These elective credit hours provide each student the opportunity to develop a concentration in one or more areas that align with the student’s particular professional and academic goals. The list of approved elective courses includes:

  • ERM 636 Advanced Studies in Second Language Testing
  • ERM 688 Contemporary Problems Seminar
  • ERM 692 Independent Study
  • ERM 711 Experimental Course
  • ERM 725 Applied Methods on Educational Research
  • ERM 726 Advanced Topics in Educational Measurement
  • ERM 730 Practicum in Educational Research and Evaluation
  • ERM 732 Hierarchical Linear Modeling
  • ERM 734 Equating
  • ERM 735 Multidimensional Item Response Theory
  • ERM 742 Advanced Topics in Evaluation of Educational Programs
  • ERM 750 Case Study Methods in Educational Research
  • TED 730 Qualitative Analysis
  • STA 551 Introduction to Probability
  • STA 552 Introduction to Mathematical Statistics

Descriptions and syllabi of many of these elective courses can be accessed on the Courses Page of the ERM website. In addition, students may take courses not on the list of elective courses pending approval from the student’s Advisory/Dissertation Committee.

In addition to coursework, students must also complete a dissertation (12 credit hours). The dissertation is conducted under the mentorship of an ERM faculty member, and typically involves a research study that advances the theory or application of one or more areas of methodology. Examples of recent ERM dissertation titles can be found on the ERM Alumni Page of the ERM website.

Who Should Apply to the Ph.D. Program?

The Ph.D. program is intended for students who are interested in becoming an expert in research methodology, the measurement sciences, or program evaluation. Individuals applying to the Ph.D. program are assumed to have previous foundational training in quantitative methodology and/or evaluation, such as a master’s degree in a related field (methodology, measurement, evaluation, statistics, or mathematics) or other graduate training that aligns with the training provided by the ERM M.S. degree. Those who are interested in pursuing the Ph.D. in ERM, but have not completed the necessary graduate training in quantitative methodology and/or evaluation, are encouraged to apply to the joint M.S./Ph.D. program.

 

4. Overview of the M.S./Ph.D. Program

The M.S./Ph.D. program in ERM provides comprehensive training that spans the curriculums of the M.S. and Ph.D. degrees. The M.S./Ph.D. program is designed to allow students to progress through the M.S. and Ph.D. degrees faster than if the student completed the M.S. and Ph.D. programs individually. The M.S./Ph.D. program contains the same core requirements of the individuals M.S. and Ph.D. programs with respect to ERM coursework, but gains efficiency by removing redundancies between the individual M.S. and Ph.D. programs completed separately.

Because the M.S./Ph.D. program is based on the curriculums of the individual M.S. and Ph.D. programs, information about the goals of the M.S./Ph.D. program can be obtained from the descriptions of the M.S. program and Ph.D. program given on this site.

Applied Field Experiences: The M.S./Ph.D. program provides its students a range of opportunities to be involved in evaluation, measurement, and data analysis projects. Some of these opportunities are internships in organizations specializing in research, large-scale testing, and program evaluation. Other opportunities stem from funded evaluation and data analysis contracts overseen by ERM faculty. Many of these hands-on, practical learning experiences are coordinated and supported by ERM’s Office of Assessment, Evaluation, and Research Services (OAERS).

Curriculum of the M.S./Ph.D. Program

The M.S./Ph.D. program consists of 84 required credit hours of coursework. Of the 84 required credit hours, 48 credit hours correspond to the following 16 core required courses:

  • ERM 600: Validity and Validation
  • ERM 604: Methods of Educational Research
  • ERM 633: Language Assessment and Testing
  • ERM 642: Evaluation of Educational Programs
  • ERM 643: Applied Educational Evaluation
  • ERM 667: Foundational of Educational Measurement Theory
  • ERM 668: Survey Research in Education
  • ERM 669: Item Response Theory
  • ERM 675: Data Presentation and Reporting
  • ERM 680: Intermediate Statistical Methods in Education
  • ERM 681: Design and Analysis of Educational Experiments
  • ERM 682: Multivariate Analysis
  • ERM 727: Computer-Based Testing: Methods and Applications
  • ERM 728: Exploratory and Confirmatory Factor Analytic Methods for Scale Construction
  • ERM 729: Advanced Item Response Theory
  • ERM 731: Structural Equation Modeling

Descriptions of these courses, along with their syllabi,  can be found on the Courses Page of the ERM website.

In addition to the 16 required courses, students must complete 24 credit hours in elective courses. These elective credit hours provide each student the opportunity to develop a concentration in one or more areas that align with the student’s particular professional and academic goals. The list of approved elective courses includes:

  • ERM 636 Advanced Studies in Second Language Testing
  • ERM 688 Contemporary Problems Seminar
  • ERM 692 Independent Study
  • ERM 711 Experimental Course
  • ERM 725 Applied Methods on Educational Research
  • ERM 726 Advanced Topics in Educational Measurement
  • ERM 730 Practicum in Educational Research and Evaluation
  • ERM 732 Hierarchical Linear Modeling
  • ERM 734 Equating
  • ERM 735 Multidimensional Item Response Theory
  • ERM 742 Advanced Topics in Evaluation of Educational Programs
  • ERM 750 Case Study Methods in Educational Research
  • TED 730 Qualitative Analysis
  • STA 551 Introduction to Probability
  • STA 552 Introduction to Mathematical Statistics

Descriptions and syllabi of many of these elective courses can be accessed on the Courses Page of the ERM website. In addition, students may take courses not on the list of elective courses pending approval from the student’s Advisory/Dissertation Committee.

In addition to coursework, students must also complete a dissertation (12 credit hours). The dissertation is conducted under the mentorship of an ERM faculty member, and typically involves a research study that advances the theory or application of one or more areas of methodology. Examples of recent ERM dissertation titles can be found on the ERM Alumni Page of the ERM website.

Who Should Apply to the M.S./Ph.D. Program?

The M.S./Ph.D. program is intended for individuals seeking a Ph.D. in ERM who have not previously completed a graduate degree (e.g., master’s degree) in a related field (e.g., quantitative methods, evaluation, statistics). Students are expected to have taken at least an introductory course in statistics, but may have undergraduate training in a broad range of disciplines that include, but are not limited to, education, psychology, mathematics, statistics, and other fields in the behavioral and health sciences.

 

5. Overview of the Doctoral Minor in ERM

The Doctoral Minor in ERM is intended to provide methodological training to non-ERM doctoral students at UNCG. By way of pursuing the minor, students are able to obtain valuable research skills that can be applied to their own substantive areas. The skills obtained via the minor can improve research practices of students while attending UNCG and throughout their career. Upon completing the minor, students will have developed a strong foundation in one or more of the following areas: research methodology, measurement, assessment, and program evaluation.

There is no formal application for the minor program, as it is available to any student enrolled in a UNCG doctoral program. In addition, the minor program can be completed concurrently with the student’s doctoral program such that ERM courses counting towards the student’s program of study can also count towards the ERM minor.

The Doctoral Minor in Educational Research Methodology requires 15 semester hours completed from the following list of courses.

  • ERM 600: Validity and Validation (3)
  • ERM 633: Language Assessment and Testing (3)
  • ERM 636: Advanced Studies in Second Language Testing (3)
  • ERM 642: Evaluation of Educational Programs (3)
  • ERM 643: Applied Educational Evaluation (3)
  • ERM 667: Foundations of Educational Measurement Theory (3)
  • ERM 668: Survey Research Methods in Education (3)
  • ERM 669: Item Response Theory (3)
  • ERM 675: Data Presentation and Reporting (3)
  • ERM 680: Intermediate Statistical Methods in Education (3)
  • ERM 681: Design and Analysis of Educational Experiments (3)
  • ERM 682: Multivariate Analysis (3)
  • ERM 685: R for Education and the Social Sciences (3)
  • ERM 693: Seminar in Advanced Research Methods (3)
  • ERM 726: Advanced Topics in Educational Measurement (3)
  • ERM 727: Computer-Based Testing: Methods and Applications (3)
  • ERM 728: Exploratory and Confirmatory Factor Analytic Methods for Scale Construction (3)
  • ERM 729: Advanced Item Response Theory (3)
  • ERM 731: Structural Equation Modeling (3)
  • ERM 732: Hierarchical Linear Modeling (3)
  • ERM 734: Equating (3)
  • ERM 735: Multidimensional Item Response Theory (3)
  • ERM 742: Advanced Topics in the Evaluation of Educational Programs (3)
  • ERM 750: Case Study Methods in Educational Research (3)
Connect with the School of Education!