Chapter 3 – Evaluation Rubric
Criteria Does Not Meet 0.01 points Meets/NA 1 point
Introductory Remarks The section is missing; or some topic areas are not included
in the Introduction or are not explained clearly.
The chapter outline is not provided and/or is unclear.
The reader is adequately oriented to the topic areas
covered. An outline of the logical flow of the chapter is
presented.
All major themes/concepts are introduced.
Research Methodology and
Design
There is a lack of alignment among the chosen research
method and design and the study’s problem, purpose, and
research questions.
There is a lack of justification and alternate choices for
methods.
For Qualitative Studies: Lacks clear discussion of the
study phenomenon, boundaries of case(s), and/or
constructs explored.
Describes how the research method and design are aligned
with the study problem, purpose, and research questions.
Uses scholarly support to describe how the design choice
is consistent with the research method, and alternate
choices are discussed.
For Qualitative Studies: Describes the study
phenomenon, boundaries of case(s) and/or constructs
explored.
Population and Sample Lacks a description of the sample, demographics, and the
representation of the sample to the broader population.
There is little to no description of the inclusion/exclusion
criteria used to select the participants (sample) of the study.
For Quantitative Studies: A power analysis is not
described and appropriately cited.
Provides a description of the target population and the
relation to the larger population.
Inclusion/exclusion criteria for selecting participants
(sample) of the study are noted.
For Quantitative Studies: Power analysis is
described and appropriately cited.
Materials/
Instrumentation
Lacks a description of the instruments associated with the
chosen research method and design used. Details missing
regarding instrument origin, reliability, and validity.
For Quantitative Studies: (e.g., tests or surveys). Lacks
explanation of any permission needed to use the
instrument(s) and cites properly. Instrument permissions are
missing in appendices
For Qualitative Studies: (e.g., observation
checklists/protocols, interview or focus group discussion
Handbooks). Did not clearly explain the process for
conducting an expert review of instruments (e.g., provides
justification of reviewers being credible – reviewers may
include, but not limited to NCU dissertation team members,
professional colleagues, peers, or non-research participants
representative of the greater population); and/or did not
clearly explain use of a field test if practicing the
administration of the instruments is warranted.
For Pilot Study: Does not clearly explain the procedure for
conducting a pilot study (did not conduct pilot) if using a
self-created instrument (e.g., survey questionnaire); does not
include explanation of a field test if practicing the
administration of the instruments is warranted.
Provides a description of the instruments associated with
the chosen research method and design used. Includes
information regarding instrument origin, reliability, and
validity.
For Quantitative Studies: (e.g., tests or surveys).
Includes any permission needed to use the instrument(s)
and cites properly.
For Qualitative Studies: (e.g., observation
checklists/protocols, interview or focus group discussion
Handbooks). Describes process for conducting an expert
review of instruments (e.g., provides justification of
reviewers being credible – reviewers may include, but not
limited to NCU dissertation team members, professional
colleagues, peers, or non-research participants
representative of the greater population); describes use of
a field test if practicing the administration of the
instruments is warranted.
For Pilot Study: Explains the procedure for conducting a
pilot study (requires IRB approval for pilot) if using a
self-created instrument (e.g., survey questionnaire);
explains use of a field test if practicing the administration
of the instruments is warranted.
Operational Definitions
of Variables
(Quantitative Studies
Only)
Discussion of the study variables examined is lacking
information and/or is unclear.
Describes study variables in terms of being measurable
and/or observable.
(Reviewer – mark Meets/NA for Qualitative studies)
Procedures Procedures are not clear or replicable. Steps are missing;
recruitment, selection, and informed consent are not
established. IRB ethical practices are missing or unclear.
Describes the procedures to conduct the study in enough
detail to practically replicate the study, including
participant recruitment and notification, and informed
consent. IRB ethical practices are noted.
Data Collection and
Analysis
Does not clearly provide a description of the data and the
processes to collect data. Lack of alignment between the
data collected and the research questions and/or hypotheses
of the study.
For Quantitative Studies: Does not clearly provide the data
analysis processes including, but not limited to, clearly
describing the statistical tests performed and the
purpose/outcome, coding of data linked to each RQ, the
software used (e.g., SPSS, Qualtrics).
For Qualitative Studies: Does not clearly identify the
coding process of data linked to RQs; does not clearly
describe the transcription of data, the software used for
textual analysis (e.g., Nivo, DeDoose), and does not justify
manual analysis by researcher. There is missing or unclear
explanation of the use of a member check to validate data
collected.
Provides a description of the data collected and the
processes used in gathering the data. Explains alignment
between the data collected and the research questions
and/or hypotheses of the study.
For Quantitative Studies: Includes the data analysis
processes including, but not limited to, describing the
statistical tests performed and the purpose/outcome,
coding of data linked to each RQ, the software used (e.g.,
SPSS, Qualtrics).
For Qualitative Studies: Identifies the coding process of
data linked to RQs. Describes the transcription of data,
the software used for textual analysis (e.g., Nivo,
DeDoose), and describes manual analysis by researcher.
Describes the use of a member check to validate data
collected.
Assumptions/Limitations/
Delimitations
Does not clearly outline the
assumptions/limitations/delimitations (or has missing
components) inherent to the choice of method and design.
For Quantitative Studies: Does not include or lacks key
elements such as, but not limited to, threats to internal and
external validity, credibility, and generalizability.
For Qualitative Studies: Does not include or lacks key
elements such as, but not limited to threats to credibility,
trustworthiness, and transferability.
Outlines the assumptions/limitations/delimitations to the
choice of method and design.
For Quantitative Studies: Includes key elements such as,
but not limited to, threats to internal and external validity,
credibility, and generalizability.
For Qualitative Studies: Includes key elements such as,
but not limited to threats to credibility, trustworthiness,
and transferability.
Ethical Assurances Lacks discussion of compliance with the standards to
conduct research as appropriate to the proposed research
design and is not aligned to IRB requirements.
Describes compliance with the standards to conduct
research as appropriate to the proposed research design
and aligned to IRB requirements.
Summary Chapter does not conclude with a summary of key points
from the Chapter – elements are missing, incomplete,
and/new information is presented.
Chapter concludes with an organized summary of key
points discussed/presented in the Chapter.