Chapter 3 – Evaluation Rubric

Criteria Does Not Meet 0.01 points Meets/NA 1 point

Introductory Remarks The section is missing; or some topic areas are not included

in the Introduction or are not explained clearly.

The chapter outline is not provided and/or is unclear.

The reader is adequately oriented to the topic areas

covered. An outline of the logical flow of the chapter is

presented.

All major themes/concepts are introduced.

Research Methodology and

Design

There is a lack of alignment among the chosen research

method and design and the study’s problem, purpose, and

research questions.

There is a lack of justification and alternate choices for

methods.

For Qualitative Studies: Lacks clear discussion of the

study phenomenon, boundaries of case(s), and/or

constructs explored.

Describes how the research method and design are aligned

with the study problem, purpose, and research questions.

Uses scholarly support to describe how the design choice

is consistent with the research method, and alternate

choices are discussed.

For Qualitative Studies: Describes the study

phenomenon, boundaries of case(s) and/or constructs

explored.

Population and Sample Lacks a description of the sample, demographics, and the

representation of the sample to the broader population.

There is little to no description of the inclusion/exclusion

criteria used to select the participants (sample) of the study.

For Quantitative Studies: A power analysis is not

described and appropriately cited.

Provides a description of the target population and the

relation to the larger population.

Inclusion/exclusion criteria for selecting participants

(sample) of the study are noted.

For Quantitative Studies: Power analysis is

described and appropriately cited.

Materials/

Instrumentation

Lacks a description of the instruments associated with the

chosen research method and design used. Details missing

regarding instrument origin, reliability, and validity.

For Quantitative Studies: (e.g., tests or surveys). Lacks

explanation of any permission needed to use the

instrument(s) and cites properly. Instrument permissions are

missing in appendices

For Qualitative Studies: (e.g., observation

checklists/protocols, interview or focus group discussion

Handbooks). Did not clearly explain the process for

conducting an expert review of instruments (e.g., provides

justification of reviewers being credible – reviewers may

include, but not limited to NCU dissertation team members,

professional colleagues, peers, or non-research participants

representative of the greater population); and/or did not

clearly explain use of a field test if practicing the

administration of the instruments is warranted.

For Pilot Study: Does not clearly explain the procedure for

conducting a pilot study (did not conduct pilot) if using a

self-created instrument (e.g., survey questionnaire); does not

include explanation of a field test if practicing the

administration of the instruments is warranted.

Provides a description of the instruments associated with

the chosen research method and design used. Includes

information regarding instrument origin, reliability, and

validity.

For Quantitative Studies: (e.g., tests or surveys).

Includes any permission needed to use the instrument(s)

and cites properly.

For Qualitative Studies: (e.g., observation

checklists/protocols, interview or focus group discussion

Handbooks). Describes process for conducting an expert

review of instruments (e.g., provides justification of

reviewers being credible – reviewers may include, but not

limited to NCU dissertation team members, professional

colleagues, peers, or non-research participants

representative of the greater population); describes use of

a field test if practicing the administration of the

instruments is warranted.

For Pilot Study: Explains the procedure for conducting a

pilot study (requires IRB approval for pilot) if using a

self-created instrument (e.g., survey questionnaire);

explains use of a field test if practicing the administration

of the instruments is warranted.

Operational Definitions

of Variables

(Quantitative Studies

Only)

Discussion of the study variables examined is lacking

information and/or is unclear.

Describes study variables in terms of being measurable

and/or observable.

(Reviewer – mark Meets/NA for Qualitative studies)

Procedures Procedures are not clear or replicable. Steps are missing;

recruitment, selection, and informed consent are not

established. IRB ethical practices are missing or unclear.

Describes the procedures to conduct the study in enough

detail to practically replicate the study, including

participant recruitment and notification, and informed

consent. IRB ethical practices are noted.

Data Collection and

Analysis

Does not clearly provide a description of the data and the

processes to collect data. Lack of alignment between the

data collected and the research questions and/or hypotheses

of the study.

For Quantitative Studies: Does not clearly provide the data

analysis processes including, but not limited to, clearly

describing the statistical tests performed and the

purpose/outcome, coding of data linked to each RQ, the

software used (e.g., SPSS, Qualtrics).

For Qualitative Studies: Does not clearly identify the

coding process of data linked to RQs; does not clearly

describe the transcription of data, the software used for

textual analysis (e.g., Nivo, DeDoose), and does not justify

manual analysis by researcher. There is missing or unclear

explanation of the use of a member check to validate data

collected.

Provides a description of the data collected and the

processes used in gathering the data. Explains alignment

between the data collected and the research questions

and/or hypotheses of the study.

For Quantitative Studies: Includes the data analysis

processes including, but not limited to, describing the

statistical tests performed and the purpose/outcome,

coding of data linked to each RQ, the software used (e.g.,

SPSS, Qualtrics).

For Qualitative Studies: Identifies the coding process of

data linked to RQs. Describes the transcription of data,

the software used for textual analysis (e.g., Nivo,

DeDoose), and describes manual analysis by researcher.

Describes the use of a member check to validate data

collected.

Assumptions/Limitations/

Delimitations

Does not clearly outline the

assumptions/limitations/delimitations (or has missing

components) inherent to the choice of method and design.

For Quantitative Studies: Does not include or lacks key

elements such as, but not limited to, threats to internal and

external validity, credibility, and generalizability.

For Qualitative Studies: Does not include or lacks key

elements such as, but not limited to threats to credibility,

trustworthiness, and transferability.

Outlines the assumptions/limitations/delimitations to the

choice of method and design.

For Quantitative Studies: Includes key elements such as,

but not limited to, threats to internal and external validity,

credibility, and generalizability.

For Qualitative Studies: Includes key elements such as,

but not limited to threats to credibility, trustworthiness,

and transferability.

Ethical Assurances Lacks discussion of compliance with the standards to

conduct research as appropriate to the proposed research

design and is not aligned to IRB requirements.

Describes compliance with the standards to conduct

research as appropriate to the proposed research design

and aligned to IRB requirements.

Summary Chapter does not conclude with a summary of key points

from the Chapter – elements are missing, incomplete,

and/new information is presented.

Chapter concludes with an organized summary of key

points discussed/presented in the Chapter.