Chapter_3_Evaluation_Rubric_v2.pdf

    Chapter 3 – Evaluation Rubric

    Criteria Does Not Meet 0.01 points Meets/NA 1 point

    Introductory Remarks The section is missing; or some topic areas are not included

    in the Introduction or are not explained clearly.

    The chapter outline is not provided and/or is unclear.

    The reader is adequately oriented to the topic areas

    covered. An outline of the logical flow of the chapter is

    presented.

    All major themes/concepts are introduced.

    Research Methodology and

    Design

    There is a lack of alignment among the chosen research

    method and design and the study’s problem, purpose, and

    research questions.

    There is a lack of justification and alternate choices for

    methods.

    For Qualitative Studies: Lacks clear discussion of the

    study phenomenon, boundaries of case(s), and/or

    constructs explored.

    Describes how the research method and design are aligned

    with the study problem, purpose, and research questions.

    Uses scholarly support to describe how the design choice

    is consistent with the research method, and alternate

    choices are discussed.

    For Qualitative Studies: Describes the study

    phenomenon, boundaries of case(s) and/or constructs

    explored.

    Population and Sample Lacks a description of the sample, demographics, and the

    representation of the sample to the broader population.

    There is little to no description of the inclusion/exclusion

    criteria used to select the participants (sample) of the study.

    For Quantitative Studies: A power analysis is not

    described and appropriately cited.

    Provides a description of the target population and the

    relation to the larger population.

    Inclusion/exclusion criteria for selecting participants

    (sample) of the study are noted.

    For Quantitative Studies: Power analysis is

    described and appropriately cited.

    Materials/

    Instrumentation

    Lacks a description of the instruments associated with the

    chosen research method and design used. Details missing

    regarding instrument origin, reliability, and validity.

    For Quantitative Studies: (e.g., tests or surveys). Lacks

    explanation of any permission needed to use the

    instrument(s) and cites properly. Instrument permissions are

    missing in appendices

    For Qualitative Studies: (e.g., observation

    checklists/protocols, interview or focus group discussion

    Handbooks). Did not clearly explain the process for

    conducting an expert review of instruments (e.g., provides

    justification of reviewers being credible – reviewers may

    include, but not limited to NCU dissertation team members,

    professional colleagues, peers, or non-research participants

    representative of the greater population); and/or did not

    clearly explain use of a field test if practicing the

    administration of the instruments is warranted.

    For Pilot Study: Does not clearly explain the procedure for

    conducting a pilot study (did not conduct pilot) if using a

    self-created instrument (e.g., survey questionnaire); does not

    include explanation of a field test if practicing the

    administration of the instruments is warranted.

    Provides a description of the instruments associated with

    the chosen research method and design used. Includes

    information regarding instrument origin, reliability, and

    validity.

    For Quantitative Studies: (e.g., tests or surveys).

    Includes any permission needed to use the instrument(s)

    and cites properly.

    For Qualitative Studies: (e.g., observation

    checklists/protocols, interview or focus group discussion

    Handbooks). Describes process for conducting an expert

    review of instruments (e.g., provides justification of

    reviewers being credible – reviewers may include, but not

    limited to NCU dissertation team members, professional

    colleagues, peers, or non-research participants

    representative of the greater population); describes use of

    a field test if practicing the administration of the

    instruments is warranted.

    For Pilot Study: Explains the procedure for conducting a

    pilot study (requires IRB approval for pilot) if using a

    self-created instrument (e.g., survey questionnaire);

    explains use of a field test if practicing the administration

    of the instruments is warranted.

    Operational Definitions

    of Variables

    (Quantitative Studies

    Only)

    Discussion of the study variables examined is lacking

    information and/or is unclear.

    Describes study variables in terms of being measurable

    and/or observable.

    (Reviewer – mark Meets/NA for Qualitative studies)

    Procedures Procedures are not clear or replicable. Steps are missing;

    recruitment, selection, and informed consent are not

    established. IRB ethical practices are missing or unclear.

    Describes the procedures to conduct the study in enough

    detail to practically replicate the study, including

    participant recruitment and notification, and informed

    consent. IRB ethical practices are noted.

    Data Collection and

    Analysis

    Does not clearly provide a description of the data and the

    processes to collect data. Lack of alignment between the

    data collected and the research questions and/or hypotheses

    of the study.

    For Quantitative Studies: Does not clearly provide the data

    analysis processes including, but not limited to, clearly

    describing the statistical tests performed and the

    purpose/outcome, coding of data linked to each RQ, the

    software used (e.g., SPSS, Qualtrics).

    For Qualitative Studies: Does not clearly identify the

    coding process of data linked to RQs; does not clearly

    describe the transcription of data, the software used for

    textual analysis (e.g., Nivo, DeDoose), and does not justify

    manual analysis by researcher. There is missing or unclear

    explanation of the use of a member check to validate data

    collected.

    Provides a description of the data collected and the

    processes used in gathering the data. Explains alignment

    between the data collected and the research questions

    and/or hypotheses of the study.

    For Quantitative Studies: Includes the data analysis

    processes including, but not limited to, describing the

    statistical tests performed and the purpose/outcome,

    coding of data linked to each RQ, the software used (e.g.,

    SPSS, Qualtrics).

    For Qualitative Studies: Identifies the coding process of

    data linked to RQs. Describes the transcription of data,

    the software used for textual analysis (e.g., Nivo,

    DeDoose), and describes manual analysis by researcher.

    Describes the use of a member check to validate data

    collected.

    Assumptions/Limitations/

    Delimitations

    Does not clearly outline the

    assumptions/limitations/delimitations (or has missing

    components) inherent to the choice of method and design.

    For Quantitative Studies: Does not include or lacks key

    elements such as, but not limited to, threats to internal and

    external validity, credibility, and generalizability.

    For Qualitative Studies: Does not include or lacks key

    elements such as, but not limited to threats to credibility,

    trustworthiness, and transferability.

    Outlines the assumptions/limitations/delimitations to the

    choice of method and design.

    For Quantitative Studies: Includes key elements such as,

    but not limited to, threats to internal and external validity,

    credibility, and generalizability.

    For Qualitative Studies: Includes key elements such as,

    but not limited to threats to credibility, trustworthiness,

    and transferability.

    Ethical Assurances Lacks discussion of compliance with the standards to

    conduct research as appropriate to the proposed research

    design and is not aligned to IRB requirements.

    Describes compliance with the standards to conduct

    research as appropriate to the proposed research design

    and aligned to IRB requirements.

    Summary Chapter does not conclude with a summary of key points

    from the Chapter – elements are missing, incomplete,

    and/new information is presented.

    Chapter concludes with an organized summary of key

    points discussed/presented in the Chapter.

                                                                                                                                      Order Now