QuantMethodsSession2.pdf

    51Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Pixabay.com

    Nasser Alajmi

    52Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    • Research is usually interested in relationships between different phenomena or concepts.

    – E.g., Intelligence, organizational performance, customer satisfaction, strategic position, etc.

    – Some phenomena are directly observable and can thus be (rather) easily measured (e.g., age, industry affiliation, revenues, etc.)

    – Other phenomena are much harder to capture.

    • Construct validity is an important concern in quantitative research

    Nasser Alajmi

    53Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Concept measurement depends on the method of data collection:

    • Archival research: Use of secondary data obtained from firms, public sources, databases, etc.

    • Survey studies: Collection of data through questionnaires

    • Experiments: Collection of data through laboratory replication of a real-world situation

    Nasser Alajmi

    54Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Examples for archival proxies:

    • “To proxy and control for the firm’s innovative capability or capacity, we include the variable cumulative number of patents, which is the total number of patents that the firm has generated since the year of its establishment until the year before the observation year.” (Jia et al. 2019, AMJ)

    • “As a proxy for project complexity, I used the total number of electrical engineering hours on a project”. (Young-Hyman, 2017, ASQ)

    Nasser Alajmi

    55Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Using surveys to measure concepts

     Example

    nasseralajmi
    Sticky Note
    Ordinar values are problomatic cause we can't put them on a regression
    Nasser Alajmi

    56Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    • Surveys often rely on so-called Likert scales to measure the extent to which the respondent agrees or disagrees with a statement.

    – Likert-scales can have different degrees of detail, but most common are 7-point scales.

    – End-points represent the most extreme positions on a question, e.g.:

    • 1 = definitely false; 7 = definitely true; 1 = strongly disagree, 7 = strongly agree, etc.

    • The different questions represent “items“. The construct (representing the concept of interest) is then a combination of these items.

    – Items either taken from prior research or newly developed scale

    – # of items: balance length of questionnaire with measurement quality; at least 3 items for a (reflective) construct

    nasseralajmi
    Highlight
    nasseralajmi
    Sticky Note
    We do this cause it is most likely already proven
    nasseralajmi
    Sticky Note
    Why more questions are better?- Help to find if people who answer actually answer it well.- It gives a good confidence at the end
    Nasser Alajmi

    57Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Compare the two regressions:

    Manifest variables (items) x1-x5:

    y = ß0 + ß1* x1 + ß2 * x2 + ß3 * x3 + ß4 * x4 + ß5 * x5 + u

    One latent construct z1:

    y = ß0 + ß1* z1 + u

    nasseralajmi
    Sticky Note
    The more valuable we have , the bigger the regression becomes and this is problematic
    Nasser Alajmi

    58Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    There are two types of multi-item (latent) constructs:

    • Reflective: Items represent manifestations of the construct

    • Formative: Items represent elements (components) of the construct

    This distinction is important, because construct validity and reliability of these constructs must be assessed differently.

    nasseralajmi
    Sticky Note
    We can do statistical relation here
    Nasser Alajmi

    59Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Monitoring difficulty

    It is not possible to supervise the salesperson’s activities closely.

    It is difficult for us to evaluate how much effort this salesperson really puts into her/his job.

    It is relatively easy for this salesperson to turn in falsified sales call reports.

    Our evaluation of this salesperson cannot be based on his/her activity and sales call reports.

    Salesperson’s ability

    This salesperson has a high degree of competence in tailoring his/her sales approach to the specific situation on hand.

    This salesperson has been very creative in designing relevant solutions to customers’ problems.

    This salesperson is a skilled and persuasive negotiator.

    This salesperson is capable of closing a deal in a tough selling situation.

    This salesperson is able to learn from past experiences and adapt them to current circumstances.

    This salesperson is skilled in extracting the unique problems faced by and the requirements of his/her customers.

    Lo et al. (2011, JMR)

    Nasser Alajmi

    60Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Construct validity (Do we really measure what we seek to measure?) and reliability (Do we measure reliably?) are a frequent concern in survey research.

    Construct validity and reliability should be approached from a judgmental and a technical perspective:

    • Judgmental (for both reflective and formative constructs):

    – Do the questions really reflect what I think the phenomenon is about? (often referred to as “face validity”)

    Nasser Alajmi

    61Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    • Technical (for reflective constructs only):

    – Internal consistency and convergent validity: do the items converge on the construct?

    – Discriminant validity: are the constructs sufficiently different from each other?

    Nasser Alajmi

    62Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Reflective constructs Formative

    constructs

    (composite

    constructs)

    Face validity Items should all reflect the underlying construct (in a similar way)

    Items should together add up to the construct (in a comprehensive way)

    Internal consistency & convergent validity

    Factor analysis:• High factor loadings of each item• High Average Variance Extracted

    (AVE)• High composite reliability• Good goodness-of fit criteria

    Discriminant validity Factor analysis:• Small cross-loadings• Square root of AVE >bivariate

    correlations between constructs (Fornell-Larcker Test)

    Nasser Alajmi

    63Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    The basis for assessing construct validity and reliability of reflective

    constructs is typically factor analysis.

    • Factor analysis:

    – Identifies (or confirms) the relationship between items and underlying constructs (factors)

    – Assumes that the correlation between items can be explained by underlying factors (of which the items are manifestations)

    – Exploratory factor analysis (EFA): no ex ante specification of factor structure: see which factors emerge from the data

    – Confirmatory factor analysis (CFA): factors and their relationships to items are specified a priori and then tested. Confirm if items load as they should.

    • Different uses in studies:

    – Only EFA or only CFA

    – First an EFA, then a CFA

    nasseralajmi
    Sticky Note
    Hypothetical construct
    nasseralajmi
    Highlight
    nasseralajmi
    Highlight
    nasseralajmi
    Sticky Note
    The automatic one!
    Nasser Alajmi

    64Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Example

    We are interested in 4 (reflective) constructs and, to this end, collected 16 variables through a survey.

    Our hypothesized factor structure is as follows:

    Factor1: v2 + v9 + v13 + v16 + v17Factor2: v6 + v7 + v8 + v11Factor3: v1 + v3 + v5 + v20Factor4: v4 + v12 + v14

    Nasser Alajmi

    65Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    v1 v2 v3 v4 v5 v6 v7 v8 v9 v11 v12 v13 v14 v16 v17 v20v1 1,00v2 -0,05 1,00v3 0,44 0,08 1,00v4 0,00 -0,07 0,10 1,00v5 0,41 0,00 0,40 0,13 1,00v6 0,02 0,02 0,07 0,02 0,19 1,00v7 0,01 -0,09 0,00 0,13 0,08 0,34 1,00v8 0,05 -0,02 0,03 0,02 0,09 0,31 0,37 1,00v9 -0,05 0,25 -0,06 0,03 -0,03 -0,12 -0,06 -0,04 1,00v11 0,04 -0,10 0,08 0,04 0,16 0,47 0,48 0,37 -0,10 1,00v12 0,11 0,06 0,08 0,29 0,17 0,12 0,03 0,06 0,03 0,03 1,00v13 -0,13 0,31 -0,03 0,05 0,05 -0,08 0,02 -0,09 0,32 -0,08 0,08 1,00v14 0,02 -0,04 0,04 0,34 0,11 -0,05 0,00 0,02 0,11 -0,07 0,25 0,10 1,00v16 -0,05 0,28 0,03 0,08 0,00 -0,13 0,00 -0,04 0,36 -0,15 0,08 0,38 0,03 1,00v17 -0,01 0,28 0,01 -0,04 -0,01 -0,14 -0,06 -0,07 0,36 -0,18 0,04 0,33 -0,01 0,29 1,00v20 0,35 0,01 0,40 0,04 0,36 0,00 0,09 -0,06 -0,05 0,00 0,10 0,00 0,03 0,03 0,07 1,00

    Let‘s first look at the correlations between the variables

    nasseralajmi
    Sticky Note
    Correlation means when I answer the survey question similarly
    nasseralajmi
    Sticky Note
    + means I have similar answersExp. first question 5 then second is 5- means I have opposite answersExp. first question 5 then second 1
    Nasser Alajmi

    66Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Fa

    Fb

    x1

    x2

    x3

    x4

    Latent factors Manifest variables

    a1

    a2

    b1

    b4

    X1 = a1*Fa + b1*Fb + u1

    X2 = a2*Fa + b2*Fb + u2

    etc

    All relationships possible  Exploratory factor analysis:

    a1, a2, b1, etc.: factor loadings

    Must specify either:- Number of factors (in line with your theory)- Minimum explanatory value of each factor (typically:

    Eigenvalue > 1).

    Two main “extraction methods“ for factors:- Principal component analysis: factors try to explain

    entire variance of variables- Principal axis factoring: factors explain only shared

    variance of variables

    EFA: finding a good factor structure

    Nasser Alajmi

    67Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Illustration of factor structure

    Factor 1

    Factor 2

    1

    0.5

    – 0.5

    – 0.5 10.5

    – 1

    – 1

    Variable x1 = 0.6 * F1 + 0.6 * F2 + u

    Nasser Alajmi

    68Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    SPSS: Analyze  Dimension Reduction  Factor

    Nasser Alajmi

    69Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS

    nasseralajmi
    Highlight
    Nasser Alajmi

    70Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS

    nasseralajmi
    Highlight
    nasseralajmi
    Sticky Note
    Dimension reduction
    Nasser Alajmi

    71Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    • Eigenvalues: an indicator for how much variance in the variables the factor explains

    • E.g., when Eigenvalue of each Factor = 1  each Factor explains the same amount of variance

    • Mathematically: the sum of the squared loadings across all variables for a given factor (see next page)

    • Variance Extracted: how much % of the shared variance in all variables is explained by the factor.

    • E.g., when Eigenvalue (Factor 1) = 1, and 5 Factors  Factor 1 explains 20% of variance

    • Factor loadings for each variable-factor pair: correlations

    Output from EFA

    Nasser Alajmi

    72Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS

    Factor Matrixa

    Factor1 2 3 4

    v1 0,336 0,419 -0,350 -0,130v2 -0,281 0,292 0,207 -0,220v3 0,319 0,508 -0,280 -0,133v4 0,120 0,259 0,122 0,559v5 0,388 0,499 -0,135 -0,036v6 0,497 -0,018 0,330 -0,103v7 0,444 0,016 0,438 -0,032v8 0,387 -0,026 0,338 -0,049v9 -0,389 0,304 0,280 -0,044v11 0,622 -0,070 0,424 -0,126v12 0,107 0,303 0,100 0,308v13 -0,370 0,364 0,345 -0,048v14 -0,002 0,244 0,049 0,515v16 -0,363 0,380 0,270 -0,066v17 -0,388 0,342 0,175 -0,180v20 0,226 0,447 -0,257 -0,118Extraction Method: Principal Axis Factoring.a. 4 factors extracted. 12 iterations required.

    nasseralajmi
    Highlight
    nasseralajmi
    Sticky Note
    The correlations between each factor
    Nasser Alajmi

    73Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Factor analysis first provides an “unrotated” solution.

    • First factor is defined in such a way that it explains most of the shared variance of all variables  almost any variable will load on this factor

    • Residual variance to be explained then by second, third, etc. factor

    Rotation changes the factor structure such that variance is explained more equally among factors.

    • The most frequently used method is “Varimax“, which defines factors in such a way that they have high or low loadings, but not medium ones.

    Unrotated and rotated factor structure

    nasseralajmi
    Highlight
    nasseralajmi
    Highlight
    Nasser Alajmi

    74Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Unrotated and rotated factor structure

    Hair et al., p. 111

    Nasser Alajmi

    75Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS

    Nasser Alajmi

    76Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS:

    rotated solution

    Rotated Factor Matrix

    Factor1 2 3 4

    v1 -0,002 -0,089 0,648 0,012v2 -0,019 0,496 0,045 -0,083v3 0,033 0,009 0,672 0,051v4 0,065 -0,007 0,040 0,635v5 0,166 0,013 0,604 0,165v6 0,593 -0,095 0,078 -0,007v7 0,619 -0,008 0,010 0,083v8 0,514 -0,047 0,006 0,031v9 -0,073 0,553 -0,073 0,084v11 0,750 -0,141 0,058 -0,022v12 0,082 0,083 0,140 0,418v13 -0,016 0,613 -0,051 0,114v14 -0,063 0,027 0,020 0,567v16 -0,060 0,584 0,003 0,093v17 -0,123 0,559 0,039 -0,042v20 -0,016 0,034 0,572 0,038Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.a. Rotation converged in 5 iterations.

    Nasser Alajmi

    77Quant. Methods 1 (M. Messner)

    Exploratory factor analysis

    Output from SPSS:

    rotated solution

    Rotated Factor Matrix

    Factor1 2 3 4

    v1 -0,002 -0,089 0,648 0,012v2 -0,019 0,496 0,045 -0,083v3 0,033 0,009 0,672 0,051v4 0,065 -0,007 0,040 0,635v5 0,166 0,013 0,604 0,165v6 0,593 -0,095 0,078 -0,007v7 0,619 -0,008 0,010 0,083v8 0,514 -0,047 0,006 0,031v9 -0,073 0,553 -0,073 0,084v11 0,750 -0,141 0,058 -0,022v12 0,082 0,083 0,140 0,418v13 -0,016 0,613 -0,051 0,114v14 -0,063 0,027 0,020 0,567v16 -0,060 0,584 0,003 0,093v17 -0,123 0,559 0,039 -0,042v20 -0,016 0,034 0,572 0,038Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.a. Rotation converged in 5 iterations.

    Guide:

    Loadings > | 0.5 | or | 0.6 |

    Cross-loadings < | 0.4 | or | 0.5 |

    Nasser Alajmi

    79Quant. Methods 1 (M. Messner)

    Confirmatory factor analysis

    Fa

    Fb

    x1

    x2

    x3

    x4

    Latent factors Manifest variables

    a1

    a2

    b3

    b4

    X1 = a1*Fa + u1

    X2 = a2*Fa + u2

    X3 = b3*Fb + u3

    X4 = b4*Fb + u4

    Relationship specified  Confirmatory factor analysis:

    a1, a2, b3, etc.: factor loadings

    CFA tests how well the specified model fits the data

    Nasser Alajmi

    80Quant. Methods 1 (M. Messner)

    Confirmatory factor analysis

    CFA produces one hypothesis test and various “fit indicators“

    Indicator Description “good“ fit

    GFI Goodness of fit GFI >= 0.95

    RMSEA Root mean square error of approximation

    RMSEA < 0.08

    CFI Comparative fit index CFI >= 0.90

    SRMR (Standardized) root meansquare residual

    (S)RMR < 0.08

    Hypothesis test (chi-squared):

    H0: “The model fits perfectly.”P-value > 0.05 so as not to reject this H0

    See also: https://www.cscu.cornell.edu/news/Handouts/SEM_fit.pdf

    Typically these indicators are calculated for each factor separately and for the entire model as a whole

    Nasser Alajmi

    81Quant. Methods 1 (M. Messner)

    Confirmatory factor analysis

    Confirmatory factor analysis output

    Model Test User Model:

    Test statistic 95.847

    Degrees of freedom 98

    P-value (Chi-square) 0.543

    cfi rmsea gfi rmr1.000 0.000 0.954 0.113

    Nasser Alajmi

    82Quant. Methods 1 (M. Messner)

    Confirmatory factor analysis

    Confirmatory factor analysis output

    Latent Variables:Estimate Std.Err z-value P(>|z|) Std.lv Std.all

    factor1 =~ v2 1.000 0.771 0.474v9 1.216 0.229 5.302 0.000 0.938 0.579v13 1.259 0.233 5.395 0.000 0.971 0.605v16 1.361 0.254 5.363 0.000 1.050 0.596v17 1.123 0.215 5.219 0.000 0.866 0.559

    Nasser Alajmi

    83Quant. Methods 1 (M. Messner)

    Additional validity indicators

    Additional checks for convergent and discriminant validity

    Convergent validity (for each factor separately):

    • High Average Variance Extracted (guide: > 0.5)• How much of the variance in the items (averaged across all items of the given

    factor) is explained by the given latent factor• E.g. AVE = 0.4  Factor explains 40% of the variation in item values

    • High Composite reliability (guide: > 0.7)• Compares the variance extracted to the error variance (not explained).

    • [Cronbach’s alpha / Coefficient alpha (guide: > 0.6 or 0.7)]

    Nasser Alajmi

    84Quant. Methods 1 (M. Messner)

    Additional validity indicators

    AVE and Composite Reliability

    Squared loading = % variance of this variable explained by the factor

    Sum of squared loadings = 0.5792 + 0.6052 + 0.5962 + 0.5592 = 1.36

    Average Variance extracted = 1.36 / 4 = 0.34

    factorloadings

    squaredloadings

    1- squared loadings

    0.579 0.335 0.6650.605 0.366 0.6340.596 0.355 0.6450.559 0.312 0.6882.339 1.369 2.631

    Composite reliability = 2.3392 / (2.3392 + 2.631) = 0.68

    nasseralajmi
    Highlight
    nasseralajmi
    Sticky Note
    Less than 50%, not good
    Nasser Alajmi

    85Quant. Methods 1 (M. Messner)

    Additional validity indicators

    Additional checks for convergent and discriminant validity

    Discriminant validity of factors:

    Is a given factor more strongly correlated with its own items than with other factors?

    Criterion: √ AVE > cross-correlation with other factors (Fornell-Larcker test)

    Why?

    • Remember: correlation2 = R2 = % of variance explained

    • AVE = average variance of items explained (by the given factor)

    nasseralajmi
    Sticky Note
    Are they different from each other
    Nasser Alajmi

    86Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    Typical procedure for technical reliability and construct validity (for reflective

    constructs)

    1. Exploratory factor analysis:

    – Number of factors with Eigenvalue > 1: in line with theory?

    – Sufficiently high factor loadings and low cross-loadings

    – Possibly delete items that do not load correctly (and re-do EFA)

    2. Confirmatory factor analysis:

    – Test hypothesized factor structure: Different Goodness of fit criteria

    3. Further criteria to ascertain convergent validity of factors: Composite Reliability, Average Variance Extracted

    4. Further criterion to ascertain discriminant validity of factors: √ AVE > cross-correlation with other factors (Fornell-Larcker test)

    Nasser Alajmi

    87Quant. Methods 1 (M. Messner)

    Example

    Ramani & Kumar (2008), Journal of Marketing

    “Composite reliability is an indicator of the shared variance among the set of observed variables used as indicators of a latent construct (Fornell and Larcker1981; Kandemir, Yaprak, and Cavusgil 2006). As Table 2, Panel A, shows,construct reliabilities for all the seven latent constructs ranged from .82 to .97, well above the recommended value. In addition, the coefficient alpha values were well above the threshold value of .7 that Nunnally (1978) recommends. The seven-factor CFA model exhibited a good fit with the data (χ2 = 253.95, d.f. = 131; CFI = .95; GFI = .82; TLI = .93; IFI = .95; and SRMR = .07). The standardized factor loadings ranged from .62 to greater than .90 and were statistically significant at the α = .95 level (see Table 2, Panel B). This provided the necessary evidence that all the constructs exhibited convergent validity. We examined discriminant validity using a procedure suggested by Fornell and Larcker (1981) and widely followed by other researchers (e.g., De Wulf, Odekerken- Schroder, and Iacobucci 2001; Kandemir, Yaprak, and Cavusgil 2006). We computed the average variance extracted by the indicators corresponding to each of the seven factors and compared it with the highest variance that each factor shared with the other factors in the model. The average variance extracted for each factor was always greater than the highest shared variance (see Table 2, Panel A).“

    Nasser Alajmi

    88Quant. Methods 1 (M. Messner)

    Example (cont’d)

    Ramani & Kumar (2008), Journal ofMarketing

    Nasser Alajmi

    89Quant. Methods 1 (M. Messner)

    Concepts and their measurement

    How to aggregate the item responses to a value of the construct?

    • When used in further regression analysis: • Add up the values from the items (“summated scale”) or average

    them• Use so-called “factor scores” (there are different ways to calculate

    such factor scores), which take into account the loadings

    • Example: “We construct our composite variables using factor scores. The correlations between our variables using factor scores and summated scales range between 0.98 and 0.99 for our four composite variables (not tabulated).” (Abernethy, Bouwens, & Kroos, 2017)

    • When part of a structural equations model:• Measurement model and structural model are estimated at the same

    time  “weights” used depend on structural model

    Nasser Alajmi

                                                                                                                                        Order Now