Clf alpha = 0

Examples of Psychometrics


research101chart_blogimgIn order for any scientific instrument to provide measurements that can be trusted, it must be both reliable and valid. These psychometrics are crucial for the interpretability and the generalizability of the constructs being measured.

Reliability is the degree to which an instrument consistently measures a construct - both across items (e.g., internal consistency, split-half reliability) and time points (e.g., test-retest reliability). One of the most common assessments of reliability is Cronbach’s Alpha, a statistical index of internal consistency that also provides an estimate of the ratio of true score to error in Classical Test Theory. A general rule of thumb is that solid scientific instruments should have a Cronbach’s Alpha of at least .7. There are exceptions to this rule in the case of brief measurements when breadth of content is of primary interest in recapturing a longer scale (see example here).

Below is an example of a reliability analysis for a Recreational Shopping scale. The analysis provides a summary of how the items within the scale perform together in measuring a person’s propensity for recreational shopping. In this example, the overall reliability statistic is .732. The analysis also elucidates the efficacy of each individual item by reporting information such as corrected item-total correlation and Cronbach’s Alpha if an item were deleted. As seen in the example below, we know that item #4 is a great item because it has a high item-total correlation (correlates strongly with the other items) and the overall reliability would drop significantly if the item were deleted from the scale. These, and other metrics all go into understanding the makings of a reliable survey.

As demonstrated in the video linked above, a measure can be reliable without being valid but it cannot be valid without being reliable. Building on reliability, validity is an index of whether or not a particular instrument measures what it purports to measure. For example, let’s say a researcher gave Samantha a paper-and-pencil survey of Extraversion. How would the researcher know that the computed score on that survey actually reflected Samantha’s true level of Extraversion?

There are several different forms of validity. Some of the most commonly assessed forms of validity include content validity, construct validity, and criterion validity. Content validity is an assessment of how well the breadth of the construct has been assessed. For example, have all the elements of Extraversion been captured in the survey (e.g., gregarious, outgoing, active)?

Construct validity is a measure of how well an instrument measures an operationalized or latent construct. Two important sub-components of construct validity include convergent (the degree to which two instruments which measure the same construct are correlated; generally the higher the better) and discriminant validity (the degree to which two unrelated measures are correlated; generally the lower the better). The validity analysis reported below represents the convergent validity of TipTap Lab’s Image Selection Task (IST) of Recreational Shopping with the original paper-and-pencil Recreational Shopping scale (a scale that has been previously scientifically validated and cited in numerous research studies).

This statistic can be interpreted like any correlation (the closer the number is to 1, the stronger the relationship). So, if the correlation is high (as we see below), convergence is strong. Convergent validity is a particularly important statistic at TipTap Labbecause we employ this methodology to convert long, paper-and-pencil measures (all previously validated in external research contexts) into short and engaging image based measurements. It is critical for us to recapture the psychometric properties of the original scales.

Lastly, criterion validity (including both predictive and concurrent validity) is an assessment of how well an instrument predicts known related behaviors or constructs. For instance, if Samantha scored high on the Extraversion scale, we know from previous research that she should be more likely (than an Introvert) to attend a party or talk to a stranger. In the course of our research, criterion validity is constantly being evaluated as more constructs and behavioral outcomes are being studied.

At TipTap Lab, we employ advanced psychometric techniques to build the most reliable and valid measurements possible. We are constantly iterating our process and improving our items as well as our methodology.



Share this article





Related Posts


Technical Recruitment process
Technical Recruitment process
Free online Aptitude training
Free online Aptitude training
Example of numerical Reasoning
Example of numerical Reasoning

Latest Posts
Technology Recruitment Agencies
Technology Recruitment…
It s good to work in tech right now…
Catering Sales Manager interview questions
Catering Sales…
Most job interviews begin with a series…
Interview questions for Nursing
Interview questions…
1) Why do you want to become a Child/Adult/Mental…
Technology Recruitment
Technology Recruitment
Evangelists and Entrepreneurs On 17th…
Psychometric Evaluations
Psychometric…
Portable Dew Point Meter is designed…
Search
Featured posts
  • Technical Recruitment process
  • Free online Aptitude training
  • Example of numerical Reasoning
  • Technical interview questions for Mechanical design engineers
  • Retail sales interview questions
  • Technology Recruitment Agencies
  • Recruitment Market Trends
  • Customer service supervisor interview questions and answers
  • Engineering internship Interview questions
Copyright © 2020 l www.floydfairnessfund.org. All rights reserved.