Reliability and Validity in Measurement Procedures

RELIABILITY & VALIDITY

• 1) Does the instrument or the test measure a variable consistently? and
• 2) Is the instrument or the test a true measure of the variable?

The first question is concerned with reliability while the second one is concerned with validity.

Definition

Reliability

A measurement procedure is said to be reliable if repeated measurements of the same individual under the same conditions produce identical (or nearly identical) values. Reliability is the stability or the consistency of measurement (ability of a measure to remain the same over time).

It is the degree to which an assessment tool produces stable and consistent results. • Reliability coefficient is a way of

Measurement of Reliability

confirming how accurate a test or measure. It is a measure of the accuracy of a test or measuring instrument obtained by measuring the same individuals twice and computing the correlation of the two sets of measures. It ranges between 0.00 to 1.00; where 0.00 means no reliability and 1.00 means perfect reliability.

A. Internal Reliability

1. Parallel-forms Reliability (equivalent forms reliability) is used to assess the consistency of the results of two tests constructed in the same way from the same content domain.

Types of Reliability

1. Split Half Reliability
   • The split-half method assesses the internal consistency of a test, such as psychometric tests and questionnaires. There, it measures the extent to which all parts of the test contribute equally to what is being measured. This is done by comparing the results of one half of a test with the results from the other half.
     Get to Know The Price Estimate For Your Paper

     Topic

     Deadline: 10 days left

     Number of pages

     Email Invalid email

     By clicking “Check Writers’ Offers”, you agree to our terms of service and privacy policy. We’ll occasionally send you promo and account related email

     "You must agree to out terms of services and privacy policy"

     Write my paper

     You won’t be charged yet!

     A test can be split in half in several ways, e.g. first half and second half, or by odd and even numbers. If the two halves of the test provide similar results this would suggest that the test has internal reliability

2. B. External Reliability
   1. Test-retest (Stability) Reliability
      • It is used to assess the consistency of a measure from one time to another. It examines the consistency of the results of repeated test administration. It is a measure of reliability obtained by administering the same test twice over a period of time to a group of individuals (a measure or test is repeated on the same subjects at a future date). The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for stability over time.
3. 2. Rater Reliability
   • It includes
     • inter-rater reliability (reliability of data collection between two or more raters) and
     • intra-rater reliability (reliability of data collection from one individual over two or more trials).

Inter-rater reliability

It is the extent to which two or more individuals (raters) agree. It is used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon. It assesses the consistency of how a measuring system is implemented. The reliability depends upon the raters to be consistent in their evaluation of behaviors or skills (ability of two or more individuals to be consistent). Training, education and monitoring skills can enhance inter-rater reliability)

Inter-rater Reliability Example:

when two or more physical therapists apply muscle testing by the use of grading muscle testing. If one researcher gives a grade one to a patient, while another researcher gives grade three to the same patient, obviously the inter-rater reliability would be inconsistent

Intra-rater reliability

It refers to the degree of agreement among multiple repetitions of a diagnostic test performed by a single rater. It is the consistency of grading by a single rater.

Definition

Validity is the measuring of what the subject claimed to measure.

Example: Palpation Technique

Assuming that the cardiac catheterization measure is the more accurate method to measure heart rate, the amount of agreement between it and palpation technique constitutes the degree of validity of the palpation technique.

Types of Validity

1. whole studies (often called inference validity) and
2. the measurement of individual variables (often called construct validity)

Types of Error in Measurement

All experimental uncertainty is due to either random errors or systematic errors.

1. Random Errors

They are statistical fluctuations (in either direction) in the measured data due to the precision limitations of the measurement device. Random errors usually result from the experimenter's inability to take the same measurement in exactly the same way to get exact the same number which may be due to fatigue or inattention.

It affect reliability

Examples of causes of random errors are:

• electronic noise in the circuit of an electrical instrument,
• irregular changes in the heat loss rate from a solar collector due to changes in the wind.

2. Systematic Errors

They are reproducible inaccuracies that are consistently in the same direction. Systematic errors are often due to a problem which persists throughout the entire experiment. Systematic errors are constant and, therefore, do not present a problem for reliability. Systematic errors are primarily a concern of validity, because, although they are consistent, test values are not true representations of the quantity being measured.