- What are the three key characteristics of overt observations, and what is their common goal?
- There are four reasons that we want to ensure the quality of measurement:
o First, Interobserver consistency and agreement can reflect whether a behavior is w___________ d______________.
o Second, wait want to behavior to be well defined in relation to the i_________________.
o Third, error variability makes it difficult to make decisions about went to change i__________________________ phases.
o Fourth, error variability interferes with d___________ e_______________. - Define the differences between accuracy and agreement. Give an example of this.
- How often should an investigator conduct checks on interobserver agreement? What are the guidelines to consider the frequency of checks?
- Define each method of calculating agreement (i.e., frequency ratio and point-by-point agreement ratio). Then provide:
o The formula for calculating this method of IOA
o Provide problems and considerations
o Provide an example
Use the chart below for question # 5:
Agreement Formula Problem and Considerations Example
Frequency Ratio
Point-by-point Agreement Ratio
- Explain how the base rate of behavior relates to the overall level of agreement.
. - What does Kazdin say about chance agreement?
- How can plotting agreement data help us to determine whether our method of calculating IOA is valid?
- Use the table below to 1) name each of the sources of artifact and bias discussed by Kazdin; 2) provide an original example of each; and 3) describe what can be done to minimize this source of artifact and bias.
Source of artifact and bias Example How to minimize this problem
- What are the considerations for determining an acceptable level of agreement when calculating IOA? Why is it not advisable to have a blanket rule of 80-90% agreement as acceptable?
- What is the overall purpose of assessing agreement among observers?
Sample Solution