1. What are the three key characteristics of overt observations, and what is their common goal?
2. There are four reasons that we want to ensure the quality of measurement:
o First, Interobserver consistency and agreement can reflect whether a behavior is w___________ d______________.
o Second, wait want to behavior to be well defined in relation to the i_________________.
o Third, error variability makes it difficult to make decisions about went to change i__________________________ phases.
o Fourth, error variability interferes with d___________ e_______________.
3. Define the differences between accuracy and agreement. Give an example of this.
4. How often should an investigator conduct checks on interobserver agreement? What are the guidelines to consider the frequency of checks?
5. Define each method of calculating agreement (i.e., frequency ratio and point-by-point agreement ratio). Then provide:
o The formula for calculating this method of IOA
o Provide problems and considerations
o Provide an example
Use the chart below for question # 5:
Agreement Formula Problem and Considerations Example
Frequency Ratio
Point-by-point Agreement Ratio
6. Explain how the base rate of behavior relates to the overall level of agreement.
.
7. What does Kazdin say about chance agreement?
8. How can plotting agreement data help us to determine whether our method of calculating IOA is valid?
9. Use the table below to 1) name each of the sources of artifact and bias discussed by Kazdin; 2) provide an original example of each; and 3) describe what can be done to minimize this source of artifact and bias.
Source of artifact and bias Example How to minimize this problem
10. What are the considerations for determining an acceptable level of agreement when calculating IOA? Why is it not advisable to have a blanket rule of 80-90% agreement as acceptable?
11. What is the overall purpose of assessing agreement among observers?
Repp, A. C., Deitz, D. E. D., Boles, S. M., Deitz, S. M., & Repp, C. F. (1976). Differences among methods for calculating interobserver agreement. Journal of Applied Behavior Analysis, 9(1), 109-113.
1. What was the purpose of this study?
2. What was the general procedure used in this study?
3. Name each method of calculation used in the study and describe the meaning of each (there are three total).
4. The same data were used to calculate IOA using three different methods. Explain how each of these methods of IOA differed using the graph on the top of page 111.
5. Overall, what does this article tell us about the selection of a method to calculate IOA?