Estimating false alarms and missed events from interobserver agreement: A rationale.

In the analysis of temporal, sequential, or contingent relations among events within sessions, the observers' or coders' false-alarm rate and missed-event rate are more useful than coefficients of interobserver agreement as indices of reliability. False-alarm and missed-event probabilities can be estimated from comparisons of two codings, either in the fixed-unit situation or when behavior has been coded in continuous time. Complex problems, such as those that arise when one computes rates in the context of two or more mutually exclusive continuous states, can often be reduced to a simple model, so long as investigators keep in