5 and mentally projecting the sequential ROC curve to the right, it seems fairly safe to assume it would still fall below the simultaneous ROC. This empirical measure of discriminability is not based on any theoretical assumptions about memory. c If, for some reason, policymakers preferred a FAR of approximately .06 because of the higher HR that could be achieved, the fact that pAUCSIM > pAUCSEQ over the tested FAR range (0 to FAR z m Such research is often focused on testing theories that may have applied significance. Huntington: Robert E. Krieger Publishing Co. Gronlund, S. D., Carlson, C. A., Neuschatz, J. S., Goodsell, C. A., Wetmore, S. A., Wooten, A., & Graham, M. (2012). Target Under those assumptions, d' can be safely compared for Condition A vs. The same receiver operating characteristic (ROC) data as in Fig. And what would the policy implications be in a case like that?

) is equal for the three procedures and that the difference in pAUC arises because the simultaneous procedure is less susceptible to the deleterious effects of criterion variability than showups and sequential lineups. In both cases, it is the structural constraints of the testing procedure itself, not a difference in underlying latent variables, that results in a difference in the empirical ROC curves. 5 except that the smooth curves generated by a theoretical (signal detection) model are drawn through the ROC data points.

Journal of Applied Psychology, 70, 556–564. volume 3, Article number: 9 (2018) A photo lineup consists of a picture of one suspect (the person who the police believe may have committed the crime) plus several additional photos of physically similar foils (i.e., fillers) who are known to be innocent. and variance σ

In other words, the probability that a suspect identified from a sequential procedure is guilty is .746, whereas the probability that a suspect identified from a simultaneous procedure is guilty is .576. max

is the same in both cases.

Journal of Experimental Psychology: Applied, 19, 345–357. Finally, criterion variance (σ The same basic model applies to a lineup, but the way it works is somewhat different. (2016).

5, it is visually obvious that pAUC for the simultaneous procedure is greater than the pAUC for the sequential procedure over the FAR range of 0 to .038 (the maximum FAR for the sequential procedure). Z Instead, empirical discriminability refers to the degree to which participants correctly sort target and foil stimuli into their true categories.


through μ Figure 4 presents the ROC data computed from the values shown in Table 1. would be estimated to be about 1.4. = 2.0, empirical discriminability for the showup procedure is impaired to a greater extent than empirical discriminability for the simultaneous lineup procedure. If, instead of using confidence ratings, instructions were used to induce conservative responding from the outset such that only IDs made with high confidence were obtained in the first place, the police would lose the potentially useful investigatory information that a suspect ID made with low or medium confidence might provide. SAE International's core competencies are life-long learning and voluntary consensus standards development. (2015). Viewed in this light, the “controversy” over ROC analysis of lineup performance actually consists of a normal scientific debate about which theory of underlying latent variables better accounts for the empirical data. Empirical is the information you received and found out, and theoretical the information that is set. Google Scholar. Then compute d’, not the diagnosticity ratio. However, unlike a filler, an innocent suspect is not known to be innocent and will be imperiled (and perhaps wrongfully convicted) if mistakenly identified. To appreciate why the two measures can go in opposite directions without contradiction, it is important to consider how d'

Wells, G. L., Yang, Y., & Smalarz, L. (2015). Their simulations showed that, in the absence of criterion variability and with d' m

However, because this “hand waving” analysis of the effect of sequential lineups on the HR and FAR is clearly insufficient, a quantitative assessment of some kind is needed. Psychology, Public Policy, and Law, 17, 99–139. For example, if the prior odds of guilt are even (i.e., half target-present lineups, half target-absent lineups), one can ask about the posterior odds of guilt for the subset of lineups that resulted in a filler ID or No ID. Computer software is needed to precisely measure the size of the shaded area, and the tutorial videos associated with Gronlund, Wixted, and Mickes (2014) explain how to use one such R program, called pROC (Robin et al., 2011), to do that. (2012) actually estimated pAUC – not d' The data in Table 1 allow one to compute not only the overall HR and FAR but also a HR and FAR separately for varying degrees of response bias specified by the different confidence ratings. Foil

) was set to 0 for the simultaneous lineup and to 0.75 for the sequential lineup, which is why the sequential lineup, despite its higher d', yields a lower ROC than the simultaneous lineup. Moreover, no model of memory would be needed to reinforce the decision as to which of the two procedures is diagnostically superior. A stickler might contend that a minimum FAR greater than 0 should also be specified, one that is equal to the FAR associated with the leftmost ROC point from the condition with the larger minimum FAR (e.g., FAR Foil Nevertheless, to be sure about that, one would have to actually perform the experiment because it is at least theoretically possible that the ROC curves would cross and the sequential procedure would become superior in that higher FAR range. In addition, a slight variation of an earlier model [1] is presented to test the sensitivity of empirical models. Hypothetical receiver operating characteristics (ROC) curve for a lineup procedure in which a 5-point confidence scale was used. Live lineups were once the norm, but nowadays, the police almost always administer photo lineups after they identify a suspect in the days or weeks following a crime.

The model shown in Fig. Both were referring to what we have here denoted d' C To compare the two procedures with respect to pAUC, it is essential to use the same FAR Wetmore, S., Neuschatz, J. S., Gronlund, S. D., Wooten, A., Goodsell, C. A., & Carlson, C. A. Wixted, J.T., Mickes, L. Theoretical vs. empirical discriminability: the application of ROC methods to eyewitness identification. The diagnostic feature-detection theory attributes the difference to a d' However, unlike Fig. Vision Research, 40, 1227–1268. However, in the presence of criterion variability (equated across the two procedures), simultaneous lineups yielded higher empirical discriminability (measured by pAUC) than showups. 1 Cameron, E. L., Tai, J. C., Eckstein, M. P., & Carrasco, M. (2004). Simulated receiver operating characteristic (ROC) data generated by a simultaneous lineup using the MAX decision rule, a sequential lineup using the “first-above-criterion” decision rule, and a showup. The absolute/relative distinction was originally advanced as a theory of response bias, with a relative judgment strategy corresponding to increased pressure to choose someone from the lineup. For example, in the latest critique of ROC analysis, Smith et al. = 1.4 and with σ In essence, that kind of comparison is what ROC analysis is all about, and it illustrates why ROC analysis is needed to unambiguously determine the diagnostically superior procedure. = μ Thus, the DR (i.e., the likelihood ratio) is equal to the correct ID rate divided by the false ID rate (HR/FAR).

(i.e., over all possible memory-strength values for a filler) is given by: Again, this is the likelihood of observing a filler ID from a target-absent lineup. Other studies have reported no significant difference between the two procedures, but with a trend still favoring the simultaneous procedure (e.g., Andersen, Carlson, Carlson, & Gronlund, 2014; Exp. (2017). advantage enjoyed by simultaneous lineups compared to the other two procedures. Here, we describe how to write the likelihood function for that probability and then describe the similar approach used to write the likelihood functions for the probability of observing a filler ID from a target-present and then from a target-absent lineup. Alternatively, as noted earlier, confidence in No IDs could be collected in such a way as to allow one to project the ROC further to the right (i.e., by collecting a confidence rating in connection with the face that the witness believes is most likely to be the perpetrator). (1975). 1 $$, $$ \mathrm{f}=@\left(\mathrm{x}\right)\ \mathrm{normpdf}\left(\mathrm{x},\mathrm{mu}\_\mathrm{t},\mathrm{sigma}\_\mathrm{t}\right).\ast \mathrm{normcdf}\left(\mathrm{x},\mathrm{mu}\_\mathrm{d},\mathrm{sigma}\_\mathrm{d}\right).\hat{\mkern6mu} \left(\mathrm{k}-1\right).\ast \Big(1-\mathrm{normcdf}\left(\mathrm{c},\mathrm{x},\mathrm{sigma}\_\mathrm{c}\right), $$, $$ \mathrm{p}\_\mathrm{t}=\mathrm{integral}\left(@\left(\mathrm{x}\right)\ \mathrm{f}\left(\mathrm{x}\right),-15,15\right).

m Retrieved 29 Mar 2016, from http://www.policeforum.org/. Only recently, however, has signal detection theory been brought to bear on this issue. Their current model policy states: “This policy recognizes that the sequential and simultaneous approaches are both valid methods of conducting an identification procedure and does not recommend one over the other.” (International Association of Chiefs of Police, 2016, p. 1). According to this model, different eyewitness identification procedures are differentially susceptible to the deleterious effects of criterion variability. JSTOR is part of ITHAKA, a not-for-profit organization helping the academic community use digital technologies to preserve the scholarly record and to advance research and teaching in sustainable ways. If these data were fit by a model to estimate d'

is usually set to 0 because no specific theory is relied upon to justify the seemingly safe assumption that if responding were infinitely conservative, both the HR and the FAR would be 0. 6, even though underlying theoretical discriminability (d' This function corresponds to the probability of observing a target ID from a target-present lineup made with a particular level of confidence associated with criterion, c. If there are five confidence criteria for making a positive ID (as in Fig. 4 (integrated from − ∞ to + ∞): Again, this is the likelihood of observing a filler ID from a target-present lineup. Finally, the area beneath the curve was estimated from a FAR of 0 to FAR In summary, according to this theory, pAUCSIM > pAUCSEQ and pAUCSIM > pAUCSHOWUP because d'm-SIM > d'm-SEQ and d'm-SIM > d'm-SHOWUP, respectively. An understanding of that distinction is important for both theoreticians and policymakers because the two measures need not agree.