Signal detection theory, yes/no

Detection or old/new recognition: one binary response per trial, counted as hits and false alarms

The decision: "old" when the evidence exceeds the criterion

Evidence on noise (new) and signal (old) trials. Shaded to the right of the criterion: the false-alarm and the hit rate. Drag the criterion, or focus it and use the arrow keys.

Task settings

None: bmm counts the "old" responses on noise and on signal trials, and no feature of the task enters their probabilities.

Parameters dist

Shape of the noise and signal distributions: normal; logistic, symmetric with heavier tails; gumbel_min or gumbel_max, skewed mirror images whose location 0 is the peak. Each enters with scale 1, so under the last three d is in units of that scale, not of the SD. In bmm: the dist argument. At the start, logistic instead of normal raises the false-alarm rate from 0.31 to 0.38.

criterion bias

d, sensitivity

Sensitivity: how far the signal distribution sits from the noise distribution, in units of √((1 + sdratio²)/2), the root mean square of the two scales. At sdratio = 1 it is d′: with normal noise, d = 1 and the criterion at 0.5 give a hit rate of 0.69 and a false-alarm rate of 0.31.

criterion, from the noise location

Where the boundary between "new" and "old" lies. criterion measures it from the noise location (0); bias measures the same boundary from the midpoint between noise and signal. Bias 0 is the midpoint, and a positive bias means fewer "old" responses, false alarms and hits alike. At the start, d = 1 puts the midpoint at 0.5, so criterion = 0.5 and bias = 0 are the same boundary.

sdratio, signal scale over noise scale

Scale of the signal distribution over the scale of the noise distribution (the SD ratio under normal noise). Above 1 the signal distribution is wider than the noise one, and the ROC is no longer symmetric. At sdratio = 2 the signal distribution sits at 1.58 and is twice as wide: the hit rate rises from 0.69 to 0.71, and the false-alarm rate stays at 0.31. bmm estimates it on the log scale.

Predicted P("old") simulate

Draws N trials, half noise and half signal, and marks the sample's false-alarm and hit rates on their bars (black lines) and as a point on the ROC. Below the panel: the expected rates and, in brackets, the sample's. The sample keeps its random numbers while the sliders move, so it changes with the parameters; New sample draws other ones. In bmm: rsdt_yn(), in the code under In bmm.

P("old") on noise and on signal trials. Dashed: bmm's prior means. Black: the sample.

ROC: every criterion at this d and sdratio

What the model says

Signal detection theory (Green & Swets, 1966) says that each trial yields a value of evidence on one axis and that the observer answers “old” (or “signal”) when the evidence exceeds a criterion \(k\). Noise trials (new items) draw the evidence from a distribution located at 0 with scale 1; for the normal distribution the scale is the SD. Signal trials (old items) draw it from the same distribution, moved to \(\delta\) and stretched by a factor \(s\). The false-alarm rate FA is the probability of “old” on a noise trial, the area under the noise density to the right of \(k\), and the hit rate H the same area under the signal density. With \(F\) the cumulative distribution function of the standard distribution (location 0, scale 1) that dist names,

\[ \text{FA} = P(\text{"old"} \mid \text{noise}) = 1 - F(k), \qquad \text{H} = P(\text{"old"} \mid \text{signal}) = 1 - F\!\left(\frac{k - \delta}{s}\right). \]

bmm calls \(k\) criterion and \(s\) sdratio. Its sensitivity parameter d, the slider on this page, sets the distance \(\delta\) between the two locations: \(\delta = d\,\sqrt{(1 + s^2)/2}\), so d is the distance divided by the root-mean-square of the two scales. With equal scales (\(s = 1\)) d and \(\delta\) are the same, and d is the familiar \(d'\); with unequal scales d is the index called \(d_a\), and \(\delta\) is larger than d when \(s > 1\). bias places the same boundary \(k\) but measures it from the midpoint between the two locations: bias \(= k - \delta/2\). Under equal variance it is the index usually written \(c\); 0 is the midpoint, and positive values put the boundary closer to the signal, so the observer answers “new” more often. A row of data holds the number of “old” responses out of n_trials noise or signal trials, which is binomial with probability FA or H.

dist sets the shape of both densities: normal, logistic, gumbel_min or gumbel_max. The logistic is symmetric like the normal, with heavier tails. The two Gumbel (extreme-value) distributions are skewed, mirror images of each other, and their location 0 is the peak, not the mean. Each enters with scale 1, so under logistic and Gumbel noise d and criterion are in units of that scale, not of the SD (the standard logistic has SD \(\pi/\sqrt{3} \approx 1.81\), the standard Gumbel \(\pi/\sqrt{6} \approx 1.28\)).

The task

One trial presents one item: a studied or a new word in recognition, a display with or without the target in detection. The participant answers old or new, yes or no. bmm fits counts: each row of the data holds n_old, the number of “old” responses, out of n_trials trials, with stimulus coded 0 for noise and 1 for signal. No task setting enters the likelihood, so the Task settings panel stays empty. How many trials a row has decides the spread of its counts, not the probability of an “old” response.

Explore

The top panel draws both evidence densities and the criterion; drag the line or move the slider. The shaded area under the noise density to the right of the line is the false-alarm rate, the shaded area under the signal density the hit rate, and the bars repeat the two numbers. With criterion selected, raising d moves the signal density to the right and raises the hit rate alone, and moving the criterion changes both rates together. sdratio above 1 widens the signal density, so the hit rate changes more slowly as the criterion moves through it. Switching to bias keeps the boundary where it is and gives its position from the midpoint between the two locations, which then appears as a dotted line; with bias held fixed, a change in d drags the criterion along with the midpoint.

The ROC (receiver operating characteristic) panel plots the hit rate against the false-alarm rate for every position of the criterion, at the current d and sdratio; the circle is the current criterion and the grey diagonal is chance, where hits and false alarms are equally likely. With equal scales the normal and logistic ROCs are symmetric about the other diagonal, from the top left to the bottom right. The Gumbel ROCs are not, even at sdratio = 1: under gumbel_min the point farthest from the chance line moves to low false-alarm rates, as it does under normal noise with sdratio above 1, and under gumbel_max to high ones.

The sliders span bmm’s default priors, the prior mean ± 3 SD, written normal(mean, SD): d normal(1, 1), from −2 to 4; criterion normal(0.5, 1.5), from −4 to 5; bias normal(0, 1.5), from −4.5 to 4.5. bmm estimates sdratio on the log scale with a normal(0, 0.3) prior, so its slider moves in steps of log(sdratio), from a ratio of \(e^{-0.9} = 0.41\) to \(e^{0.9} = 2.46\), and the box shows the ratio. Unless a model formula names sdratio, bmm fixes it at 1. The dashed bars and the dashed ROC are the prior means, d = 1, criterion = 0.5 (the same boundary as bias = 0 at these values) and sdratio = 1, so at the start values they lie under the solid ones.

Across task settings

The yes/no model has no task setting in its likelihood, so there is no design variable to vary while the parameters stay fixed. What an experiment can vary between conditions is where the observer puts the criterion, and that is what identifies sdratio. One condition gives one pair of rates, one point on the ROC, and a point does not fix the curve. sdt_d(), bmm’s function for the equal-variance d of a hit and a false-alarm rate, gives an observer with sdratio = 1, the same criterion and another d who produces the same two rates. Once sdratio differs from 1, that observer’s ROC appears as the dotted curve, through the circle. Two rates cannot fix three parameters (d, criterion, sdratio), so sdratio needs conditions that share d and move the boundary. In bmm’s formula syntax, d ~ 1 keeps one d for all conditions and criterion ~ 0 + condition gives each condition its own criterion: that is the design for a base-rate or payoff manipulation. A criterion that is the same in every row lets sdratio trade against d, and bmm() warns for it. With bias in the formula instead, a constant bias moves the criterion whenever d changes, so conditions that differ in d also give several points on the curve; the estimate then rests on the assumption that the observer keeps the bias, not the criterion, constant.

In bmm

The code updates with the controls. sdt_yn() names the data columns (response, the count of “old” responses; stimulus; n_trials) and the evidence distribution, dsdt_yn() returns the log likelihood of the counts of “old” responses, and bmm() estimates the parameters from data; my_data stands for your own data, and fitting runs Stan through brms. dsdt_yn() takes sdratio as the ratio itself, while the model estimates its log. See the signal detection article and the reference pages for sdt_yn(), dsdt_yn() and sdt_d().