Receiver Operating Characteristic (ROC)
Receiver Operating Characteristic curves are common in psychophysical research. Here I'll use an interactive graph coded in Python using Plotly. These curves are rather simple, but can be used in many different fields of study. I'll primarily discuss the ROC as it is used in both psychology and neuroscience research.
In the graph below two distributions are plotted. In this example, these distributions correspond to the firing rates of two separate neurons while an animal views a particular image. Let's say each of these neurons has a preference for a particular object. Although simplistic, lets also assume that the firing rates of these neurons determines how I categorize a particular object. Notice the red vertical line situated at the far left hand side of the plot- this a criterion, or threshold. If the firing rate of my neuron is higher than this threshold, I'll categorize the object according to the neuron's preference.
However, I have two neurons in this example, and depending on where I place my criterion, and the firing rate distributions, it might be equally likely that both neurons are firing above threshold. In such a case, the brain would do a poor job correctly categorizing the image being viewed. An ROC curve is an intuitive way of quantifying the selectivity of single neurons - that is, how well can the brain differentiate images given their individual firing rates while viewing a particular image.
In the upper right of the graph is an inset plot of an example ROC curve. In traditional signal detection theory, the X and Y axes are the probabilities of obtaining True and False Positives. If we're diagnosing cancer, this is the frequency that I correctly diagnose a tumor as malignant and the frequency I incorrectly diagnose a tumor as malignant. In our example, however, the X axis is the probability that Neuron 1 achieves threshold, and the Y axis is the probability that Neuron 2 achieves threshold. The ROC curve is created by shifting the threshold, or criterion, to different firing rates. For a visual illustration, try using the slider on the bottom to alter the firing rate threshold, and watch the red dot move along the ROC curve in the inset graph. The area under this curve is indicative of how similar the firing rate distributions are between these neurons. It can also quantify how well the neurons do at categorizing objects.
In the graph below two distributions are plotted. In this example, these distributions correspond to the firing rates of two separate neurons while an animal views a particular image. Let's say each of these neurons has a preference for a particular object. Although simplistic, lets also assume that the firing rates of these neurons determines how I categorize a particular object. Notice the red vertical line situated at the far left hand side of the plot- this a criterion, or threshold. If the firing rate of my neuron is higher than this threshold, I'll categorize the object according to the neuron's preference.
However, I have two neurons in this example, and depending on where I place my criterion, and the firing rate distributions, it might be equally likely that both neurons are firing above threshold. In such a case, the brain would do a poor job correctly categorizing the image being viewed. An ROC curve is an intuitive way of quantifying the selectivity of single neurons - that is, how well can the brain differentiate images given their individual firing rates while viewing a particular image.
In the upper right of the graph is an inset plot of an example ROC curve. In traditional signal detection theory, the X and Y axes are the probabilities of obtaining True and False Positives. If we're diagnosing cancer, this is the frequency that I correctly diagnose a tumor as malignant and the frequency I incorrectly diagnose a tumor as malignant. In our example, however, the X axis is the probability that Neuron 1 achieves threshold, and the Y axis is the probability that Neuron 2 achieves threshold. The ROC curve is created by shifting the threshold, or criterion, to different firing rates. For a visual illustration, try using the slider on the bottom to alter the firing rate threshold, and watch the red dot move along the ROC curve in the inset graph. The area under this curve is indicative of how similar the firing rate distributions are between these neurons. It can also quantify how well the neurons do at categorizing objects.
The plot below further illustrates this concept. The slider on the bottom of the plot will now separate the two firing rate distributions. When the curves are nearly overlapping, the neurons will always reach threshold with equal probabilities regardless of the criterion, resulting in an ROC curve that is essentially a diagonal line, and an area of 0.5. As the distributions separate, you'll notice the ROC curve bends and eventually approaches a 90 degree angle. The largest area for an ROC curve will therefore be 1, and would be indicative of a pair of neurons that never make "mistakes" when choosing between each of their preferred objects.
This particular example scenario was used in previous research to determine whether the performance of indvidual neurons could predict an entire brain region's performance on an object recognition task. For more information check out Nicole Rust and James DiCarlo's 2010 paper, Selectivity and Tolerance (“Invariance”) Both Increase as Visual Information Propagates from Cortical Area V4 to IT.