Amino acids. Of the 198 test materials, 12 were amino acids. Figure 4 shows p-value ranges for the pairings of amino acids. The green indicates p-values ≥0.1, a more stringent qualification than the test device's normal threshold of 0.05 for a pass result. For all amino
acids, the test device correctly verified each material. The test device also correctly rejected nonmatching materials, with
p-values ≤1 x 10–8 .
Figure 5: Confusion matrix for organic solvent subset.
Solvents. Test materials also included 15 organic solvents. Figure 5 shows p-value ranges for the test pairings of these solvents. The green area indicates p-values ≥0.1, again, a more stringent qualification than the test device's normal "pass" threshold of 0.05. For this category,
the test device correctly verified each material. It also correctly rejected nonmatching materials, with p-values ≤1 x 10–8 , as indicated by red squares in Figure 5.
Figure 6: Confusion matrix for long-chain hydrocarbon subset.
Long-chain hydrocarbons. The p-values for 13 long-chain hydrocarbons are shown in Figure 6. Unlike the other categories of test substances discussed previously,
some of these pairings resulted in higher p-values when a method was tested against a similar material in this class of compounds. For larger molecules such as these,
the Raman signal corresponding to the long hydrocarbon chain portion of the molecule can dominate the signal making spectral
features arising from other functional groups difficult to detect. This is in contrast to small molecules where single atomic
or functional group substitutions are easily detected (e.g. acetone/methyl ethyl ketone and calcium sulfate/magnesium sulfate).
Categories of materials.
In almost all cases, the long-chain hydrocarbon pairings produced exclusive (completely selective), pass verdicts with p-values >0.1, indicated by green in Figure 6. The one exception was the magnesium stearate–sorbitan monopalmitate pairing.
In this case, the Raman signal from the long-chain hydrocarbon (differing between the two materials by only one carbon) overwhelms
differences present in the rest of the molecule. About half of the negative pairings produced very low p-values ≤1 x 10–8 , indicated by red in Figure 6. Nineteen pairings of long-chain hydrocarbons resulted in p-values in the range of 0.01 to
0.1 (yellow in Figure 6). Off-diagonal p-values in this range represent cases in which clear gaps in selectivity between methods exist (p-value >0.05) or where the pairings are not so far removed, lying in the 0.01–0.05 range, that there is risk for inadequate
selectivity. It is important to identify materials like these during method validation to identify potential false positives
and assess their degree of risk. Another group, the 54 pairings indicated by orange in Figure 6, produced p-values in the range 1 x 10–8 to 0.01 which are of significantly less concern during method validation and robustness testing.