A new Covid vaccine is in test phase.

They find that for every 100 positive results, 3 are
actually spurious.
False positive rate: 3%

They also find that for 100 subjects, definitely
non-infected, 2 test positive.
False positive rate: 2 %

PS Ever notice they never mention the Covid test error rates?

--
Rich

On 07/09/2022 22:32, RichD wrote:
> A new Covid vaccine is in test phase.
>
> They find that for every 100 positive results, 3 are
> actually spurious.
> False positive rate: 3%

No, the false positive rate is the ratio of

tested people who are uninfected but test positive
--------------------------------------------------
tested people who are uninfected

Your 3% above is:

tested people who are uninfected but test positive
--------------------------------------------------
tested people who test positive

>
> They also find that for 100 subjects, definitely
> non-infected, 2 test positive.
> False positive rate: 2 %

Yes, that's the FPR.

>
>
> PS Ever notice they never mention the Covid test error rates?
>
> --
> Rich
>

RichD expressed precisely :
> A new Covid vaccine is in test phase.
>
> They find that for every 100 positive results, 3 are
> actually spurious.
> False positive rate: 3%
>
> They also find that for 100 subjects, definitely
> non-infected, 2 test positive.
> False positive rate: 2 %
>
>
> PS Ever notice they never mention the Covid test error rates?

Most people will misunderstand them.

On September 7, FromTheRafters wrote:
>> They find that for every 100 positive results, 3 are
>> actually spurious.
>> False positive rate: 3%
>> They also find that for 100 subjects, definitely
>> non-infected, 2 test positive.
>> False positive rate: 2 %
>> PS Ever notice they never mention the Covid test error rates?
>
> Most people will misunderstand them.

Most people are incapable of applying Bayesian reasoning?

--
Rich

On September 7, Mike Terry wrote:
>> They find that for every 100 positive results, 3 are
>> actually spurious.
>> False positive rate: 3%
>
> No, the false positive rate is the ratio of
>
> tested people who are uninfected but test positive
> --------------------------------------------------
> tested people who are uninfected
>
> Your 3% above is:
>
> tested people who are uninfected but test positive
> --------------------------------------------------
> tested people who test positive
> >
> > They also find that for 100 subjects, definitely
> > non-infected, 2 test positive.
> > False positive rate: 2 %
>
> Yes, that's the FPR.

Both of these statistics are of interest to the public health
authority. Why is it, that only one has a unique name?

Also for the individual - when tested, he wonders: what's the
chance of a positive result, if I'm uninfected? And: if the result
is positive, what's the chance it's erroneous?

--
Rich

RichD formulated the question :
> On September 7, FromTheRafters wrote:
>>> They find that for every 100 positive results, 3 are
>>> actually spurious.
>>> False positive rate: 3%
>>> They also find that for 100 subjects, definitely
>>> non-infected, 2 test positive.
>>> False positive rate: 2 %
>>> PS Ever notice they never mention the Covid test error rates?
>>
>> Most people will misunderstand them.
>
> Most people are incapable of applying Bayesian reasoning?

Sadly, I believe so. I can't find any statistics though to back it up.

There is an explainer though, if you ignore the headline bold typo.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7729143/

On 08/09/2022 20:00, RichD wrote:
> On September 7, Mike Terry wrote:
>>> They find that for every 100 positive results, 3 are
>>> actually spurious.
>>> False positive rate: 3%
>>
>> No, the false positive rate is the ratio of
>>
>> tested people who are uninfected but test positive
>> --------------------------------------------------
>> tested people who are uninfected
>>
>> Your 3% above is:
>>
>> tested people who are uninfected but test positive
>> --------------------------------------------------
>> tested people who test positive
>>>
>>> They also find that for 100 subjects, definitely
>>> non-infected, 2 test positive.
>>> False positive rate: 2 %
>>
>> Yes, that's the FPR.
>
> Both of these statistics are of interest to the public health
> authority. Why is it, that only one has a unique name?

Perhaps both of them have names, I don't know. (Also, it wouldn't surprise me if there were
different uses of the names, so always read the small print! :) )

I'd expect the categorisations true positive, false positive, true negative, false negative have
clear meanings, but adding "rate" maybe not so much. (Rate needs a qualification "compared to
what?" which isn't so obvious.)

I'll suggest one factor you may not have thought much about. The FPR (as I defined above) is an
attempt to extract a meaningful and hopefully "stable" characteristic of a particular diagnostic
test. What I mean is that if you have 1000000 uninfected people, and apply the test, we expect a
stable proportion of them to still test positive, due to the way the test works and is applied.
Similarly if we have 1000000 infected people, some stable proportion will give negative results when
tested. The two rates are typically different, represent different modes of failure in the test,
applying in different circumstances (one when a patient is infected, one when not) - but the hope is
that those two rates are fixed, given the test.

The previous paragraph is admitedly an idealisation. E.g. perhaps there isn't one single FPR for
everyone, and the test behaves qualitatively differently for men/women, or on different ethnicities
and so on, so they should all have their own rates.

Anyway, contrast this with the rate:

>> tested people who are uninfected but test positive
>> --------------------------------------------------
>> tested people who test positive

This isn't purely a characteristic of the test itself, and it's value will depend crucially on the
proportion of the population that is currently infected which can go up and down. So it does not
make so much sense to quote this as a stable property of the test.

The problem is that the people who test positive are a mixture of correct result on infected people
and incorrect results on uninfected people, and we are mixing the two physical "stable" rates TPR
and FNR - which we can handle with Bayes theorem, but need to know the prior probibilities.

>
> Also for the individual - when tested, he wonders: what's the
> chance of a positive result, if I'm uninfected?

that's the FPR, but the problem is he doesn't know whether he is actually infected, so it's not as
useful as it sounds... What he really wants to know is "am I /actually/ infected", but no test is
going to answer that 100% accurately.

> And: if the result
> is positive, what's the chance it's erroneous?

He could work this out from published rates, provided he also knows the prior-probability that he
was infected. This is not easy to know - even a region-wide "prevalence" figure is tricky for
authorities to estimate, and an individual might reason that they are "much more careful than most
people", and so are less likely to be infected than an averaged prevalence figure would suggest
etc.. So things are not simple...

Anyway, all this is quite complicated to explain. Maybe you have a natural affinity for the
subtleties of the probabilities and rates, and all the above seems obvious to you, but most members
of the public simply would not be able to appreciate Baysian inferences that easily.

That's probably why little attempt is made by governments to explain this aspect thoroughly to their
populations! Also there's the problem that with a low prevalence for infections, there can be a
significant chance (say around 50%) that a positive result is a false positive, even if the FPR is
very low for the test (like less than 0.1%). This might tempt people into deciding not to isolate
when they test positive, which is not the message the government wants to project! Best to give a
simple rule to follow: if you test positive, isolate at home for 10 days, or whatever...

In the past I've seen good examples on the internet where calculations are given making the need for
prior probabilities clearer. (I can't instantly find them now!) I suppose I could make up some
rates/figures to illustrate how a person testing positive can have a significant probability that
it's a false positive, even though the FPR for the test is very low.

Mike.