The half-life of facts in immunology
Working specification
1. Goal
Find out whether an AI reading the immunology literature can measure heterogeneity of knowledge: how far sources agree on the answer to a question, at one point in time. The first test is B cell immunophenotypes (Section 5).
The half-life of facts is the second aim, for later. It is the same measure followed across years, so the tool built for the first aim answers the second when run on older papers.
Heterogeneity comes first because one recent snapshot answers it, and because it is what a quantitative systems pharmacology (QSP) modeler needs now: whether the biology a model is built on is agreed on today. How often biology changed in the past bears on that only indirectly.
Out of scope. Building or fitting a QSP model, and judging which answer is correct. The project measures agreement, not truth.
2. Definitions
Question. Something sources can take a position on, with a set of possible answers. “Which markers define a DN2 B cell?”
Claim. One answer to a question, as a source states it. “DN2 B cells are IgD− CD27− CD21− CD11c+.”
Source. A paper, or a person answering a survey.
Agreement on a claim. At a given time, the share of sources giving that answer, out of the sources that take a position on the question. Sources that do not address the question are left out of the denominator.
Unknown. An answer in its own right: the source says the question is not settled. A field that agrees the answer is unknown has low heterogeneity and high uncertainty, and the measure must not read that as a settled fact. “What stops T cell expansion?” may be such a question.
Fact. A claim whose agreement is above a threshold, provisionally 90%. Being a fact is a status a claim holds at a time, not a property of the claim. This puts a fact at the far end of the heterogeneity scale: a question almost every source answers the same way.
Heterogeneity. How spread the answers to one question are at one time. The measure proposed is the chance that two sources picked at random give the same answer, the sum of the squared shares of each answer. If 60% of papers use one definition, 30% a second and 10% a third, the chance is 0.36 + 0.09 + 0.01 = 0.46. A value near 1 means the field agrees. With k answers in use, the lowest possible value is 1/k, when they are evenly split. Ecologists call this the Simpson index.
Half-life of facts. Take the claims that were facts at one time. The half-life is how long it takes for half of them to fall below the threshold.
3. Kinds of Claim
Questions differ in what agreement on them means, and are kept apart in every analysis.
| Kind | What disagreement means | Can it be wrong? | Example |
|---|---|---|---|
| Definition | Different names, or different markers, for the same thing | No; only agreed on or not | Which markers define a DN2 cell |
| Empirical | Different findings | Yes | Whether DN2 cells are expanded in lupus |
| Mechanism | Competing explanations | Yes, though often slow to settle | Whether T cells go naive → effector → memory or naive → memory → effector |
A half-life of facts applies to empirical claims and mechanisms. Definitions change by convention, and a refined definition, such as double-negative cells split into DN1 to DN3, has not been shown untrue.
Agreement is probably highest on definitions and lowest on mechanisms, while mechanisms are what decide a QSP model’s structure. B cell immunophenotypes are a place to build and test the method; the answer most useful for models is likely to be in the mechanisms.
4. Background
The half-life of facts. The term is from Samuel Arbesman’s book The Half-Life of Facts (2012). The one published estimate in medicine that the Google Doc cites is about 45 years for hepatitis and cirrhosis. That figure is presumably Poynard and colleagues, “Truth survival in clinical research” (Annals of Internal Medicine, 2002), in which experts reread conclusions from papers going back decades and judged which still held. ⚠️ Not checked against the paper.
The 2024 survey idea. The Google Doc proposed a 5 to 10 minute survey of immunologists, run by a team of five or fewer modelers and posted to Zenodo, asking which areas of immuno-oncology are understood well enough to model. Its reasons not to do it still apply:
- Modelers who work with immunologists already accept that much of the immune system is poorly understood.
- A survey takes a lot of work, and its answers change with time.
- It is hard to stay motivated unless the answer is needed for one’s own work. The doc names B cell subsets and depth of response in autoimmune disease as work that would need it.
What AI changes. Asking an AI to extract the answer to the same question from hundreds of papers is much cheaper than a survey, and it covers every published author rather than the people who reply.
Papers are not beliefs. Papers lag behind what their authors now think, and they rarely state what their authors take for granted. A survey measures belief directly but reaches fewer people. Running both on the same questions shows how far published agreement differs from what immunologists believe.
5. Starting Point: B Cell Immunophenotypes
B cell subsets in blood are identified by flow cytometry from a few surface markers. The markers most used are CD19 and CD20 (B cell lineage), CD27 (memory), IgD (naive and unswitched), CD38 and CD24 (transitional and plasmablast), and CD21, CD11c, T-bet and CXCR5 (atypical subsets). The table is a common scheme, written from memory; papers differ, and measuring how they differ is this project. ⚠️ Not checked against a source.
| Subset | A common definition |
|---|---|
| Transitional | CD19+ CD24hi CD38hi |
| Naive | CD19+ IgD+ CD27− |
| Unswitched memory | IgD+ CD27+ |
| Switched memory | IgD− CD27+ |
| Double negative (DN) | IgD− CD27−, split into DN1 (CD21+ CD11c−), DN2 (CD21− CD11c+) and DN3 (CD21− CD11c−) |
| Atypical or age-associated B cells | CD21lo CD11c+ T-bet+, overlapping DN2 |
| Plasmablasts | CD27hi CD38hi, CD20− |
The B cell area suits a first test because a gating strategy is a short, explicit list of markers that can be extracted and compared without judgment, because the double-negative and atypical subsets were named in the last 15 years and the same cells carry several names, and because the B cell reset project depends on which subsets are measured and how they are defined.
Candidate questions, by kind:
| Kind | Question |
|---|---|
| Definition | Which markers define naive, memory, DN1, DN2 and DN3 B cells? |
| Definition | Is a CD21lo CD11c+ B cell called DN2, atypical memory, or age-associated? |
| Empirical | Are DN2 cells expanded in lupus, and in which other autoimmune diseases? |
| Empirical | Which subsets return first after B cell depletion? |
| Mechanism | Do DN2 cells come from naive cells outside germinal centres, or from memory cells? |
| Mechanism | Does a shift toward naive B cells after depletion cause lasting remission, or only accompany it? |
6. How AI Would Measure It
Heterogeneity. For each question, gather papers from a recent window, 2024 to 2026 to start. From each paper, extract its answer, which may be “unknown”, or record that it takes no position. Keep the passage the answer rests on, with the year, disease area and lab. Compute agreement on each answer and the heterogeneity of each question, leaving “no position” out. For definition questions, also count how many names each marker combination goes by.
Half-life, later. Run the same extraction on earlier windows, such as five-year blocks back to 2000. Take the claims above the fact threshold in an early window, and follow each one’s agreement forward.
Checks on the AI.
- A person checks a random sample of extractions against the papers, and the error rate is reported.
- The search and extraction prompts are saved, so a second run can be compared with the first.
- A short survey of immunologists on a few of the same questions shows how far the papers differ from what people believe (Section 4).
7. Open Questions
- Is 90% the right threshold for a fact, and does it differ by kind of claim?
- Should a source count once, or be weighted, for example so that one lab’s ten papers do not count ten times?
- Is the Simpson index the right measure for mechanism questions, where answers are not cleanly separate options?
- Which immuno-oncology questions from the 2024 survey draft come next, such as T cell differentiation markers or cytokine effects on each cell type?
- The project has not read any sources yet. When it does, it needs a
references.qmd, starting with Arbesman and Poynard.
8. Steps
- Fold in the remaining thoughts from the Google Doc.
- Search for published reviews of B cell gating strategies and standardization efforts, so the extraction starts from what is already known.
- Settle the B cell question list and the answer options for each.
- Run a pilot extraction on 50 recent papers and check it by hand.
- Decide from the pilot whether to scale up, and whether to add the survey.
- Check the Poynard estimate and its method before starting on half-life.