Nine published studies, encoded as Negative-Results Records

Summary

The data model held. We designed it before reading any of these papers, and nothing in it had to be renamed across nine studies. Twenty-nine things were added, nearly all about where a value came from and how much to trust it, and the number of additions per study fell from twelve for the first to two for the fifth, then 1, 0, 1 and 0 across four chemistry papers. A format that stops changing when you feed it new kinds of science is a format that has found its shape. The records validate, the build is deterministic, and cross-study questions run over shared identifiers and vocabularies.

The nine studies

Each publishable study can also be read as a discourse graph: the question, the hypothesis, the attempts that addressed it and the findings they produced, with the edge each finding earns under the informativeness rule. The graphs are drawn from fields the records already hold; nothing was added to the schema to make them.

The first two were chosen because they were the obvious cases: a large AI-driven screen and a clean human-run informative failure. The next three were chosen to make the model fail. The last four were chosen to take it outside biology, since the materials profile had never met a real paper, and they come from a screen of American Chemical Society journals for CC BY articles whose finding is a negative result. Each is described below in the terms of the test it was meant to be.

What a record holds

A wet-lab attempt from the Robin study carries one finding per arm of the screen. This is the round-two ripasudil finding, abridged: the compound identifier, the dose from the supplementary drug table, the per-well values from the supplementary workbook, the designated positive control with its observed value, the vehicle control, the sensitivity statement from the Reporting Summary, and who proposed the compound.

{
  "question": "Does Ripasudil at 100 µM increase phagocytosis by ARPE-19 cells of pHrodo beads (1 h pre-treatment, 3 h uptake)?",
  "target": {"label": "Ripasudil", "identifier": "pubchem:CID9863672", "type": "compound"},
  "outcomeClass": "positive",
  "effect": {"metric": "normalised MFI (fold of DMSO)", "value": 1.8906, "n": 3, "sem": 0.2378,
             "perReplicate": [2.3619, 1.7096, 1.6002], "test": "Dunnett's test versus DMSO"},
  "controls": {"positive": {"kind": "designated", "label": "MFGE8", "observed": "1.46x (n=2)", "passed": true},
               "negative": {"kind": "vehicle", "label": "0.5% DMSO", "passed": true}},
  "sensitivity": "Reporting Summary: 3 replicates reliably distinguish the MFGE8 control from vehicle",
  "conditions": {"dose": "100 µM", "vendor": "Cayman Chemical Company", "catalog": "19920", "doseReference": "doi:10.3390/ijms25063229"},
  "proposedBy": "robin-round-2", "proposedByConfidence": "inferred"
}

The Alcalase knock-in finding shows the rule refusing to call a negative evidence of absence: no positive control for editing existed, so the class is inconclusive-no-positive-control, and the record states the detection limit that the 40 sequenced colonies do support.

{
  "outcomeClass": "inconclusive-no-positive-control",
  "informativeness": "uninformative",
  "informativenessReason": "no positive control",
  "sensitivity": "0 of 40 sequenced colonies edited: an on-target rate above 7.5% among NAT-resistant colonies
                  would have been detected with 95% probability (rule of three, derived by portolan-ingest)"
}

The Raman study forced the most useful change of the five. Its controls are not reagents but tasks, and they are better controls than anything in the wet-lab studies. The positive control is a classification the pipeline should be able to do. The negative control is one it should not: predicting which plate a spectrum came from, when the three plates are end-to-end replicates of one protocol. The adversarial control scores a perfect 1.0, so the strain result is confounded.

{
  "question": "Can strain identity be predicted from Raman spectra when whole experimental replicates are held out?",
  "outcomeClass": "inconclusive-controls-failed",
  "informativeness": "uninformative",
  "informativenessReason": "negative control not clean; no sensitivity or power statement",
  "controls": {
    "positive": {"kind": "known-positive-task", "label": "species classification on the same spectra and pipeline",
                 "expected": "recoverable biological signal if the spectra and pipeline work",
                 "observed": "MCC 0.97 under leave-one-plate-out cross-validation", "passed": true, "source": "Figure 6"},
    "negative": {"kind": "adversarial-label", "label": "predicting plate identity, which should be unpredictable",
                 "expected": "chance performance (MCC 0), because the three plates are end-to-end replicates",
                 "observed": "MCC 1.0", "passed": false, "source": "Figure 4"}
  }
}

Does the rule discriminate?

The informativeness rule says a negative counts as evidence of absence only when a positive control passed, the negative control stayed clean, and a sensitivity or power statement exists. A rule that refuses everything is a filter, not a diagnosis. So the question is whether it fires in different ways on different data, and names the actual defect each time.

It does. Across nine studies every clause has fired on real published work, and each of the twenty-seven refusals carries the reason. In chemistry the rule both passes negatives and refuses them: the TMEDA kinetics at 60 °C and the two-week PFOA result are informative because those papers ran a positive control, a clean background and defined what they could detect; the zeolite syntheses and the XeF₆ reactions are inconclusive because no positive control was run and no detection limit was stated, and in the XeF₆ case the authors say as much themselves.

The last row is the one worth dwelling on. After batch correction the Raman adversarial control comes back clean at MCC 0.1 and the positive control still passes at 0.93, so the only thing missing is a statement of how large an effect the experiment could have detected. The rule flags exactly that, and it matches the authors’ own caveat in the pub. The rule is reading the same weakness the scientists did.

One study forced an exemption. The Raman paper does not merely fail to find a strain signal; it refutes an earlier, stronger result obtained on the same data under a weaker validation scheme. An affirmative refutation is judged on the refuting analysis, not on the controls of the run it overturns, so refuted is now exempt from the control requirement. That is the kind of change only a real paper produces.

What we learned, in plain words

Nine studies, five in biology and four in chemistry, read closely enough to turn every result into a record a machine can check. Here is what that taught us, written for someone who has never run an experiment.

1. The failures were never written down as failures. The Robin paper reports three drugs that worked. Its spreadsheet holds about ninety that did nothing, and nobody says so anywhere. To write the sentence “fingolimod at this dose did not help these cells” we had to pull one number from a spreadsheet, a dose from a table in a PDF, a sample size from a form, and a verdict from the main text. The AI needs that sentence too: in its second round Robin is told “these drugs have already been tested, do not suggest them again”. That memory lives in one prompt and is gone when the run ends.
2. “Nothing happened” only means something if you could have seen something. If your kitchen scale reads in whole kilograms, “the weight did not change” does not tell you whether a gram was added. So every negative result in our records has to show three things: a test that was supposed to work and did (a positive control), a test that was supposed to do nothing and did nothing (a negative control), and how small a change the measurement could have picked up. Of 217 findings, 105 negatives clear that bar and 27 do not, and the record names which piece is missing each time. The chemistry papers split two and two. In one of the two that fail, the authors say the same thing themselves: “it may be that our reaction conditions are not right, yet”.
3. The same data can give opposite answers. In the Robin study the same cell measurements made the control pass when a person analysed them and fail when the AI did. In the Raman study the same spectra and the same model scored 0.79 or 0.32 depending only on how the data were split for testing. A record that says “no effect” without saying who analysed it, and how, will be wrong some of the time. So the analysis is its own entry, linked to the experiment.
4. Data gets uploaded and then forgotten. Four datasets across two studies were deposited in public archives and never analysed in the paper. You cannot see that by reading the paper. You can see it by comparing the archive’s file list with the text, which a program can do.
5. Papers lose the facts that matter most. Which AI model proposed which drug. How many times a reaction was tried (one chemistry paper never says). Whether the compound tested was the one the computer modelled (in one case it was a mixture, of which the modelled one was about a quarter). Every one of these was known at the moment of the experiment and gone by the time of the paper.
6. PDFs hide the evidence. In two of the four chemistry papers the key tables are pictures inside a PDF. We read them cell by cell. None of the four shares its data anywhere else, apart from one crystal structure in a public database. That is the concrete thing a publisher could change.
7. Licences decide what can be shared. We encoded nine studies and can republish eight. The ninth is a Nature paper whose licence forbids sharing adapted material, so its records stay on our machines. The difference is a licence, not a technical limit.
8. Someone still has to judge. Which wells are the controls, whether every non-hit counts as a failure, which AI proposed which candidate. Each call is flagged in the record, and an expert should review them. The real system is an AI proposing records and a person approving them, not a converter.

What happens when the robot writes the record itself

Everything above came from reading papers after the fact. The obvious fix is to have the system that runs the experiment write the record while it still knows everything. On 3 October 2026 we built the first version of that: a six-question checklist a record must pass at the moment it is created, and two programs that try to fill it in from real systems.

We tried it on both halves of a lab. Robin, the AI from the Nature paper, decides what to try; it is open source and writes a folder of files when it runs. Data Hub, Arcadia Science’s open-source system, catches the raw data from lab instruments. Each kept something the other lost.

• Robin’s folder has every idea it ranked with its score, every file can be fingerprinted, and its data-analysis step names the model that did it. But it never writes down which model proposed the ideas or judged them. The checklist refused those records with a clear message, and we had to supply that fact from the paper. The fix is one line inside Robin: write its own settings into the folder it already creates.
• Data Hub knows the instrument, the time, the settings, the files and who ran them, and it serves all of that to programs. It has no idea why the experiment was run or what the controls were. It is a filing cabinet, not a scientist. A person or an AI adds a short claim on top, and the checklist is what makes sure the claim includes the controls and the detection limit. Data Hub also does not store a fingerprint per file, so our program computes one when it can read the file.

Nothing in the record format had to change for either. The code, a worked example from a local Data Hub, and the write-up are in the repository under docs/capture-profile.md. No real experiment has been captured yet; that is the next step and it needs a lab.

Schema changes and the convergence test

Every change is logged with the study that forced it in the change log. The count per study fell as studies were added, which is what convergence looks like if it is real.

Three points in a falling sequence are not evidence, and the first four studies had been chosen for what they would teach rather than for what they would break. So the fifth was run as a pre-registered test. The criterion was fixed before encoding began: converging if the study needs two or fewer additive changes and nothing structural, meaning no new record level and no change to the shape of a finding, a control or an effect. The study was chosen to maximise the chance of failure. The easiest remaining candidate, a toxicity screen nearly the same shape as the Robin work, was deliberately passed over in favour of a comparative-genomics paper with no experiment in it at all.

It passed: two changes, both single vocabulary terms, a phenotype entity type and a data-curation contributor role. Nothing structural. What carried unchanged is the substance of the result. The field built to hold literature databases searched held six reference databases. The field built for a mouse troubleshooting loop held post-hoc statistical recoding. The effect structure held an Akaike information criterion and a phylogeny-corrected slope with no new fields at all.

The convergence test left one direction open: every study was biology. On 2 October 2026 four chemistry papers were encoded, each converter run first with no schema change so that validation, not judgement, counted what the study forced. The result was 1, 0, 1 and 0: a material entity type for zeolite framework codes, and a measurement-artifact failure mode for an apparent effect produced by sample loss rather than by the process under test. Nothing structural, across nine studies and two domains. Three failure modes written for materials in v0.1 and never used fired for the first time; the rest of that profile was not needed. The write-up, with the judgement calls each record makes, is docs/case-study-acs.md.

Questions an agent can ask

The store answers named queries over all nine studies at once. Examples, with what came back:

• Has ripasudil been tried as an RPE phagocytosis enhancer? Three records: proposed in round two, 1.89× in ARPE-19 at 100 µM, 1.54× in primary cells.
• What do we know about fingolimod at what dose? Three informative negatives at 1 µM, with the sensitivity statement attached.
• Which negatives are not evidence of absence, and why? Twenty-seven findings across seven studies, each naming the failing or missing control.
• Has PFOA been tested with a laccase mediator system? Yes, and the informative answer is no: a two-week dosing experiment with carbamazepine as the positive control, p = 0.29 against the untreated reactor, and the apparent 64 % loss at 0.1 µM refuted by mass balance.
• Where could an agent pick up work? Fifty-four entries: twenty-eight next steps the authors proposed, sixteen questions they asked openly, four datasets deposited and never analysed, six inferred during ingest.
• Which results depend on a validation scheme rather than on data? Seven refutations across four studies: the Raman strain claim with both scores and both schemes, three zeolite predictions with the syntheses that refuted them, the TMEDA inhibition test above 60 °C, and two apparent PFAS losses with the mass balance that explains them.
• Are the DOIs this record depends on still intact? Every cited source carries version and integrity status and the time of the check.

What is published, and what is not

• Code and schema: github.com/portolansoft/nrr, Apache-2.0 for code, CC BY 4.0 for the schema documents. All nine converters are there. The Nature-derived inputs are fetched by a script at build time and are not redistributed.
• Records: github.com/portolansoft/nrr-records, CC BY 4.0. Thirty-eight records from the four Arcadia and four ACS studies are published, with attribution to the authors and to the datasets they build on.
• Graphs: the eight publishable paths drawn as discourse graphs, in the grammar of Discourse Graphs (question, hypothesis, experiment, result, issue). A rendering of existing fields by scripts/render_graphs.py; the valence of each edge comes from the finding’s outcome class and informativeness.
• Not published: the eleven records derived from the Nature paper. Its licence forbids distributing adapted material, and per-well values plus quotations from a paper are arguably that. We would rather the authors deposit their own results, and we are asking. The converter that produces them is open, so anyone with the paper can rebuild them locally.

So nine studies were encoded and eight are redistributable. The difference is a licence, not a technical limit, and it is the single clearest argument for building the commons on permissively licensed sources.

Next steps

1. Encode a materials paper with a deposited dataset. The four chemistry papers tested the model outside biology, but only one of them deposits anything machine-readable, so the materials profile has been exercised against prose and PDF tables, not against structured data.
2. Ask the people who build the agents to write their own settings into their output. For Robin it is one line, and the record format is already waiting for it.
3. Run the ingest as an agent-plus-review loop on ten more permissively licensed sources and see which schema additions recur.
4. Capture one real experiment end to end: a lab instrument run caught by Data Hub, the claim written by the agent that ran it with its controls and detection limit, and the record arriving in a registry. The checklist and the two emitters exist; what is missing is a lab.
5. Put the discrepancies the pilot surfaced in front of the scientists who produced them, starting with a compound that worked in primary cells but not in the cell line and a control that passed under one analysis and failed under another.
6. Settle the licensing question for non-commercial, no-derivatives sources before ingesting more of them.

Comments and corrections to [email protected]. Back to portolansoft.com