BASEN: Accuracy and run-to-run stability in detail

This page presents the analyses supporting Results section 2 of the main report: L1 distance for every Oral run at full depth and at 25,000 reads, per-species variation across the Oral runs, run-to-run variation in all three mock communities, and a summary. Each standard was classified against a Kraken2 database containing its corresponding catalogue genomes.

1. Oral: L1 distance per run, at full depth and at 25,000 reads

L1 = Σ |estimated share − reference share| over the 12 catalogue species
both as fractions summing to 1 (0 = identical, 2 = no species in common). The six Oral runs were generated from one library on a single flow cell and differed primarily in depth. Subsampling to 25,000 reads (seqtk 1.3, seed 42) standardized sequencing depth across runs. Bold: lowest value in each column within each composition. Reads: Q ≥ 20 reads entering Kraken2.

Full depth

	Method	Run 1 165,929 reads	Run 2 111,559 reads	Run 3 85,055 reads	Run 4 75,026 reads	Run 5 30,780 reads	Run 6 36,485 reads	Mean ± SD runs 1–6
% gDNA
	Kraken2	0.291	0.324	0.331	0.366	0.315	0.293	0.320 ± 0.028
	Bracken	0.291	0.324	0.331	0.367	0.315	0.293	0.320 ± 0.028
	BASEN-nf	0.328	0.322	0.320	0.315	0.332	0.322	0.323 ± 0.006
	BASEN	0.330	0.323	0.321	0.316	0.334	0.323	0.324 ± 0.007
% genome copies
	Kraken2	0.323	0.359	0.366	0.408	0.341	0.316	0.352 ± 0.034
	Bracken	0.322	0.359	0.367	0.409	0.341	0.315	0.352 ± 0.034
	BASEN-nf	0.354	0.348	0.346	0.341	0.358	0.348	0.349 ± 0.006
	BASEN	0.356	0.349	0.347	0.342	0.360	0.349	0.350 ± 0.007

Subsampled to 25,000 reads

	Method	Run 1 25,000 reads	Run 2 25,000 reads	Run 3 25,000 reads	Run 4 25,000 reads	Run 5 25,000 reads	Run 6 25,000 reads	Mean ± SD runs 1–6
% gDNA
	Kraken2	0.290	0.315	0.337	0.366	0.314	0.293	0.319 ± 0.029
	Bracken	0.290	0.315	0.337	0.366	0.314	0.293	0.319 ± 0.029
	BASEN-nf	0.328	0.319	0.332	0.319	0.330	0.324	0.325 ± 0.006
	BASEN	0.330	0.321	0.333	0.320	0.331	0.326	0.327 ± 0.006
% genome copies
	Kraken2	0.323	0.358	0.366	0.408	0.338	0.314	0.351 ± 0.034
	Bracken	0.323	0.358	0.367	0.408	0.338	0.314	0.351 ± 0.034
	BASEN-nf	0.354	0.345	0.358	0.347	0.356	0.350	0.352 ± 0.005
	BASEN	0.356	0.347	0.359	0.348	0.357	0.352	0.353 ± 0.005

• Mean L1 to % gDNA was 0.320–0.324 for the four methods at full depth and 0.319–0.327 at 25,000 reads, indicating comparable accuracy across this depth range. The SD over runs was 0.006–0.007 for BASEN-nf and BASEN against 0.028 for Kraken2 and Bracken at full depth, and 0.006 against 0.029 at 25,000 reads: the reduction in error variation therefore persisted after depth standardization.

2. Oral: per-species variation across runs

CV (coefficient of variation) = SD / mean of a species' estimate across the six runs (full depth): relative variation in abundance estimates across runs (0.10 = 10%). Bold: lowest CV in each row; all ties are shown in bold. Genome: the reference genome length given to BASEN, in Mb. Species are listed in descending order of % gDNA.

Species	Genome (Mb)	Kraken2	Bracken	BASEN-nf	BASEN
S. parasanguinis	2.14	0.07	0.07	0.03	0.03
N. subflava	2.20	0.10	0.10	0.03	0.03
V. parvula	2.18	0.05	0.05	0.01	0.01
P. nigrescens	2.68	0.03	0.03	0.02	0.02
H. parainfluenzae	2.14	0.08	0.08	0.07	0.07
S. mitis	1.87	0.03	0.03	0.04	0.04
S. odontolytica	2.46	0.15	0.15	0.11	0.11
R. dentocariosa	2.46	0.11	0.11	0.10	0.10
F. nucleatum	2.18	0.25	0.25	0.26	0.26
S. mutans	2.02	0.11	0.11	0.16	0.16
S. salivarius	2.20	0.18	0.18	0.19	0.19
P. gingivalis	2.40	0.06	0.06	0.11	0.11
Median		0.09	0.09	0.08	0.08

• Species-level variation differed little between methods: BASEN-nf or BASEN had a lower CV than both read-based summaries for 7 of the 12 species, with a median CV of 0.08 compared with 0.09. The lower run-to-run SD of BASEN's L1 (section 1) therefore does not imply a comparable reduction in variation for every species. The reduction is primarily attributable to the four most abundant species (16% of the DNA each), for which BASEN yielded CV values of 0.01–0.03 (read counts 0.03–0.10); these species contribute most strongly to L1, which sums absolute abundance differences, whereas the 1% species, for which BASEN showed greater relative variation, contribute less.

3. Run-to-run variation in all three standards

SD of L1 over the six runs at full depth, for every standard (the per-run values are on the Genome-size details page). Microbial and Gut runs are separate Flongle flow cells with low read counts, so between-run variation reflects differences in depth, flow cell and handling; Oral is the only standard whose runs differed primarily in depth. Bold: lowest SD in each row; all ties are shown in bold.

Standard	Composition	Kraken2	Bracken	BASEN-nf	BASEN	Reads per run
Oral	% gDNA	0.028	0.028	0.006	0.007	30,780–165,929
	% genome copies	0.034	0.034	0.006	0.007
Microbial	% gDNA	0.164	0.333	0.124	0.129	105–11,351
	% genome copies	0.059	0.236	0.116	0.117
Gut	% gDNA	0.136	0.135	0.219	0.222	365–7,688
	% genome copies	0.138	0.136	0.228	0.222

• In the high-depth Oral runs, the SD of BASEN's error was fourfold lower than that of the read-based methods (SD 0.007 vs 0.028 to % gDNA). In the low-depth Microbial and Gut runs, BASEN showed no consistent stability advantage: its SD was lower only for Microbial % gDNA (0.129 vs 0.164) and higher otherwise (e.g., Gut % genome copies 0.222 vs 0.138).

4. Summary

• Accuracy: With genome sizes within 1.4-fold, the four methods achieved comparable accuracy (mean L1 to % gDNA 0.320–0.324), at full depth and at 25,000 reads.
• Stability at high depth: The SD of BASEN's L1 error was fourfold lower across the Oral runs, including after subsampling every run to 25,000 reads. Species-level variation was comparable (median CV 0.08 vs 0.09 for Kraken2; lower for 7 of 12 species).
• Stability at low depth: In the Microbial and Gut runs (hundreds to a few thousand reads) BASEN showed no consistent stability advantage over read-count methods.