Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Five-Year GO Impact Report

Source notebook: ../../notebooks/03_go_diff_with_duckdb.ipynb

This tutorial treats GO temporal change analysis as a configured convenience report. The report command runs the two enrichments, performs the threshold-crossing comparison, writes Parquet plus YAML/JSON metadata, and records the timing and parameters used. The notebook then inspects those report artifacts with DuckDB instead of rebuilding the workflow step by step.

uv run --project python/genesets-workflows genesets-workflows go-impact evals/go_impact_5y_expression500.yaml

Output:

Wrote notebooks/generated/go_impact_5y_expression500/summary.yaml
Diff rows: gained_significant=23378, lost_significant=954, shared_significant=2760
Timings: old=3.925s, new=5.114s, compare=0.174s, total=9.744s

Output:

500expression gene sets
26,138new significant rows
23,378gained calls
954lost calls
21,995rankable GO terms
9.74stotal runtime

Output:

Report parameters
parameter value
config evals/go_impact_5y_expression500.yaml
query source evals/expression_like/generated/msigdb_gse_500
query limit 500
left snapshot 2021-05-01 (all)
right snapshot 2026-03-25 (all)
min overlap 2
adjusted p cutoff 0.05
ranked term scope goslim_generic descendants from right snapshot
max ranked term size 1,000

Output:

Generated artifacts
artifact path
diff_metadata notebooks/generated/go_impact_5y_expression500/go_2021_05_01_vs_go_2026_03_25_expression500.diff.yaml
diff_results notebooks/generated/go_impact_5y_expression500/go_2021_05_01_vs_go_2026_03_25_expression500.diff.parquet
new_results notebooks/generated/go_impact_5y_expression500/go_2026_03_25_expression500.parquet
old_results notebooks/generated/go_impact_5y_expression500/go_2021_05_01_expression500.parquet
summary_json notebooks/generated/go_impact_5y_expression500/summary.json
summary_yaml notebooks/generated/go_impact_5y_expression500/summary.yaml

Output:

Report timings on this laptop
stage seconds
select 500 expression sets 0.006
2021-05-01 GO enrichment 3.925
2026-03-25 GO enrichment 5.114
threshold-crossing compare 0.174
total report 9.744

Output:

Significance status from 2021 to 2026
class row_count affected_query_sets affected_go_terms
gained_significant 23378 500 1067
shared_significant 2760 204 494
lost_significant 954 128 235

For each high-impact expression signature, the term columns identify the GO term with the largest signed change in -log10(Bonferroni adjusted p) after applying the configured ranked-term scope. In this report, candidates are current GO terms in goslim_generic or descendants of a goslim_generic term, with at most 1,000 annotated genes in the relevant snapshot. Positive values mean stronger enrichment in 2026; negative values mean weaker enrichment. For gained/lost calls, the non-significant side is anchored at the report cutoff (0.05) because this report stores significant-only result tables.

This is an inclusion slim plus a size guard. The complementary antislim is the same idea with the go_term_filter join inverted, which is useful when the goal is to explicitly remove a known broad region of the ontology from a report.

Output:

Expression signatures with the most threshold crossings; largest terms use the configured ranked-term scope
query_id gained_calls lost_calls total_crossing_calls largest_crossing_term crossing_status largest_crossing_delta largest_overall_term overall_status largest_overall_delta
GSE29618_BCELL_VS_MDC_DAY7_FLU_VACCINE_DN 177 31 208 GO:0002366 leukocyte activation involved in immune response lost -7.35 GO:0030141 secretory granule shared +8.37
GSE29618_PDC_VS_MDC_DAY7_FLU_VACCINE_DN 178 27 205 GO:0002366 leukocyte activation involved in immune response lost -9.80 GO:0002252 immune effector process shared -11.55
GSE29618_PDC_VS_MDC_DN 167 32 199 GO:0002366 leukocyte activation involved in immune response lost -17.21 GO:0002366 leukocyte activation involved in immune response lost -17.21
GSE29618_MONOCYTE_VS_PDC_UP 158 26 184 GO:0042119 neutrophil activation lost -22.09 GO:0002274 myeloid leukocyte activation shared -22.42
GSE29618_BCELL_VS_MONOCYTE_DN 143 26 169 GO:0002283 neutrophil activation involved in immune response lost -26.45 GO:0002274 myeloid leukocyte activation shared -26.64
GSE28726_NAIVE_VS_ACTIVATED_CD4_TCELL_DN 133 36 169 GO:0060205 cytoplasmic vesicle lumen gained +4.92 GO:0000278 mitotic cell cycle shared +8.56
GSE29164_UNTREATED_VS_CD8_TCELL_TREATED_MELANOMA_DAY7_UP 138 21 159 GO:0006954 inflammatory response gained +4.89 GO:0006954 inflammatory response gained +4.89
GSE29618_MONOCYTE_VS_MDC_UP 127 27 154 GO:0042119 neutrophil activation lost -32.05 GO:0042119 neutrophil activation lost -32.05
GSE29618_BCELL_VS_MDC_DN 117 35 152 GO:0002252 immune effector process lost -13.79 GO:0002252 immune effector process lost -13.79
GSE28726_NAIVE_CD4_TCELL_VS_NAIVE_VA24NEG_NKTCELL_UP 123 22 145 GO:0006281 DNA repair gained +5.33 GO:0000278 mitotic cell cycle shared +13.66
GSE29618_MONOCYTE_VS_PDC_DAY7_FLU_VACCINE_UP 115 26 141 GO:0002274 myeloid leukocyte activation lost -16.20 GO:0002274 myeloid leukocyte activation lost -16.20
GSE29618_BCELL_VS_MONOCYTE_DAY7_FLU_VACCINE_DN 110 29 139 GO:0002283 neutrophil activation involved in immune response lost -16.97 GO:0002283 neutrophil activation involved in immune response lost -16.97
GSE27859_MACROPHAGE_VS_CD11C_INT_F480_HI_MACROPHAGE_UP 128 7 135 GO:0002218 activation of innate immune response gained +4.13 GO:0002218 activation of innate immune response gained +4.13
GSE32034_LY6C_HIGH_VS_LOW_ROSIGLIZATONE_TREATED_MONOCYTE_DN 126 2 128 GO:0030595 leukocyte chemotaxis gained +4.12 GO:0006954 inflammatory response shared +5.98
GSE11057_PBMC_VS_MEM_CD4_TCELL_UP 111 17 128 GO:0002366 leukocyte activation involved in immune response lost -11.30 GO:0002366 leukocyte activation involved in immune response lost -11.30
GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP 100 28 128 GO:0042119 neutrophil activation lost -32.99 GO:0042119 neutrophil activation lost -32.99
GSE12845_NAIVE_VS_PRE_GC_TONSIL_BCELL_DN 97 27 124 GO:1903047 mitotic cell cycle process gained +9.84 GO:1903047 mitotic cell cycle process gained +9.84
GSE2826_WT_VS_BTK_KO_BCELL_DN 119 4 123 GO:0030097 hemopoiesis gained +6.38 GO:0030097 hemopoiesis gained +6.38
GSE29618_BCELL_VS_PDC_UP 117 5 122 GO:0045321 leukocyte activation gained +9.80 GO:0045321 leukocyte activation gained +9.80
GSE30971_WBP7_HET_VS_KO_MACROPHAGE_DN 111 10 121 GO:0046649 lymphocyte activation gained +2.72 GO:1903131 mononuclear cell differentiation shared +2.92
GSE29618_BCELL_VS_PDC_DAY7_FLU_VACCINE_DN 116 2 118 GO:0000323 lytic vacuole gained +3.72 GO:0000323 lytic vacuole gained +3.72
GSE2935_UV_INACTIVATED_VS_LIVE_SENDAI_VIRUS_INF_MACROPHAGE_DN 114 4 118 GO:0006954 inflammatory response gained +4.94 GO:0030097 hemopoiesis shared +6.15
GSE30971_WBP7_HET_VS_KO_MACROPHAGE_2H_LPS_STIM_DN 107 11 118 GO:0007259 cell surface receptor signaling pathway via JAK-STAT gained +5.49 GO:0007259 cell surface receptor signaling pathway via JAK-STAT gained +5.49
GSE12845_PRE_GC_VS_DARKZONE_GC_TONSIL_BCELL_UP 114 2 116 GO:0140546 defense response to symbiont gained +5.14 GO:0140546 defense response to symbiont gained +5.14
GSE3039_NKT_CELL_VS_B2_BCELL_DN 111 4 115

The next table asks which GO terms changed status across the largest number of expression signatures. It reuses the configured goslim_generic descendant scope and adds a stricter display guard of at most 500 genes so the table stays biologically interpretable.

Output:

Most repeatedly affected specific GO terms within the configured ranked-term scope
class target_id target_name target_size affected_query_sets
gained_significant GO:0009897 external side of plasma membrane 392 25
gained_significant GO:0046649 lymphocyte activation 414 21
gained_significant GO:0002764 immune response-regulating signaling pathway 468 20
gained_significant GO:0002521 leukocyte differentiation 339 18
gained_significant GO:1903131 mononuclear cell differentiation 294 16
gained_significant GO:0006954 inflammatory response 447 16
gained_significant GO:0002768 immune response-regulating cell surface receptor signaling pathway 380 15
gained_significant GO:0030098 lymphocyte differentiation 223 14
gained_significant GO:0002757 immune response-activating signaling pathway 398 14
gained_significant GO:0006897 endocytosis 429 14
gained_significant GO:0002253 activation of immune response 466 14
lost_significant GO:0043312 neutrophil degranulation 482 14
lost_significant GO:0002283 neutrophil activation involved in immune response 488 14
lost_significant GO:0002446 neutrophil mediated immunity 496 14
lost_significant GO:0042119 neutrophil activation 498 14
gained_significant GO:0006281 DNA repair 475 13
gained_significant GO:0042254 ribosome biogenesis 311 11
gained_significant GO:0098687 chromosomal region 399 11
gained_significant GO:0006261 DNA-templated DNA replication 162 10
gained_significant GO:0060205 cytoplasmic vesicle lumen 331 10
gained_significant GO:0031983 vesicle lumen 332 10
gained_significant GO:0005819 spindle 441 10
gained_significant GO:0005765 lysosomal membrane 500 10
gained_significant GO:0098852 lytic vacuole membrane 500 10
gained_significant GO:0042110 T cell activation 278 9
gained_significant GO:1904813 ficolin-1-rich granule lumen 125 8
gained_significant GO:0006310 DNA recombination 241 8
gained_significant GO:0000228 nuclear chromosome 242 8
gained_significant GO:0034774 secretory granule lumen 326 8
gained_significant GO:0030183 B cell differentiation 96 7

Output:

Largest individual threshold-crossing changes within the configured ranked-term scope
class query_id target_id target_name signed_delta_score abs_delta_score left_p_adjust right_p_adjust left_overlap right_overlap left_target_size right_target_size
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0042119 neutrophil activation -32.99 32.99 5.068e-35 57 498
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0036230 granulocyte activation -32.75 32.75 8.905e-35 57 503
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0043312 neutrophil degranulation -32.59 32.59 1.292e-34 56 482
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002274 myeloid leukocyte activation -32.32 32.32 2.383e-34 60 589
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002283 neutrophil activation involved in immune response -32.29 32.29 2.568e-34 56 488
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0042119 neutrophil activation -32.05 32.05 4.497e-34 56 498
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002446 neutrophil mediated immunity -31.90 31.90 6.323e-34 56 496
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0036230 granulocyte activation -31.81 31.81 7.815e-34 56 503
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002275 myeloid cell activation involved in immune response -31.57 31.57 1.360e-33 57 528
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0043299 leukocyte degranulation -31.37 31.37 2.125e-33 56 507
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002444 myeloid leukocyte mediated immunity -30.72 30.72 9.523e-33 56 521
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002283 neutrophil activation involved in immune response -30.15 30.15 3.524e-32 54 488
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002366 leukocyte activation involved in immune response -29.31 29.31 2.424e-31 59 635
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0043312 neutrophil degranulation -29.26 29.26 2.759e-31 53 482
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0045055 regulated exocytosis -29.16 29.16 3.437e-31 61 697
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002263 cell activation involved in immune response -29.16 29.16 3.461e-31 59 639
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002446 neutrophil mediated immunity -28.61 28.61 1.225e-30 53 496
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002275 myeloid cell activation involved in immune response -28.34 28.34 2.279e-30 54 528
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002263 cell activation involved in immune response -28.32 28.32 2.406e-30 58 639
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0043299 leukocyte degranulation -28.12 28.12 3.822e-30 53 507
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0045321 leukocyte activation -27.82 27.82 7.515e-30 67 930
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002444 myeloid leukocyte mediated immunity -27.51 27.51 1.562e-29 53 521
lost GSE29618_MONOCYTE_VS_MDC_UP GO:0002366 leukocyte activation involved in immune response -27.38 27.38 2.083e-29 57 635
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0006887 exocytosis -27.17 27.17 3.366e-29 62 786
lost GSE29618_BCELL_VS_MONOCYTE_DN GO:0002283 neutrophil activation involved in immune response -26.45 26.45 1.780e-28 51 488
lost GSE29618_BCELL_VS_MONOCYTE_DN GO:0042119 neutrophil activation -26.01 26.01 4.876e-28 51 498
lost GSE29618_BCELL_VS_MONOCYTE_DN GO:0036230 granulocyte activation -25.80 25.80 8.000e-28 51 503
lost GSE29618_BCELL_VS_MONOCYTE_DN GO:0002446 neutrophil mediated immunity -24.97 24.97 5.319e-27 50 496
lost GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP GO:0002443 leukocyte mediated immunity -24.91 24.91 6.146e-27 59 761
lost GSE29618_BCELL_VS_MONOCYTE_DN GO:0002275 myeloid cell activation involved in immune response -24.76 24.76 8.763e-27 51 528