Five-Year GO Impact Report
Source notebook: ../../notebooks/03_go_diff_with_duckdb.ipynb
This tutorial treats GO temporal change analysis as a configured convenience report. The report command runs the two enrichments, performs the threshold-crossing comparison, writes Parquet plus YAML/JSON metadata, and records the timing and parameters used. The notebook then inspects those report artifacts with DuckDB instead of rebuilding the workflow step by step.
uv run --project python/genesets-workflows genesets-workflows go-impact evals/go_impact_5y_expression500.yaml
Output:
Wrote notebooks/generated/go_impact_5y_expression500/summary.yaml
Diff rows: gained_significant=23378, lost_significant=954, shared_significant=2760
Timings: old=3.925s, new=5.114s, compare=0.174s, total=9.744s
Output:
Output:
| parameter | value |
|---|---|
| config | evals/go_impact_5y_expression500.yaml |
| query source | evals/expression_like/generated/msigdb_gse_500 |
| query limit | 500 |
| left snapshot | 2021-05-01 (all) |
| right snapshot | 2026-03-25 (all) |
| min overlap | 2 |
| adjusted p cutoff | 0.05 |
| ranked term scope | goslim_generic descendants from right snapshot |
| max ranked term size | 1,000 |
Output:
| artifact | path |
|---|---|
| diff_metadata | notebooks/generated/go_impact_5y_expression500/go_2021_05_01_vs_go_2026_03_25_expression500.diff.yaml |
| diff_results | notebooks/generated/go_impact_5y_expression500/go_2021_05_01_vs_go_2026_03_25_expression500.diff.parquet |
| new_results | notebooks/generated/go_impact_5y_expression500/go_2026_03_25_expression500.parquet |
| old_results | notebooks/generated/go_impact_5y_expression500/go_2021_05_01_expression500.parquet |
| summary_json | notebooks/generated/go_impact_5y_expression500/summary.json |
| summary_yaml | notebooks/generated/go_impact_5y_expression500/summary.yaml |
Output:
| stage | seconds |
|---|---|
| select 500 expression sets | 0.006 |
| 2021-05-01 GO enrichment | 3.925 |
| 2026-03-25 GO enrichment | 5.114 |
| threshold-crossing compare | 0.174 |
| total report | 9.744 |
Output:
| class | row_count | affected_query_sets | affected_go_terms |
|---|---|---|---|
| gained_significant | 23378 | 500 | 1067 |
| shared_significant | 2760 | 204 | 494 |
| lost_significant | 954 | 128 | 235 |
For each high-impact expression signature, the term columns identify the GO term with the largest signed change in -log10(Bonferroni adjusted p) after applying the configured ranked-term scope. In this report, candidates are current GO terms in goslim_generic or descendants of a goslim_generic term, with at most 1,000 annotated genes in the relevant snapshot. Positive values mean stronger enrichment in 2026; negative values mean weaker enrichment. For gained/lost calls, the non-significant side is anchored at the report cutoff (0.05) because this report stores significant-only result tables.
This is an inclusion slim plus a size guard. The complementary antislim is the same idea with the go_term_filter join inverted, which is useful when the goal is to explicitly remove a known broad region of the ontology from a report.
Output:
| query_id | gained_calls | lost_calls | total_crossing_calls | largest_crossing_term | crossing_status | largest_crossing_delta | largest_overall_term | overall_status | largest_overall_delta |
|---|---|---|---|---|---|---|---|---|---|
| GSE29618_BCELL_VS_MDC_DAY7_FLU_VACCINE_DN | 177 | 31 | 208 | GO:0002366 leukocyte activation involved in immune response | lost | -7.35 | GO:0030141 secretory granule | shared | +8.37 |
| GSE29618_PDC_VS_MDC_DAY7_FLU_VACCINE_DN | 178 | 27 | 205 | GO:0002366 leukocyte activation involved in immune response | lost | -9.80 | GO:0002252 immune effector process | shared | -11.55 |
| GSE29618_PDC_VS_MDC_DN | 167 | 32 | 199 | GO:0002366 leukocyte activation involved in immune response | lost | -17.21 | GO:0002366 leukocyte activation involved in immune response | lost | -17.21 |
| GSE29618_MONOCYTE_VS_PDC_UP | 158 | 26 | 184 | GO:0042119 neutrophil activation | lost | -22.09 | GO:0002274 myeloid leukocyte activation | shared | -22.42 |
| GSE29618_BCELL_VS_MONOCYTE_DN | 143 | 26 | 169 | GO:0002283 neutrophil activation involved in immune response | lost | -26.45 | GO:0002274 myeloid leukocyte activation | shared | -26.64 |
| GSE28726_NAIVE_VS_ACTIVATED_CD4_TCELL_DN | 133 | 36 | 169 | GO:0060205 cytoplasmic vesicle lumen | gained | +4.92 | GO:0000278 mitotic cell cycle | shared | +8.56 |
| GSE29164_UNTREATED_VS_CD8_TCELL_TREATED_MELANOMA_DAY7_UP | 138 | 21 | 159 | GO:0006954 inflammatory response | gained | +4.89 | GO:0006954 inflammatory response | gained | +4.89 |
| GSE29618_MONOCYTE_VS_MDC_UP | 127 | 27 | 154 | GO:0042119 neutrophil activation | lost | -32.05 | GO:0042119 neutrophil activation | lost | -32.05 |
| GSE29618_BCELL_VS_MDC_DN | 117 | 35 | 152 | GO:0002252 immune effector process | lost | -13.79 | GO:0002252 immune effector process | lost | -13.79 |
| GSE28726_NAIVE_CD4_TCELL_VS_NAIVE_VA24NEG_NKTCELL_UP | 123 | 22 | 145 | GO:0006281 DNA repair | gained | +5.33 | GO:0000278 mitotic cell cycle | shared | +13.66 |
| GSE29618_MONOCYTE_VS_PDC_DAY7_FLU_VACCINE_UP | 115 | 26 | 141 | GO:0002274 myeloid leukocyte activation | lost | -16.20 | GO:0002274 myeloid leukocyte activation | lost | -16.20 |
| GSE29618_BCELL_VS_MONOCYTE_DAY7_FLU_VACCINE_DN | 110 | 29 | 139 | GO:0002283 neutrophil activation involved in immune response | lost | -16.97 | GO:0002283 neutrophil activation involved in immune response | lost | -16.97 |
| GSE27859_MACROPHAGE_VS_CD11C_INT_F480_HI_MACROPHAGE_UP | 128 | 7 | 135 | GO:0002218 activation of innate immune response | gained | +4.13 | GO:0002218 activation of innate immune response | gained | +4.13 |
| GSE32034_LY6C_HIGH_VS_LOW_ROSIGLIZATONE_TREATED_MONOCYTE_DN | 126 | 2 | 128 | GO:0030595 leukocyte chemotaxis | gained | +4.12 | GO:0006954 inflammatory response | shared | +5.98 |
| GSE11057_PBMC_VS_MEM_CD4_TCELL_UP | 111 | 17 | 128 | GO:0002366 leukocyte activation involved in immune response | lost | -11.30 | GO:0002366 leukocyte activation involved in immune response | lost | -11.30 |
| GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | 100 | 28 | 128 | GO:0042119 neutrophil activation | lost | -32.99 | GO:0042119 neutrophil activation | lost | -32.99 |
| GSE12845_NAIVE_VS_PRE_GC_TONSIL_BCELL_DN | 97 | 27 | 124 | GO:1903047 mitotic cell cycle process | gained | +9.84 | GO:1903047 mitotic cell cycle process | gained | +9.84 |
| GSE2826_WT_VS_BTK_KO_BCELL_DN | 119 | 4 | 123 | GO:0030097 hemopoiesis | gained | +6.38 | GO:0030097 hemopoiesis | gained | +6.38 |
| GSE29618_BCELL_VS_PDC_UP | 117 | 5 | 122 | GO:0045321 leukocyte activation | gained | +9.80 | GO:0045321 leukocyte activation | gained | +9.80 |
| GSE30971_WBP7_HET_VS_KO_MACROPHAGE_DN | 111 | 10 | 121 | GO:0046649 lymphocyte activation | gained | +2.72 | GO:1903131 mononuclear cell differentiation | shared | +2.92 |
| GSE29618_BCELL_VS_PDC_DAY7_FLU_VACCINE_DN | 116 | 2 | 118 | GO:0000323 lytic vacuole | gained | +3.72 | GO:0000323 lytic vacuole | gained | +3.72 |
| GSE2935_UV_INACTIVATED_VS_LIVE_SENDAI_VIRUS_INF_MACROPHAGE_DN | 114 | 4 | 118 | GO:0006954 inflammatory response | gained | +4.94 | GO:0030097 hemopoiesis | shared | +6.15 |
| GSE30971_WBP7_HET_VS_KO_MACROPHAGE_2H_LPS_STIM_DN | 107 | 11 | 118 | GO:0007259 cell surface receptor signaling pathway via JAK-STAT | gained | +5.49 | GO:0007259 cell surface receptor signaling pathway via JAK-STAT | gained | +5.49 |
| GSE12845_PRE_GC_VS_DARKZONE_GC_TONSIL_BCELL_UP | 114 | 2 | 116 | GO:0140546 defense response to symbiont | gained | +5.14 | GO:0140546 defense response to symbiont | gained | +5.14 |
| GSE3039_NKT_CELL_VS_B2_BCELL_DN | 111 | 4 | 115 |
The next table asks which GO terms changed status across the largest number of expression signatures. It reuses the configured goslim_generic descendant scope and adds a stricter display guard of at most 500 genes so the table stays biologically interpretable.
Output:
| class | target_id | target_name | target_size | affected_query_sets |
|---|---|---|---|---|
| gained_significant | GO:0009897 | external side of plasma membrane | 392 | 25 |
| gained_significant | GO:0046649 | lymphocyte activation | 414 | 21 |
| gained_significant | GO:0002764 | immune response-regulating signaling pathway | 468 | 20 |
| gained_significant | GO:0002521 | leukocyte differentiation | 339 | 18 |
| gained_significant | GO:1903131 | mononuclear cell differentiation | 294 | 16 |
| gained_significant | GO:0006954 | inflammatory response | 447 | 16 |
| gained_significant | GO:0002768 | immune response-regulating cell surface receptor signaling pathway | 380 | 15 |
| gained_significant | GO:0030098 | lymphocyte differentiation | 223 | 14 |
| gained_significant | GO:0002757 | immune response-activating signaling pathway | 398 | 14 |
| gained_significant | GO:0006897 | endocytosis | 429 | 14 |
| gained_significant | GO:0002253 | activation of immune response | 466 | 14 |
| lost_significant | GO:0043312 | neutrophil degranulation | 482 | 14 |
| lost_significant | GO:0002283 | neutrophil activation involved in immune response | 488 | 14 |
| lost_significant | GO:0002446 | neutrophil mediated immunity | 496 | 14 |
| lost_significant | GO:0042119 | neutrophil activation | 498 | 14 |
| gained_significant | GO:0006281 | DNA repair | 475 | 13 |
| gained_significant | GO:0042254 | ribosome biogenesis | 311 | 11 |
| gained_significant | GO:0098687 | chromosomal region | 399 | 11 |
| gained_significant | GO:0006261 | DNA-templated DNA replication | 162 | 10 |
| gained_significant | GO:0060205 | cytoplasmic vesicle lumen | 331 | 10 |
| gained_significant | GO:0031983 | vesicle lumen | 332 | 10 |
| gained_significant | GO:0005819 | spindle | 441 | 10 |
| gained_significant | GO:0005765 | lysosomal membrane | 500 | 10 |
| gained_significant | GO:0098852 | lytic vacuole membrane | 500 | 10 |
| gained_significant | GO:0042110 | T cell activation | 278 | 9 |
| gained_significant | GO:1904813 | ficolin-1-rich granule lumen | 125 | 8 |
| gained_significant | GO:0006310 | DNA recombination | 241 | 8 |
| gained_significant | GO:0000228 | nuclear chromosome | 242 | 8 |
| gained_significant | GO:0034774 | secretory granule lumen | 326 | 8 |
| gained_significant | GO:0030183 | B cell differentiation | 96 | 7 |
Output:
| class | query_id | target_id | target_name | signed_delta_score | abs_delta_score | left_p_adjust | right_p_adjust | left_overlap | right_overlap | left_target_size | right_target_size |
|---|---|---|---|---|---|---|---|---|---|---|---|
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0042119 | neutrophil activation | -32.99 | 32.99 | 5.068e-35 | 57 | 498 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0036230 | granulocyte activation | -32.75 | 32.75 | 8.905e-35 | 57 | 503 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0043312 | neutrophil degranulation | -32.59 | 32.59 | 1.292e-34 | 56 | 482 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002274 | myeloid leukocyte activation | -32.32 | 32.32 | 2.383e-34 | 60 | 589 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002283 | neutrophil activation involved in immune response | -32.29 | 32.29 | 2.568e-34 | 56 | 488 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0042119 | neutrophil activation | -32.05 | 32.05 | 4.497e-34 | 56 | 498 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002446 | neutrophil mediated immunity | -31.90 | 31.90 | 6.323e-34 | 56 | 496 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0036230 | granulocyte activation | -31.81 | 31.81 | 7.815e-34 | 56 | 503 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002275 | myeloid cell activation involved in immune response | -31.57 | 31.57 | 1.360e-33 | 57 | 528 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0043299 | leukocyte degranulation | -31.37 | 31.37 | 2.125e-33 | 56 | 507 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002444 | myeloid leukocyte mediated immunity | -30.72 | 30.72 | 9.523e-33 | 56 | 521 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002283 | neutrophil activation involved in immune response | -30.15 | 30.15 | 3.524e-32 | 54 | 488 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002366 | leukocyte activation involved in immune response | -29.31 | 29.31 | 2.424e-31 | 59 | 635 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0043312 | neutrophil degranulation | -29.26 | 29.26 | 2.759e-31 | 53 | 482 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0045055 | regulated exocytosis | -29.16 | 29.16 | 3.437e-31 | 61 | 697 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002263 | cell activation involved in immune response | -29.16 | 29.16 | 3.461e-31 | 59 | 639 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002446 | neutrophil mediated immunity | -28.61 | 28.61 | 1.225e-30 | 53 | 496 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002275 | myeloid cell activation involved in immune response | -28.34 | 28.34 | 2.279e-30 | 54 | 528 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002263 | cell activation involved in immune response | -28.32 | 28.32 | 2.406e-30 | 58 | 639 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0043299 | leukocyte degranulation | -28.12 | 28.12 | 3.822e-30 | 53 | 507 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0045321 | leukocyte activation | -27.82 | 27.82 | 7.515e-30 | 67 | 930 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002444 | myeloid leukocyte mediated immunity | -27.51 | 27.51 | 1.562e-29 | 53 | 521 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_UP | GO:0002366 | leukocyte activation involved in immune response | -27.38 | 27.38 | 2.083e-29 | 57 | 635 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0006887 | exocytosis | -27.17 | 27.17 | 3.366e-29 | 62 | 786 | |||
| lost | GSE29618_BCELL_VS_MONOCYTE_DN | GO:0002283 | neutrophil activation involved in immune response | -26.45 | 26.45 | 1.780e-28 | 51 | 488 | |||
| lost | GSE29618_BCELL_VS_MONOCYTE_DN | GO:0042119 | neutrophil activation | -26.01 | 26.01 | 4.876e-28 | 51 | 498 | |||
| lost | GSE29618_BCELL_VS_MONOCYTE_DN | GO:0036230 | granulocyte activation | -25.80 | 25.80 | 8.000e-28 | 51 | 503 | |||
| lost | GSE29618_BCELL_VS_MONOCYTE_DN | GO:0002446 | neutrophil mediated immunity | -24.97 | 24.97 | 5.319e-27 | 50 | 496 | |||
| lost | GSE29618_MONOCYTE_VS_MDC_DAY7_FLU_VACCINE_UP | GO:0002443 | leukocyte mediated immunity | -24.91 | 24.91 | 6.146e-27 | 59 | 761 | |||
| lost | GSE29618_BCELL_VS_MONOCYTE_DN | GO:0002275 | myeloid cell activation involved in immune response | -24.76 | 24.76 | 8.763e-27 | 51 | 528 |