Skip to content

Benchmark generator

BenchmarkRunOutputGenerator dataclass

Base class for recording data required for generating benchmarking outputs.

Attributes:

Name Type Description
prioritisation_type_file_prefix str

Prefix for the prioritisation type output file.
y_label str

Label for the y-axis in benchmarking outputs.
generate_benchmark_run_results Callable

Callable to generate benchmark run results. Takes parameters: input and results directory, score order, threshold, rank comparison, and returns BenchmarkRunResults.
stats_comparison_file_suffix str

Suffix for the rank comparison file.
Source code in src/pheval/analyse/benchmark_generator.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
@dataclass
class BenchmarkRunOutputGenerator:
    """Base class for recording data required for generating benchmarking outputs.

    Attributes:
        prioritisation_type_file_prefix (str): Prefix for the prioritisation type output file.
        y_label (str): Label for the y-axis in benchmarking outputs.
        generate_benchmark_run_results (Callable): Callable to generate benchmark run results.
            Takes parameters: input and results directory, score order, threshold, rank comparison,
            and returns BenchmarkRunResults.
        stats_comparison_file_suffix (str): Suffix for the rank comparison file.
    """

    prioritisation_type_file_prefix: str
    y_label: str
    generate_benchmark_run_results: Callable[
        [TrackInputOutputDirectories, str, float, defaultdict], BenchmarkRunResults
    ]
    stats_comparison_file_suffix: str

DiseaseBenchmarkRunOutputGenerator dataclass

Bases: BenchmarkRunOutputGenerator

Subclass of BenchmarkRunOutputGenerator specialised for producing disease prioritisation benchmarking outputs.

This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes specifically for disease prioritisation benchmarking.

Attributes:

Name Type Description
prioritisation_type_file_prefix str

Prefix for the disease prioritisation type file. Defaults to DISEASE_PLOT_FILE_PREFIX.
y_label str

Label for the y-axis in disease prioritisation benchmarking outputs. Defaults to DISEASE_PLOT_Y_LABEL.
generate_benchmark_run_results Callable

Callable to generate disease prioritisation benchmark run results. Defaults to benchmark_disease_prioritisation. Takes parameters: input and results directory, score order, threshold, rank comparison, and returns BenchmarkRunResults.
stats_comparison_file_suffix str

Suffix for the disease rank comparison file. Defaults to "-disease_summary.tsv".
Source code in src/pheval/analyse/benchmark_generator.py 
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
@dataclass
class DiseaseBenchmarkRunOutputGenerator(BenchmarkRunOutputGenerator):
    """
    Subclass of BenchmarkRunOutputGenerator specialised
    for producing disease prioritisation benchmarking outputs.

    This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes
    specifically for disease prioritisation benchmarking.

    Attributes:
        prioritisation_type_file_prefix (str): Prefix for the disease prioritisation type file.
            Defaults to DISEASE_PLOT_FILE_PREFIX.
        y_label (str): Label for the y-axis in disease prioritisation benchmarking outputs.
            Defaults to DISEASE_PLOT_Y_LABEL.
        generate_benchmark_run_results (Callable): Callable to generate disease prioritisation
            benchmark run results. Defaults to benchmark_disease_prioritisation.
            Takes parameters: input and results directory, score order, threshold, rank comparison,
            and returns BenchmarkRunResults.
        stats_comparison_file_suffix (str): Suffix for the disease rank comparison file.
            Defaults to "-disease_summary.tsv".
    """

    prioritisation_type_file_prefix: str = DISEASE_PLOT_FILE_PREFIX
    y_label: str = DISEASE_PLOT_Y_LABEL
    generate_benchmark_run_results: Callable[
        [TrackInputOutputDirectories, str, float, defaultdict], BenchmarkRunResults
    ] = benchmark_disease_prioritisation
    stats_comparison_file_suffix: str = "-disease_summary.tsv"

GeneBenchmarkRunOutputGenerator dataclass

Bases: BenchmarkRunOutputGenerator

Subclass of BenchmarkRunOutputGenerator specialised for producing gene prioritisation benchmarking outputs.

This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes specifically for gene prioritisation benchmarking.

Attributes:

Name Type Description
prioritisation_type_file_prefix str

Prefix for the gene prioritisation type file. Defaults to GENE_PLOT_FILE_PREFIX.
y_label str

Label for the y-axis in gene prioritisation benchmarking outputs. Defaults to GENE_PLOT_Y_LABEL.
generate_benchmark_run_results Callable

Callable to generate gene prioritisation benchmark run results. Defaults to benchmark_gene_prioritisation. Takes parameters: input and results directory, score order, threshold, rank comparison, and returns BenchmarkRunResults.
stats_comparison_file_suffix str

Suffix for the gene rank comparison file. Defaults to "-gene_summary.tsv".
Source code in src/pheval/analyse/benchmark_generator.py 
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
@dataclass
class GeneBenchmarkRunOutputGenerator(BenchmarkRunOutputGenerator):
    """
    Subclass of BenchmarkRunOutputGenerator specialised
    for producing gene prioritisation benchmarking outputs.

    This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes
    specifically for gene prioritisation benchmarking.

    Attributes:
        prioritisation_type_file_prefix (str): Prefix for the gene prioritisation type file.
            Defaults to GENE_PLOT_FILE_PREFIX.
        y_label (str): Label for the y-axis in gene prioritisation benchmarking outputs.
            Defaults to GENE_PLOT_Y_LABEL.
        generate_benchmark_run_results (Callable): Callable to generate gene prioritisation
            benchmark run results. Defaults to benchmark_gene_prioritisation.
            Takes parameters: input and results directory, score order, threshold, rank comparison,
            and returns BenchmarkRunResults.
        stats_comparison_file_suffix (str): Suffix for the gene rank comparison file.
            Defaults to "-gene_summary.tsv".
    """

    prioritisation_type_file_prefix: str = GENE_PLOT_FILE_PREFIX
    y_label: str = GENE_PLOT_Y_LABEL
    generate_benchmark_run_results: Callable[
        [TrackInputOutputDirectories, str, float, defaultdict], BenchmarkRunResults
    ] = benchmark_gene_prioritisation
    stats_comparison_file_suffix: str = "-gene_summary.tsv"

VariantBenchmarkRunOutputGenerator dataclass

Bases: BenchmarkRunOutputGenerator

Subclass of BenchmarkRunOutputGenerator specialised for producing variant prioritisation benchmarking outputs.

This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes specifically for variant prioritisation benchmarking.

Attributes:

Name Type Description
prioritisation_type_file_prefix str

Prefix for the variant prioritisation type file. Defaults to VARIANT_PLOT_FILE_PREFIX.
y_label str

Label for the y-axis in variant prioritisation benchmarking outputs. Defaults to VARIANT_PLOT_Y_LABEL.
generate_benchmark_run_results Callable

Callable to generate variant prioritisation benchmark run results. Defaults to benchmark_variant_prioritisation. Takes parameters: input and results directory, score order, threshold, rank comparison, and returns BenchmarkRunResults.
stats_comparison_file_suffix str

Suffix for the variant rank comparison file. Defaults to "-variant_summary.tsv".
Source code in src/pheval/analyse/benchmark_generator.py 
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
@dataclass
class VariantBenchmarkRunOutputGenerator(BenchmarkRunOutputGenerator):
    """
    Subclass of BenchmarkRunOutputGenerator specialised
    for producing variant prioritisation benchmarking outputs.

    This subclass inherits from BenchmarkRunOutputGenerator and specialises its attributes
    specifically for variant prioritisation benchmarking.

    Attributes:
        prioritisation_type_file_prefix (str): Prefix for the variant prioritisation type file.
            Defaults to VARIANT_PLOT_FILE_PREFIX.
        y_label (str): Label for the y-axis in variant prioritisation benchmarking outputs.
            Defaults to VARIANT_PLOT_Y_LABEL.
        generate_benchmark_run_results (Callable): Callable to generate variant prioritisation
            benchmark run results. Defaults to benchmark_variant_prioritisation.
            Takes parameters: input and results directory, score order, threshold, rank comparison,
            and returns BenchmarkRunResults.
        stats_comparison_file_suffix (str): Suffix for the variant rank comparison file.
            Defaults to "-variant_summary.tsv".

    """

    prioritisation_type_file_prefix: str = VARIANT_PLOT_FILE_PREFIX
    y_label: str = VARIANT_PLOT_Y_LABEL
    generate_benchmark_run_results: Callable[
        [TrackInputOutputDirectories, str, float, defaultdict], BenchmarkRunResults
    ] = benchmark_variant_prioritisation
    stats_comparison_file_suffix: str = "-variant_summary.tsv"