Skip to content

ingest

Usage:

$ ingest [OPTIONS] COMMAND [ARGS]...

Options:

  • --version
  • --install-completion: Install completion for the current shell.
  • --show-completion: Show completion for the current shell, to copy it or customize the installation.
  • --help: Show this message and exit.

Commands:

  • closure
  • download: Downloads data defined in download.yaml
  • export
  • jsonl
  • merge: Merge nodes and edges into kg
  • neo4j-csv
  • release: Copy data to Monarch GCP data buckets
  • report: Run Koza QC on specified Monarch ingests
  • solr
  • sqlite
  • transform: Run Koza transformation on specified...

ingest closure

Usage:

$ ingest closure [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest download

Downloads data defined in download.yaml

Usage:

$ ingest download [OPTIONS]

Options:

  • -i, --ingest TEXT: Run a single ingest (see download.yaml for a list)
  • --ingests TEXT: Which ingests to download data for
  • --ingest_file PATH: A yaml file which has a newline seperated list of ingests to perform.
  • --all / --no-all: Download all ingest datasets [default: no-all]
  • --write-metadata / --no-write-metadata: Write versions of ingests to metadata.yaml [default: no-write-metadata]
  • --help: Show this message and exit.

ingest export

Usage:

$ ingest export [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest jsonl

Usage:

$ ingest jsonl [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest merge

Merge nodes and edges into kg

Usage:

$ ingest merge [OPTIONS]

Options:

  • --input-dir TEXT: Directory with nodes and edges to be merged [default: output/transform_output]
  • --output-dir TEXT: Directory to output data [default: output]
  • -d, --debug / -q, --quiet: Use --quiet to suppress log output, --debug for verbose
  • --closure / --no-closure: Apply closure to merged graph [default: no-closure]
  • --kg-name, --kg_name TEXT: The name of the kg being produced. Merge artificat will be ultimately be stored in output/$KG_NAME.tar.gz [default: monarch-kg]
  • --help: Show this message and exit.

ingest neo4j-csv

Usage:

$ ingest neo4j-csv [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest release

Copy data to Monarch GCP data buckets

Usage:

$ ingest release [OPTIONS]

Options:

  • --dir TEXT: Directory with kg to be released [default: output]
  • --kghub / --no-kghub: Also release to kghub S3 bucket [default: no-kghub]
  • --help: Show this message and exit.

ingest report

Run Koza QC on specified Monarch ingests

Usage:

$ ingest report [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest solr

Usage:

$ ingest solr [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest sqlite

Usage:

$ ingest sqlite [OPTIONS]

Options:

  • --help: Show this message and exit.

ingest transform

Run Koza transformation on specified Monarch ingests

Usage:

$ ingest transform [OPTIONS]

Options:

  • -o, --output-dir TEXT: Directory to output data [default: output]
  • -i, --ingest TEXT: Run a single ingest (see ingests.yaml for a list)
  • --ingests TEXT: Which ingests to download data for
  • --ingest_file PATH: A yaml file which has a newline seperated list of ingests to perform.
  • --phenio / --no-phenio: Run the phenio transform [default: no-phenio]
  • -a, --all: Ingest all sources
  • -f, --force: Force ingest, even if output exists (on by default for single ingests)
  • --rdf / --no-rdf: Output rdf files along with tsv [default: no-rdf]
  • -d, --debug / -q, --quiet: Use --quiet to suppress log output, --debug for verbose, including Koza logs
  • -l, --log: Write DEBUG level logs to ./logs/ for each ingest
  • -n, --row-limit INTEGER: Number of rows to process
  • --write-metadata / --no-write-metadata: Write data/package versions to output_dir/metadata.yaml [default: no-write-metadata]
  • --help: Show this message and exit.