Preprocessing

cellarium.ml.preprocessing.kotliar_compute_highly_variable_genes(var_names_g: list | ndarray, mean_g: ndarray | Tensor, var_g: ndarray | Tensor, n_top_genes: int | None = 2000, expected_fano_threshold: float | None = None, minimal_mean: float = 0.5, plot: bool = False)[source]

Helper function to run the highly variable gene selection procedure from Kotliar et al. 2019, implemented in the function get_highvar_genes_sparse in the dylkot/cNMF repository. Modified to work in cellarium based on a run of a onepass_mean_var_std model on the data.

NOTE: taken from https://github.com/dylkot/cNMF/blob/5dbc5baaa0b9079b55bce554d801caa235a50457/src/cnmf/cnmf.py#L136-L188

Parameters:

  • mean_g (ndarray | Tensor) – The mean expression levels of genes

  • var_g (ndarray | Tensor) – The variance of expression levels of genes

  • var_names_g (list | ndarray) – The names of the genes

  • n_top_genes (int | None) – The number of highly variable genes to select. If None, uses a threshold-based approach

  • expected_fano_threshold (float | None) – If n_top_genes is None, this threshold is used to select highly variable genes based on their Fano factor relative to the expected Fano factor. If None, a default threshold is computed based on the standard deviation of the Fano factors of genes that pass a winsorized box filter.

  • minimal_mean (float) – The minimum mean expression level for a gene to be considered highly variable. This is used only in the threshold-based approach (i.e. when n_top_genes is None)

  • plot (bool) – Whether to plot the mean-variance relationship and the Fano factor distribution. Useful for debugging.

Returns:

A DataFrame with columns - mean: The mean expression level of each gene - var: The variance of each gene - fano: The Fano factor of each gene (variance / mean) - fano_fit: The expected Fano factor of each gene based on the fitted line - fano_ratio: The ratio of the observed Fano factor to the expected Fano factor - highly_variable: A boolean indicating whether the gene is selected as highly variable

cellarium.ml.preprocessing.seurat_compute_highly_variable_genes(var_names_g: list | ndarray, mean_g: Tensor, var_g: Tensor, n_top_genes: int | None = None, min_disp: float | None = 0.5, max_disp: float | None = inf, min_mean: float | None = 0.0125, max_mean: float | None = 3, n_bins: int = 20, batch_mean_bg: Tensor | None = None, batch_var_bg: Tensor | None = None, batch_ids: list[str] | None = None) → DataFrame[source]

Annotate highly variable genes using the seurat flavor.

Replicates scanpy.pp.highly_variable_genes with flavor='seurat'. Optionally accepts per-batch statistics for batch-aware selection.

References:

  1. Highly Variable Genes from Scanpy.

Parameters:

  • var_names_g (list | ndarray) – Ensembl gene ids.

  • mean_g (Tensor) – Overall gene expression means in count space (shape n_genes).

  • var_g (Tensor) – Overall gene expression variances in count space (shape n_genes).

  • n_top_genes (int | None) – Number of highly-variable genes to keep.

  • min_disp (float | None) – Ignored when n_top_genes is set.

  • max_disp (float | None) – Ignored when n_top_genes is set.

  • min_mean (float | None) – Ignored when n_top_genes is set.

  • max_mean (float | None) – Ignored when n_top_genes is set.

  • n_bins (int) – Number of bins for mean-expression binning.

  • batch_mean_bg (Tensor | None) – Per-batch means in count space of shape (n_batch, n_genes).

  • batch_var_bg (Tensor | None) – Per-batch variances in count space of shape (n_batch, n_genes).

  • batch_ids (list[str] | None) – Batch labels of length n_batch.

Returns:

DataFrame indexed by var_names_g with columns highly_variable, means, dispersions, dispersions_norm, mean_bin (single-batch), highly_variable_nbatches and highly_variable_intersection (batch mode).

Return type:

DataFrame