squidpy.gr.calculate_niche_utag

squidpy.gr.calculate_niche_utag(data, *, resolutions, n_neighbors=15, use_layer=None, spatial_connectivities_key='spatial_connectivities', embedding_key_added='niche_embedding', use_rep=None, key_added='utag_niche', min_niche_size=None, cluster_mask=None, copy=False, table_key=None, flavor='igraph', n_iterations=-1, rng=None)[source]

Compute spatial niches from UTAG-style feature aggregation.

UTAG [Kim et al., 2022], adapted from https://github.com/ElementoLab/utag/blob/main/utag/segmentation.py

This method computes a spatially aggregated feature representation for each observation and clusters that representation with Leiden. The resulting niches group observations that occur in similar local molecular environments.

A spatial connectivity graph must already be available in adata.obsp[spatial_connectivities_key]. The graph determines how features are propagated or aggregated across spatially neighboring observations; this function does not construct the graph itself.

The method proceeds as follows:

  1. Selects an input feature matrix from adata.X or from adata.layers[use_layer].

  2. Performs a normalized (by number of cell-neighbors) aggregation of features over the spatial connectivity graph, producing a new feature matrix in which each observation reflects information from its local spatial neighborhood.

  3. Reduces that matrix with PCA; the scores are the niche embedding.

  4. Constructs a k-nearest-neighbor graph in that embedding space.

  5. Applies Leiden clustering at each requested resolution.

If the input contains gene expression values, the embedding describes local expression programs rather than merely the categorical composition of neighboring cells.

This differs from calculate_niche_neighborhood(), which uses counts or proportions of a categorical groups annotation. UTAG-style aggregation can identify niches that have similar local expression patterns even when their neighborhoods contain different annotated cell types, or when cell-type labels are unavailable.

Parameters:

  • adata – Annotated data object.

  • resolutions (float | list[float]) – Resolution parameter(s) for Leiden clustering. A single value produces one niche annotation. Supplying multiple values runs Leiden clustering independently at each resolution and adds one niche column per resolution to adata.obs.

  • n_neighbors (int) – Number of nearest neighbors used to construct the k-nearest-neighbor graph on the spatially aggregated embedding before Leiden clustering.

  • use_layer (str | None) –

    Key in adata.layers containing the feature matrix to aggregate. If None, uses adata.X.

    Typically, this should contain a normalized expression matrix or another observation-by-feature representation appropriate for local aggregation. The selected matrix determines what biological signal is used to define niches.

  • spatial_connectivities_key (str) – Key in anndata.AnnData.obsp containing the spatial connectivity matrix.

  • cluster_mask (Series | None) – Boolean pandas.Series indexed like anndata.AnnData.obs. False observations are labeled 'not_a_niche' and take no part in the clustering, though they still reach their neighbors through the graph. Observations it omits are kept. To drop them from the graph as well, subset and rebuild it with spatial_neighbors().

  • use_rep (str | None) – Key in anndata.AnnData.obsm holding the representation to aggregate over each neighborhood, or 'X' for X. Taken as already reduced, so the PCA of the aggregate is skipped. Rejected together with use_layer.

  • key_added (str) – Stem of the anndata.AnnData.obs columns the labels are written to, one per resolution, named f"{key_added}_res={resolution}".

  • embedding_key_added (str) – Controls the name of the key to be added in anndata.AnnData.obsm, which will hold the niche embedding computed.

  • min_niche_size (int | None) – Minimum number of observations required for a niche. Niches with fewer observations are relabeled 'not_a_niche'.

  • copy (bool) – If True, return the result, otherwise save it to the adata object.

  • table_key (str | None) – Key in spatialdata.SpatialData.tables where the table is stored. Required when adata is a spatialdata.SpatialData object and ignored otherwise.

  • flavor (Literal['igraph', 'leidenalg']) – Leiden backend passed to scanpy.tl.leiden(). Defaults to 'igraph' (the 'leidenalg' backend is deprecated in scanpy).

  • n_iterations (int) – Number of Leiden iterations. -1 iterates until convergence.

  • rng (int | integer | Sequence[int] | SeedSequence | Generator | BitGenerator | None) – Pseudorandom number generator state, following SPEC 7. When None, a new numpy.random.Generator is created using entropy from the operating system. Types other than numpy.random.Generator are passed to numpy.random.default_rng() to instantiate a generator. Every resolution is clustered with an independent rng derived from it.

Return type:

AnnData | None

Returns:

If copy=True, returns a copy of adata with the aggregated features in .obsm[embedding_key_added], PCA-reduced unless use_rep was given, and niche assignments added to .obs. Otherwise, modifies adata in place and returns None.