lazyslide.tl.feature_extraction

lazyslide.tl.feature_extraction#

feature_extraction(wsi, model=None, model_path=None, model_name=None, jit=False, token=None, load_kws=None, transform=None, device=None, amp=None, autocast_dtype=None, tile_key='tiles', key_added=None, batch_size=32, num_workers=0, pbar=None, return_features=False, **kwargs)#

Extract features from WSI tiles using a pre-trained vision models.

To list all timm models:

>>> import timm
>>> timm.list_models(pretrained=True)

To list all lazyslide built-in models:

>>> import lazyslide as zs
>>> zs.models.list_models()
Parameters:
wsiWSIData

The whole-slide image object.

modelstr or model object

The model used for image feature extraction. A list of built-in foundation models can be found in Models. Other models can be loaded from Hugging Face, but only models with feature extraction head implemented.

model_pathstr or Path

The path to the model file. Either model or model_path must be provided. If you don’t have internet access, you can download the model file and load it from the local path. You can also load custom models from local files.

model_namestr, optional

If you provide your own model, you can specify the model name for the key_added. Or you can override the model name by providing a new model name.

jitbool, default: False

Whether the model is a JIT model. If True, use torch.jit.load to load the model.

tokenstr, optional

The token for downloading the model from Hugging Face Hub for foundation models.

load_kwsdict, optional

Options to pass to the model creation function.

transformcallable, optional

The transform function for the input image. If not provided, a default ImageNet transform function will be used.

devicestr, optional

The device to use for inference. If not provided, the device will be automatically selected.

ampbool, default: False

Whether to use automatic mixed precision.

autocast_dtypetorch.dtype, default: torch.float16

The dtype for automatic mixed precision.

tile_keystr, default: ‘tiles’

The key of the tiles dataframe in the spatial data object.

key_addedstr, optional

The key to store the extracted features.

batch_sizeint, optional

The batch size for inference.

num_workersint, optional

  • mode=’batch’, The number of workers for data loading.

  • mode=’chunk’, The number of workers for parallel inference.

pbarbool, default: True

Whether to show progress bar.

return_featuresbool, default: False

Whether to return the extracted features.

Returns:
numpy.ndarray or None

If return_features is True, return the extracted features.

Note

The feature matrix will be added to {model_name}_{tile_key} in tables slot of WSIData object.

Examples

>>> import lazyslide as zs
>>> wsi = zs.datasets.sample()
>>> zs.pp.find_tissues(wsi)
>>> zs.pp.tile_tissues(wsi, 256, mpp=0.5)
>>> zs.tl.feature_extraction(wsi, "resnet50")
>>> wsi.fetch.features_anndata("resnet50")