​
VisionLanguageEmbedding

class VisionLanguageEmbedding:
Provides image embedding functionalities using multimodal model. Parameters:
  • model_name: The model type to be used for generating embeddings. And the default value is: obj:openai/clip-vit-base-patch32.

​
init

def __init__(self, model_name: str = 'openai/clip-vit-base-patch32'):
Initializes the: obj: VisionLanguageEmbedding class with a specified model and return the dimension of embeddings. Parameters:
  • model_name (str, optional): The version name of the model to use. (default: :obj:openai/clip-vit-base-patch32)

​
embed_list

def embed_list(self, objs: List[Union[Image.Image, str]], **kwargs: Any):
Generates embeddings for the given images or texts. Parameters:
  • objs (List[Image.Image|str]): The list of images or texts for which to generate the embeddings.
  • image_processor_kwargs: Extra kwargs passed to the image processor.
  • tokenizer_kwargs: Extra kwargs passed to the text tokenizer (processor).
  • model_kwargs: Extra kwargs passed to the main model.
Returns: List[List[float]]: A list that represents the generated embedding as a list of floating-point numbers.

​
get_output_dim

def get_output_dim(self):
Returns: int: The dimensionality of the embedding for the current model.