Shortcuts

modelzoo.transformers

modelzoo.transformers.deploy(pipeline: transformers.pipelines.Pipeline, model_name: Optional[str] = None, resources_config: Optional[Dict] = None, api_key: Optional[str] = None, wait_until_healthy: bool = True, demo: bool = False) → None

Deploy a transformers.Pipeline.

Note

This function will serialize a model to a temporary directory on the filesystem before uploading it to Model Zoo.

Note

Currently, the Model Zoo free tier only supports transformers.TextGenerationPipeline and transformers.SummarizationPipeline.

Parameters
  • pipeline

    A transformers.Pipeline object to deploy.

  • model_name – Optional string name of the model. If not provided, a random name will be generated. Model name must be unique across all of a user’s models.

  • resources_config – An optional modelzoo.ResourcesConfig that specifies the resources (e.g. memory, CPU) to use for the model. Defaults to modelzoo.ResourcesConfig().

  • api_key – Optional API key that, if provided, will override the API key available to the environment.

  • wait_until_healthy – If True (default), this function will refrain from returning until the model has reached a HEALTHY state.

Returns

The name of the created model.

modelzoo.transformers.generate(model_name: str, input_str: Optional[str] = '', api_key: Optional[str] = None, **kwargs) → Dict[str, Any]

Send a prediction to a Transformers model. In addition to an optional input string, this function accepts the same arguments as transformers.PreTrainedModel.generate().

Parameters
  • model_name – String name of the model.

  • input_str – An optional input string that will be used as the beginning of the sample. Defaults to an empty string.

  • api_key – Will override the environment api key, if present.

  • **kwargs

    Add any keyword arguments that would be accepted by transformers.PreTrainedModel.generate().

Returns

The output prediction