Skip to main content

​
CACHE_DURATION_IN_SECONDS

1 day

​
CloudUploadSettings Objects

Settings for uploading data into GCP cloud storage. These apply for each individual upload and will overwrite the RequestsSettings() defined during EncordUserClient creation.

​
max_retries

Maximum number of allowed retries when uploading a file.

​
backoff_factor

Factor used to calculate exponential backoff between retries. The delay before each retry is computed as: backoff_factor * (2 ** (retry_number - 1)).

​
allow_failures

Whether to allow partial failures during upload. If set to True, the upload will proceed even if some items fail after all retries. Failed uploads will be logged, and the process will continue. This is useful for large batch uploads where a few failures are acceptable.

​
get_batches

Split an iterable into fixed-size batches. Arguments:
  • iterable List - The input list to be split.
  • n int - The maximum size of each batch.
Returns:
  • List[List] - A list of lists where each sublist represents a batch.
Notes: This can be replaced with itertools.batched <https://docs.python.org/3/library/itertools.html#itertools.batched>_` in Python 3.12+.

​
UploadToSignedUrlFailure Objects

Details of a failed upload attempt to a signed URL.

​
exception

The exception instance raised during the failed upload.

​
file_path

Path to the file that failed to upload.

​
title

Title or identifier associated with the file.

​
signed_url

The signed URL that the file upload was attempted against.

​
upload_to_signed_url_list_for_single_file

Attempt to upload a single file to a signed URL, appending failures if any occur. Arguments:
  • failures List[UploadToSignedUrlFailure] - A list to append failures to.
  • file_path Union[str, Path] - Path of the file to upload.
  • title str - Title or identifier for the file.
  • signed_url str - The signed URL to upload the file to.
  • upload_item_type StorageItemType - The type of the file being uploaded.
  • max_retries int - Maximum number of retries in case of failure.
  • backoff_factor float - Backoff factor for retry delays.
  • upload_headers Optional[Dict[str, str]] - Backend-specified headers (e.g. Azure’s x-ms-blob-type) to forward on the PUT.

​
UploadPresignedUrlsGetParams Objects

Parameters for requesting presigned URLs for file uploads.

​
count

Number of presigned URLs to request, typically matching the batch size of files.

​
upload_item_type

The type of item being uploaded (e.g., IMAGE, VIDEO, AUDIO, DICOM).

​
upload_to_signed_url_list

Upload multiple files to signed URLs and return upload results. Arguments:
  • file_paths Iterable[Union[str, Path]] - Paths of files to upload.
  • config BaseConfig - Configuration object with request settings.
  • api_client ApiClient - API client used to fetch presigned URLs.
  • upload_item_type StorageItemType - Type of items being uploaded.
  • cloud_upload_settings CloudUploadSettings - Upload configuration.
Returns:
  • List[Dict] - A list of dictionaries containing upload metadata:
    • data_hash (str): Unique identifier for the file.
    • file_link (str): Link to the uploaded file in storage.
    • title (str): File name or title.
Raises:
  • EncordException - If any file path does not exist.
  • CloudUploadError - If uploads fail and allow_failures is False.

​
download_signed_urls_as_json

Concurrently GET each URL and JSON-decode the body. Returns a mapping of url -> decoded JSON for the URLs that resolved successfully. Transient failures (5xx, 429, connection resets, timeouts) are retried with exponential back off and jitter via the underlying session. Any URL that still fails — a non-200 status (e.g. an expired or missing object), an exhausted retry, or a body that won’t JSON-parse — is omitted from the result rather than raised, so a single bad URL never fails the whole batch and callers can treat absence as “no content for this URL”. Duplicate and falsy URLs are ignored.