vikit.common.file_tools¶

Overview¶

Function¶
`copy_file_from_gcs`(bucket, blob_path, destination_file_name)	Copy a file from Google Cloud storage using the API
`download_or_copy_file`(url, local_path, force_download, enable_gcs_url_rewrite)	Download a file from a URL to a local file asynchronously
`file_url_exists`(url)	Check if a file URL exists locally
`gcs_file_exists`(bucket, file_path)	-
`get_canonical_name`(file_path)	Get the canonical name of a file, without the extension
`get_max_path_length`(path)	get the max file name for the current OS
`get_max_remote_path_length`()	Get the maximum length of a remote path
`get_path_type`(path)	Validate the path and return its type
`get_public_uri_from_gcs_path`(gcs_path)	-
`get_safe_filename`(filename)	-
`is_valid_filename`(filename)	Check if the provided string is a valid filename for the local file system.
`is_valid_path`(path)	Check if the path is valid: could be a local path or a remote one
`upload_to_bucket`(source_file_name, destination_blob, destination_file_name, bucket_name)	-
`url_exists`(url)	Check if a URL exists somewhere on the internet or locally. To be superseded by a more
`web_url_exists`(url)	-

Attributes¶
`TIMEOUT`	-

Functions¶

vikit.common.file_tools.copy_file_from_gcs(bucket: str, blob_path: str, destination_file_name: str = 'downloaded_file')¶

Copy a file from Google Cloud storage using the API

params:: gcs_full_url: the full URL of the file to copy blob_path: the name of the object to copy destination_file_name: the local path of the copied file

return: the local path of the copied file

async vikit.common.file_tools.download_or_copy_file(url: str, local_path: str, force_download: bool = False, enable_gcs_url_rewrite: bool = True) → str¶

Download a file from a URL to a local file asynchronously

Parameters:

url (str) -- The URL to download the file from (supported: http, https, gs, local)
local_path (str) -- The filename to save the file to.
force_download (bool) -- If True, the file will be downloaded even if it already exists
enable_gcs_url_rewrite (bool) -- If True, the GCS Cloud Storage libraries will be used for HTTP(S) URLs pointing to cloud storage resource. If False, the file will be downloaded using the HTTP(S) protocol. This parameter exists mainly for testing purposes. We recommend using the GCS library whenever possible.

Returns:

The filename of the downloaded file.

Return type:

str

vikit.common.file_tools.file_url_exists(url)¶: Check if a file URL exists locally

vikit.common.file_tools.gcs_file_exists(bucket: str, file_path: str) → bool¶

vikit.common.file_tools.get_canonical_name(file_path: str)¶: Get the canonical name of a file, without the extension

vikit.common.file_tools.get_max_path_length(path='.')¶

get the max file name for the current OS

params:: path: the file path

return: file name max length.

vikit.common.file_tools.get_max_remote_path_length()¶: Get the maximum length of a remote path

vikit.common.file_tools.get_path_type(path: str | os.PathLike | None) → dict¶

Validate the path and return its type

Parameters:: path (str, os.PathLike,) -- The path to validate
Returns:: The path type and the path itself Path type can be local, http, https, s3, gs, None, undefined, error, error : message if the path is invalid, None if no error
Return type:: dict

vikit.common.file_tools.get_public_uri_from_gcs_path(gcs_path: str) → str¶

vikit.common.file_tools.get_safe_filename(filename)¶

vikit.common.file_tools.is_valid_filename(filename: str) → bool¶

Check if the provided string is a valid filename for the local file system.

Parameters:: filename (str) -- The filename to check.
Returns:: True if valid, False otherwise.
Return type:: bool

vikit.common.file_tools.is_valid_path(path: str | os.PathLike | None) → bool¶

Check if the path is valid: could be a local path or a remote one (http, etc). We don’t test the actual access and credentials at this stage, just the path format.

Parameters:: path (str, os.PathLike) -- The path to validate
Returns:: True if the path is valid, False otherwise
Return type:: bool

async vikit.common.file_tools.upload_to_bucket(source_file_name, destination_blob, destination_file_name, bucket_name)¶

vikit.common.file_tools.url_exists(url: str)¶

Check if a URL exists somewhere on the internet or locally. To be superseded by a more versatile and unified library in the future.

Parameters:: url (str) -- The URL to check
Returns:: True if the URL exists, False otherwise
Return type:: bool

vikit.common.file_tools.web_url_exists(url)¶

Attributes¶

vikit.common.file_tools.TIMEOUT = 10¶