.. _f-azureconnection: module ``remote.azure_connection`` ================================== .. inheritance-diagram:: pyenbc.remote.azure_connection Short summary +++++++++++++ module ``pyenbc.remote.azure_connection`` A class to help connect with a remote machine and send command line. :githublink:`%|py|6` Classes +++++++ +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | class | truncated documentation | +===================================================================+============================================================================================================+ | :class:`AzureClient ` | A simple class to access and communicate with `Azure `_. It requires modules: | +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ Static Methods ++++++++++++++ +------------------------------------------------------------------------------+------------------------------------------------------------------+ | staticmethod | truncated documentation | +==============================================================================+==================================================================+ | :meth:`mask_string ` | return empty string or as many ``*`` as the length of the string | +------------------------------------------------------------------------------+------------------------------------------------------------------+ Methods +++++++ +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | method | truncated documentation | +==========================================================================================+===============================================================================================================================================================+ | :py:meth:`__init__ ` | constructor | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__str__ ` | usual | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_interpret_path ` | replace variavble such as ``$PSEUDO``, ``$USERNAME`` | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`copy_blob ` | copy a blob | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`delete_blob ` | delete a blob | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`delete_folder ` | delete a folder and its content | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`df_head ` | Downloads the beginning of a stream and displays as a DataFrame. | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`download ` | Downloads data from a blob storage to a file. No more than 64Mb can be downloaded at the same, it needs to be split ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`download_data ` | Downloads data from a blob storage and return bytes. No more than 64Mb can be downloaded at the same, it needs ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`download_merge ` | Downloads all files from a folder in a blob storage to a single local file. Files will be merged. No more ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`exists ` | test the existence of a path on the blob storage | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`get_status ` | return the status of the webHCatUrl server | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`get_version ` | return the status of the WebHCat version | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`hive_submit ` | Submits a :epkg:`HIVE` job, the function uploads it to the cluster as well as the dependencies. The code ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`job_kill ` | kills a job see `Delete Job — DELETE queue/:jobid `_ ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`job_queue ` | returns the list of jobs It uses the API `Job Information — GET queue/:jobid `_. ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`job_status ` | return the status of a job see `List Versions — GET version `_ ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`ls ` | return the content of a blob storage | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`open_blob_service ` | open a blob service | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`pig_submit ` | Submits a :epkg:`PIG` job, the function uploads it to the cluster as well as the dependencies. The code ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`standard_outputs ` | returns the standard output and error for a specific job id | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`upload ` | Uploads data from a file to a blob storage. No more than 64Mb can be uploaded at the same, it needs to be split ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`upload_data ` | Uploads data (bytes) to a blob storage. No more than 64Mb can be uploaded at the same, it needs to be split into ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`url_blob ` | returns an url for a blob file name | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`url_webHCatUrl ` | returns an url to the cluster | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`wait_job ` | wait until a job has run or failed | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`wasb_prefix ` | when using an instruction ``LOAD`` in a PIG script, file blob name must be reference using a wasb syntax. ... | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`wasb_to_file ` | return something like ``wasb://demo@myblobstorage.blob...`` | +------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ Documentation +++++++++++++ .. automodule:: pyenbc.remote.azure_connection :members: :special-members: __init__ :show-inheritance: