.. _f-magicremotessh: module ``remote.magic_remote_ssh`` ================================== .. inheritance-diagram:: pyenbc.remote.magic_remote_ssh Short summary +++++++++++++ module ``pyenbc.remote.magic_remote_ssh`` Magic command to communicate with an Hadoop cluster (Linux). :githublink:`%|py|6` Classes +++++++ +-------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------+ | class | truncated documentation | +=========================================================================+=========================================================================================================================+ | :class:`MagicRemoteSSH ` | Defines commands to access a remote machine (bridge) through SSH, for the time being, all the command do not accept ... | +-------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------+ Functions +++++++++ +----------------------------------------------------------------------------------+---------------------------------------------------------+ | function | truncated documentation | +==================================================================================+=========================================================+ | :func:`register_magics_ssh ` | register magics function, can be called from a notebook | +----------------------------------------------------------------------------------+---------------------------------------------------------+ Properties ++++++++++ +-----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+ | property | truncated documentation | +=====================================================================================================+===============================================================================================================+ | :meth:`Context ` | return the context or None | +-----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+ | :meth:`cross_validation_lock ` | A contextmanager for running a block with our cross validation lock set to True. At the end of the block, ... | +-----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+ Static Methods ++++++++++++++ +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | staticmethod | truncated documentation | +===============================================================================================================+=====================================================================+ | :meth:`dfs_ls_parser ` | defines the way to parse the magic command ``%dfs_ls`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`dfs_mkdir_parser ` | defines the way to parse the magic command ``%dfs_mkdir`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`dfs_rm_parser ` | defines the way to parse the magic command ``%dfs_rm`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`HIVE_parser ` | defines the way to parse the magic command ``%%HIVE`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`hive_submit_parser ` | defines the way to parse the magic command ``%hive_submit`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`job_syntax_parser ` | defines the way to parse the magic command ``%job_syntax`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`open_remote_shell_parser ` | defines the way to parse the magic command ``%open_remote_shell`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`PIG_parser ` | defines the way to parse the magic command ``%%PIG`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`pig_submit_parser ` | defines the way to parse the magic command ``%pig_submit`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_down_cluster_parser ` | defines the way to parse the magic command ``%remote_down_cluster`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_down_parser ` | defines the way to parse the magic command ``%remote_down`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_ls_parser ` | defines the way to parse the magic command ``%remote_ls`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_open_parser ` | defines the way to parse the magic command ``%remote_open`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_py_parser ` | defines the way to parse the magic command ``%remote_py`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_up_cluster_parser ` | defines the way to parse the magic command ``%remote_up_cluster`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ | :meth:`remote_up_parser ` | defines the way to parse the magic command ``%remote_up`` | +---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+ Methods +++++++ +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | method | truncated documentation | +=================================================================================================+==================================================================================+ | :py:meth:`_replace_params ` | replaces parameter such ``__PASSWORD__`` by variable in the notebook environment | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`close_remote_shell ` | Defines ``%close_remote_shell`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`dfs_ls ` | returns the content of a folder on the cluster as a dataframe Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`dfs_mkdir ` | Creates a folder on the cluster. Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`dfs_rm ` | remove a file on the cluster Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`get_connection ` | returns the connection stored in the workspace | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`HIVE ` | defines command ``%%HIVE`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`hive_submit ` | defines command ``%hive_submit`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`job_syntax ` | defines command ``%job_syntax`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`open_remote_shell ` | Defines ``%open_remote_shell`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`PIG ` | defines command ``%%PIG`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`pig_submit ` | defines command ``%pig_submit`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_close ` | close a SSH connection and store the connection into the notebook workspace | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_cmd ` | run a command on the remote machine Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_cmd_text ` | run a command on the remote machine and returns raw text (not HTML) Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_down ` | download a file from the remote machine, Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_down_cluster ` | download a file from the cluster to the local machine through the bridge Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_ls ` | returns the content of a folder on the remote machine as a dataframe Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_open ` | open a SSH connection and store the connection into the notebook workspace | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_py ` | defines command ``%remote_py`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_up ` | upload a file to the remote machine, Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`remote_up_cluster ` | upload a file to the remote machine and then to the remote cluster, Example | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`shell_remote ` | Defines ``%shell_remote`` and ``%%shell_remote`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ | :meth:`shell_remote_text ` | Defines ``%shell_remote_text`` and ``%%shell_remote_text`` | +-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+ Documentation +++++++++++++ .. automodule:: pyenbc.remote.magic_remote_ssh :members: :special-members: __init__ :show-inheritance: