slycat.web.server.remote

Functions for managing cached remote ssh sessions.

Slycat makes extensive use of ssh and the Slycat Agent to access remote resources located on the high performance computing platforms used to generate ensembles. This module provides functionality to create cached remote ssh / agent sessions that can be used to retrieve data from remote hosts. This functionality is used in a variety of ways:

  • Web clients can browse the filesystem of a remote host.
  • Web clients can create a Slycat model using data stored on a remote host.
  • Web clients can retrieve images on a remote host (an essential part of the Parameter Image Model).
  • Web clients can retrieve video compressed from still images on a remote host.

When a remote session is created, a connection to the remote host over ssh is created, an agent is started (only if the required configuration is present), and a unique session identifier is returned. Callers use the session id to retrieve the cached session and communicate with the remote host / agent. A “last access” time for each session is maintained and updated whenever the cached session is accessed. If a session times-out (a threshold amount of time has elapsed since the last access) it is automatically deleted, and subsequent use of the expired session id will fail.

Each session is bound to the IP address of the client that created it - only the same client IP address is allowed to access the session.

class slycat.web.server.remote.Session(client, username, hostname, ssh, sftp, agent=None)[source]

Bases: object

Encapsulates an open session connected to a remote host.

Examples

Calling threads must serialize access to the Session object. To facilitate this, a Session is a context manager - callers should always use a with statement when accessing a session:

>>> with slycat.web.server.remote.get_session(sid) as session:
...   print session.username
accessed

Return the time the session was last accessed.

browse(path, file_reject, file_allow, directory_reject, directory_allow)[source]
cancel_job(jid)[source]

Submits a command to the slycat-agent to cancel a running job on a cluster running SLURM.

Parameters:jid (int) – Job ID
Returns:response – A dictionary with the following keys: jid, output, errors
Return type:dict
checkjob(jid)[source]

Submits a command to the slycat-agent to check the status of a submitted job to a cluster running SLURM.

Parameters:jid (int) – Job ID
Returns:response – A dictionary with the following keys: jid, status, errors
Return type:dict
client

Return the IP address of the client that created the session.

close()[source]
get_file(path, **kwargs)[source]
get_image(path, **kwargs)[source]
get_job_output(jid, path)[source]

Submits a command to the slycat-agent to fetch the content of the a job’s output file from a cluster running SLURM.

Note that the expected format for the output file is slurm-[jid].out.

Parameters:jid (int) – Job ID
Returns:response – A dictionary with the following keys: jid, output, errors
Return type:dict
get_user_config()[source]

Submits a command to the slycat-agent to fetch the content of a user’s .slycatrc file in their home directory.

Returns:response – A dictionary with the configuration values
Return type:dict
get_video(vsid)[source]
get_video_status(vsid)[source]
hostname

Return the remote hostname accessed by the session.

launch(command)[source]

Submits a single command to a remote location via the slycat-agent or SSH.

Parameters:command (string) – Command
Returns:response – A dictionary with the following keys: command, output, errors
Return type:dict
post_video(content_type, images)[source]
run_agent_function(wckey, nnodes, partition, ntasks_per_node, time_hours, time_minutes, time_seconds, fn, fn_params, uid)[source]

Submits a command to the slycat-agent to run a predefined function on a cluster running SLURM.

Parameters:
  • wckey (string) – Workload characterization key
  • nnodes (int) – Number of nodes requested for the job
  • partition (string) – Name of the partition where the job will be run
  • ntasks_per_node (int) – Number of tasks to run on a node
  • ntasks (int) – Number of tasks allocated for the job
  • ncpu_per_task (int) – Number of CPUs per task requested for the job
  • time_hours (int) – Number of hours requested for the job
  • time_minutes (int) – Number of minutes requested for the job
  • time_seconds (int) – Number of seconds requested for the job
  • fn (string) – Name for the Slycat agent function
  • fn_params (dict) – Additional params for the agent function
Returns:

response – A dictionary with the following keys: jid, errors

Return type:

dict

set_user_config(config)[source]

Submits a command to the slycat-agent to set the content of a user’s .slycatrc file in their home directory.

Returns:response
Return type:dict
sftp
submit_batch(filename)[source]

Submits a command to the slycat-agent to start an input batch file on a cluster running SLURM.

Parameters:filename (string) – Name of the batch file
Returns:response – A dictionary with the following keys: filename, jid, errors
Return type:dict
username

Return the username used to create the session.

slycat.web.server.remote.cache_object(pid, key, content_type, content)[source]
slycat.web.server.remote.check_session(sid)[source]

Return a true if session is active

If the session has timed-out or doesn’t exist, returns false

Parameters:sid (string) – Unique session identifier returned by slycat.web.server.remote.create_session().
Returns:
Return type:boolean
slycat.web.server.remote.create_session(hostname, username, password, agent)[source]

Create a cached remote session for the given host.

Parameters:
  • hostname (string) – Name of the remote host to connect via ssh.
  • username (string) – Username for ssh authentication.
  • password (string) – Password for ssh authentication.
  • agent (bool) – Used to require / prevent agent startup.
Returns:

sid – A unique session identifier.

Return type:

string

slycat.web.server.remote.delete_session(sid)[source]

Delete a cached remote session.

Parameters:sid (string, required) – Unique session identifier returned by slycat.web.server.remote.create_session().
slycat.web.server.remote.get_session(sid)[source]

Return a cached remote session.

If the session has timed-out or doesn’t exist, raises a 404 exception.

Parameters:sid (string) – Unique session identifier returned by slycat.web.server.remote.create_session().
Returns:session – Session object that encapsulates the connection to a remote host.
Return type:slycat.web.server.remote.Session