Container Managers

Container managers abstract away the low-level tasks of creating and managing Clipper’s Docker containers in order to allow Clipper to be deployed with different container orchestration frameworks. Clipper currently provides two container manager implementations, one for running Clipper locally directly on Docker, and one for running Clipper on Kubernetes.

class clipper_admin.DockerContainerManager(cluster_name='default-cluster', docker_ip_address='localhost', clipper_query_port=1337, clipper_management_port=1338, clipper_rpc_port=7000, redis_ip=None, redis_port=6379, prometheus_port=9090, docker_network='clipper_network', extra_container_kwargs={})[source]

Bases: clipper_admin.container_manager.ContainerManager

__init__(cluster_name='default-cluster', docker_ip_address='localhost', clipper_query_port=1337, clipper_management_port=1338, clipper_rpc_port=7000, redis_ip=None, redis_port=6379, prometheus_port=9090, docker_network='clipper_network', extra_container_kwargs={})[source]
Parameters:
  • cluster_name (str) – A unique name for this Clipper cluster. This can be used to run multiple Clipper clusters on the same node without interfering with each other.
  • docker_ip_address (str, optional) – The public hostname or IP address at which the Clipper Docker containers can be accessed via their exposed ports. This should almost always be “localhost”. Only change if you know what you’re doing!
  • clipper_query_port (int, optional) – The port on which the query frontend should listen for incoming prediction requests.
  • clipper_management_port (int, optional) – The port on which the management frontend should expose the management REST API.
  • clipper_rpc_port (int, optional) – The port to start the Clipper RPC service on.
  • redis_ip (str, optional) – The address of a running Redis cluster. If set to None, Clipper will start a Redis container for you.
  • redis_port (int, optional) – The Redis port. If redis_ip is set to None, Clipper will start Redis on this port. If redis_ip is provided, Clipper will connect to Redis on this port.
  • docker_network (str, optional) – The docker network to attach the containers to. You can read more about Docker networking in the Docker User Guide.
  • extra_container_kwargs (dict) – Any additional keyword arguments to pass to the call to docker.client.containers.run().
connect()[source]

Use the cluster name to update ports. Because they might not match as in start_clipper the ports might be changed. :return: None

get_logs(logging_dir)[source]

Get the container logs for all Docker containers launched by Clipper.

This will get the logs for both Clipper core containers and any model containers deployed by Clipper admin. Any previous log files from existing containers will be overwritten.
Parameters:logging_dir (str) – The directory to write the log files to. If the directory does not exist, it will be created.
Returns:The list of all log files created.
Return type:list(str)
stop_models(models)[source]

Stops all replicas of the specified models.

Parameters:models (dict(str, list(str))) – For each entry in the dict, the key is a model name and the value is a list of model versions. All replicas for each version of each model will be stopped.
stop_all(graceful=True)[source]

Stop all resources associated with Clipper.

Parameters:graceful (bool) – If set to True, Clipper will try to shutdown all containers gracefully. This option will only work in Docker (Using Docker stop instead of kill).
class clipper_admin.KubernetesContainerManager(cluster_name='default-cluster', kubernetes_proxy_addr=None, redis_ip=None, redis_port=6379, useInternalIP=False, namespace='default', create_namespace_if_not_exists=False)[source]

Bases: clipper_admin.container_manager.ContainerManager

__init__(cluster_name='default-cluster', kubernetes_proxy_addr=None, redis_ip=None, redis_port=6379, useInternalIP=False, namespace='default', create_namespace_if_not_exists=False)[source]
Parameters:
  • cluster_name (str) – A unique name for this Clipper cluster. This can be used to run multiple Clipper clusters on the same Kubernetes cluster without interfering with each other. Kubernetes cluster name must follow Kubernetes label value naming rule, namely: Valid label values must be 63 characters or less and must be empty or begin and end with an alphanumeric character ([a-z0-9A-Z]) with dashes (-), underscores (_), dots (.), and alphanumerics between. See more at: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
  • kubernetes_proxy_addr (str, optional) – The proxy address if you are proxying connections locally using kubectl proxy. If this argument is provided, Clipper will construct the appropriate proxy URLs for accessing Clipper’s Kubernetes services, rather than using the API server addres provided in your kube config.
  • redis_ip (str, optional) – The address of a running Redis cluster. If set to None, Clipper will start a Redis deployment for you.
  • redis_port (int, optional) – The Redis port. If redis_ip is set to None, Clipper will start Redis on this port. If redis_ip is provided, Clipper will connect to Redis on this port.
  • useInternalIP (bool, optional) – Use Internal IP of the K8S nodes . If useInternalIP is set to False, Clipper will throw an exception if none of the nodes have ExternalDNS. If useInternalIP is set to true, Clipper will use the Internal IP of the K8S node if no ExternalDNS exists for any of the nodes.
  • namespace (str, optional) – The Kubernetes namespace to use . If this argument is provided, all Clipper artifacts and resources will be created in this k8s namespace. If not “default” namespace is used.
  • create_namespace_if_not_exists (bool, False) – Create a k8s namespace if the namespace doesnt already exist. If this argument is provided and the k8s namespace does not exist a new k8s namespace will be created.

Note

Clipper stores all persistent configuration state (such as registered application and model information) in Redis. If you want Clipper to be durable and able to recover from failures, we recommend configuring your own persistent and replicated Redis cluster rather than letting Clipper launch one for you.

get_logs(logging_dir)[source]

Get the container logs for all Docker containers launched by Clipper.

This will get the logs for both Clipper core containers and any model containers deployed by Clipper admin. Any previous log files from existing containers will be overwritten.
Parameters:logging_dir (str) – The directory to write the log files to. If the directory does not exist, it will be created.
Returns:The list of all log files created.
Return type:list(str)
stop_models(models)[source]

Stops all replicas of the specified models.

Parameters:models (dict(str, list(str))) – For each entry in the dict, the key is a model name and the value is a list of model versions. All replicas for each version of each model will be stopped.
stop_all(graceful=True)[source]

Stop all resources associated with Clipper.

Parameters:graceful (bool) – If set to True, Clipper will try to shutdown all containers gracefully. This option will only work in Docker (Using Docker stop instead of kill).