Benjamin Satzger 2eb85f14cb Introduce Inference Endpoints ...

This introduces the concept of inference endpoints and inference endpoint
replicas. An inference endpoint describes a model and is backed by a
private network, a load balancer and multiple replicas. A replica
is basically a VM that runs the model. The replica VM is part of the
inference endpoints private subnet and attached to its load balancer.

2024-10-01 15:36:57 +02:00

7.7 KiB

Raw Permalink Blame History

View Raw