Zesty launches automated scoring GPU, bringing NVIDIA CUDA native GPU segmentation automation
With the open-source release of NVIDIA for the Run : ai platform, Zesty's new autonomous capabilities allow for continuous adjustment of GPU memory allocation, computational sharing, and workload placement as needs change, enabling each piece of GPU to run more workloads.
San Mateo, California, USA, October 15th / PRNewswire / -- Kubernetes The company Zesty announced today its support for the NVIDIA native GPU segmentation. Following NVIDIA's open-sourcing of Run : ai as part of NVIDIA DSX AI Factory Platform, Zesty's Automated Fractional GPUs has gained an additional layer of autonomous decision-making capability, allowing it to sustainably determine GPU memory allocation and computing sharing patterns, and to adjust workload placement in response to fluctuations in AI workload demands. Zesty states that with these autonomous capabilities, the team can improve GPU utilization, run more workloads on existing infrastructure, reduce the need for additional GPU capacity, and lower infrastructure costs while maintaining application performance.
According to Zesty, 66% of organizations that use generative AI models run part or all of their inference tasks on Kubernetes. However, the GPU capacity required for these workloads is expensive and usually limited. In the past, teams would typically allocate an entire GPU to a single workload, or share it through time-slicing, NVIDIA Multi - Process Service (MPS), or NVIDIA Multi - Instance GPU (MIG). The latter method divided each GPU into fixed hardware partitions. Zesty indicates that these allocation methods could not keep up with the dynamic changes in workload and real-time demands. As a result, GPU memory and computing power were left idle, and teams continued to purchase more GPU even when there was unused capacity available.
Zesty indicates that NVIDIA and the recently open-sourced Run: ai introduces the original GPU split into Kubernetes through GPU Fractioning Operator. This operator can enforce the GPU memory limit for each workload and work in conjunction with the Kubernetes scheduler. Zesty points out that the partitioning mechanism enables flexibility and defines how workloads are split into blocks GPU, but it does not determine how these capacities are allocated. Another layer of decision-making is needed to determine which workloads share each block GPU, how much memory each workload receives, and how these allocations change as demand increases or decreases.
What Zesty provides is exactly this layer. Its Automated Fractional GPU function continuously determines which workloads should share the same GPU, how many workloads can run safely simultaneously, how much GPU memory each workload should receive, which sharing mode is most suitable for each workload, and when these allocations should be increased or decreased. The relevant decisions are based on the actual GPU memory usage and utilization, the behavior of workloads observed over time, as well as the predefined latency requirements and resource limitations set by the team. All changes will remain within the performance and availability strategies established by the team for each workload.
Zesty, Vice President of R&D, stated: " GPU are among the most expensive and restricted resources managed by today's infrastructure teams. They are difficult to secure, incur high costs, and it's challenging to share them with workloads that are constantly changing in demand. Now, NVIDIA has opened up GPU to the ecosystem, and Zesty is making this distribution more intelligent and autonomous."
He stated, " Autonomous GPU Resource Management will adjust the memory and computing resources shared within GPU as the demands of each workload fluctuate, and it will also guide the placement of workloads. This allows the team to accommodate more workloads on existing hardware without compromising on performance quality, reducing the need for new GPU purchases and lowering the costs of running AI in a production environment."
This feature is deployed in the form of a lightweight proxy within the Kubernetes cluster and works in conjunction with the schedulers of NVIDIA Run : ai, GPU Fractioning Operator, and Kubernetes. It supports time-slicing and MPS. There is no need to modify the application code or models, and the installation does not require restarting any running workloads. On the supported GPU technology stack, Zesty can manage the necessary Kubernetes configurations for optimization without the need to manually edit inventory files.
Automated Fractional GPUs is now available as part of the Zesty platform. For more information, please visit http :// zesty.co / platform / automated-fractional-gpus.
About Zesty
Zesty helps enterprises reduce infrastructure costs through the optimization of their own resources. This platform continuously matches resources with real-time workload demands, automatically eliminates overconfiguration, and improves infrastructure efficiency while protecting application performance. By replacing continuous manual resource adjustments and performance tuning with policy-driven automation, Zesty enables teams to operate a more streamlined Kubernetes environment without increasing operational risks.
Leveraging Automated Fractional GPUs, Zesty is extending this method to the GPU infrastructure, helping teams to derive more value from the existing GPU capacity while controlling the continuously rising costs of the AI workload.
Zesty was established in 2019, focusing on making infrastructure more efficient, automated, and cost-effective.
For more information, please visit zesty.co.
Media Contact
Kate Schoenstadt
[ email protected ]
IL +972(0)-54-7776684
