Lepton AI

What is NVIDIA DGX Cloud Lepton?

NVIDIA DGX Cloud Lepton is a unified GPU computing platform designed for AI developers and corporate teams, used for building, training, fine-tuning, and deploying models. It connects NVIDIA Cloud Partners, the GPU market, cloud service providers, and users’ own computing resources, offering a consistent workspace and management experience.

This product builds on the platform and development tools of Lepton AI; the existing official documentation and open-source repositories all use the name NVIDIA DGX Cloud Lepton. The term “Lepton AI” can be retained in the directory as an old name and for search purposes, but the main text should use the current brand name.

Product evolution and current position

In May 2025, NVIDIA announced DGX Cloud Lepton, a platform that integrates global GPU supply with developers’ workflows. It is not a regular chatbot or a directory of models; rather, it represents an AI infrastructure that covers development environments, training tasks, inference services, and resource orchestration.

The platform works in conjunction with software stacks such as NVIDIA NIM, NeMo, and Blueprints; it also supports custom containers, open-source models, and user-owned infrastructure. The available regions, GPUs, and services vary depending on the workspace and supplier.

Core functions

Inference endpoint

Endpoints deploy models or containers as accessible network services, and they offer support for replicas, automatic scaling, health checks, logging, monitoring, and access control. Users can create endpoints from container images, LLM runtimes, NVIDIA NIM, or templates.

Dev Pods development environment

Dev Pod offers an interactive development environment equipped with CPU or GPU, and it can be connected to a terminal, SSH, Jupyter, or VS Code. It is suitable for debugging code, preparing data, validating containers, and testing models.

Batch processing and training tasks

Batch Jobs are used for training, data processing, and offline inference; PyTorch, MPI, or custom containers can be selected. These tasks support multi-node operation, priority settings, pre-emption, storage mounting, environment variables, and retry on failure.

Ray and Slurm clusters

The platform supports RayCluster and Slurm clusters, offering familiar scheduling methods for distributed training and computing. The clusters continue to use the selected resources while they are running, and it is necessary to delete them once their usage is complete.

GPU resources and Node Groups

A Node Group organizes specific GPU nodes into a resource pool, and defines the combination of GPUs, CPUs, memory, and storage through Resource Shapes. Teams can use the nodes provided by the platform, or they can connect their own machines using BYOC.

Work area, storage, and observability

Workspace provides centralized management of members, roles, tokens, keys, image repositories, and workloads. The platform also offers capabilities for logging, metrics, events, health checks, and persistent storage.

Python SDK and lep CLI

The official LeptonAI Python library, together with the lep command-line tool, allows for the creation and management of endpoints, tasks, Pods, clusters, storage resources, keys, and workspaces. The SDK client can also read the OpenAPI structure of endpoints and invoke services in a manner similar to Python functions.

List of main functions

• Discover and utilize GPU resources from various regions and suppliers within the unified workspace.
• Deploy models, NIMs, or custom containers as flexible REST APIs.
• Create a GPU development Pod and connect to it via the terminal, SSH, Jupyter, or VS Code.
• Run standalone, multi-GPU, or multi-node training and batch processing tasks.
• Create Ray and Slurm clusters to handle distributed workloads.
• Manage resource combinations through Node Groups and Resource Shapes.
• Connect to your own GPU infrastructure while utilizing a unified management interface.
• Mount local volumes or NFS shared storage to save data and models.
• Manage environment variables, keys, image repositories, and access tokens.
• View logs, metrics, events, health status, and replica information.
• Automate operations and maintenance using the Python SDK, REST API, and lep CLI.
• Use roles, tokens, IP allowlists, and endpoint access policies to control permissions.

What use cases are suitable?

• AI startup teams: Quickly scale from model prototypes to production inference endpoints.
• Model development team: Uses multiple GPUs or multi-node resources to train and fine-tune models.
• Data team: Handles batch feature processing, embedding generation, and offline inference.
• Enterprise Platform Team: Provides unified management of multi-cloud and on-premises GPU computing resources.
• Research institutions: Create isolated development environments and distributed clusters based on specific projects.
• Inference service provider: Deploy custom LLM, vision, audio, and multimodal models.
• DevOps team: Accesses the CI/CD process via CLI, SDKs, and service accounts.
• Sovereign AI project: Select computing resources based on region and plan for data location.

What situations are not very suitable

• Ordinary individual users who simply want to chat, write, or create images.
• Beginner users who have no experience with containers, Linux, GPUs, or cloud cost management.
• Projects that require a fixed low-price package and cannot accept dynamic pricing for resources.
• Organizations that require the open-source source code for all platform control interfaces and the ability to deploy them entirely on their own.
• Teams with a very limited budget that only run lightweight CPU scripts occasionally.
• Users who are unable to assume the responsibilities related to model licensing, data compliance, and infrastructure security.

Comparison of workload types

Quick Start Tutorial

Create a workspace and prepare resources.

1. Use your NVIDIA account to access DGX Cloud Lepton and create or join a workspace.
2. Verify the location, available suppliers, GPU models, and account billing status.
3. Create or select a Node Group, and view the current Resource Shape.
4. Configure member roles, tokens, keys, and repository credentials.
5. First, conduct tests using resources of a smaller scale, and then increase the number of replicas or nodes.

Deploy the inference endpoint

1. Choose to create an endpoint from an LLM, NVIDIA NIM, container image, or template.
2. Enter the endpoint name, image, execution command, and service port.
3. Select the Node Group, GPU specifications, number of replicas, and auto-scaling policy.
4. Inject the model token via Secret; do not include it in the image or regular environment variables.
5. Enable endpoint tokens or IP restrictions to prevent sensitive services from being publicly accessible by default.
6. After deployment, check the health status, logs, latency, and GPU memory usage.
7. Use test requests to verify the output, and then integrate real business traffic.

Run batch tasks

1. Prepare a container image that includes the code and dependencies.
2. Choose a custom, PyTorch, or MPI task template.
3. Configure GPU specifications, the number of nodes, working processes, and execution commands.
4. Mount data, models, and checkpoint storage, and set the archiving time.
5. Set failure retry, priority, and whether preemption is allowed.
6. After starting the task, monitor the logs, events, and resource utilization.
7. Save the results and checkpoints, and release resources once it is confirmed that there are no further tasks.

Use the Python SDK and CLI

1. Install the latest version of the leptonai package in Python 3.10 or a compatible environment.
2. Log in using the workspace ID and a short-term API token.
3. First, list the workspaces, Node Groups, and available Resource Shapes.
4. Use the lep command to create endpoints, Pods, or tasks and view their status.
5. Use service account tokens and roles with the minimum required permissions in automated processes.
6. Manual confirmation and budget checks are added for deletion, scaling, and high-cost tasks.

Comparison of endpoint creation methods

Pricing and billing methods

DGX Cloud Lepton does not have a single, fixed pricing schedule that applies across all regions. The costs vary depending on the GPU supplier, region, model, configuration of resources, duration of use, capacity options, as well as storage and network traffic; these details are specified in the workspace or business quote.

Therefore, the old GPU hourly rates from the earlier Lepton pages or estimates provided by third parties should not be used as the current official pricing. Before making a purchase, it is necessary to check the available resources and pricing in the actual working environment; for enterprise-level capacities and long-term reservations, one should contact the sales team.

Comparison of managed computing and BYOC

Automatic scaling and resource release

According to the official documentation, the new endpoint uses a single copy by default, and it can be scaled down to zero automatically after one hour of inactivity. This scaling down helps reduce the costs associated with idle time for intermittent services, but it may cause a cold-start delay for the next request.

• Set reasonable minimum and maximum numbers of replicas for sudden traffic spikes.
• Enable scaling down to zero for low-frequency test endpoints.
• Production services retain necessary copies and monitor queues and delays.
• Dev Pod, Ray, and Slurm clusters are not automatically released in an equivalent manner once a task is completed.
• After the batch processing is complete, check the remaining storage, logs, and node reservations.

Storage and data management

The platform supports Node Local Volume and Static NFS Volume. The former stores data locally on the same node, without any data replication across nodes; the latter allows multiple nodes to mount a shared file system, but its performance and durability depend on the NFS infrastructure.

Permissions can be configured based on volume paths and members; once the rules are enabled, paths that have not been explicitly authorized cannot be accessed. Backup, cleanup, and recovery plans should be established for training data, model weights, and checkpoints as well.

Security and access management

Endpoints can be configured with access tokens, IP addresses, or CIDR restrictions, and it is possible to set the visibility of the workspace. The official documentation recommends enabling access tokens for endpoints; services that contain sensitive models or business logic should not be made publicly accessible.

Personal API tokens have a relatively short validity period by default, while service account tokens are suitable for CI/CD and shared automation. Service account tokens are displayed only once when they are created; they should be stored in an enterprise key management system and rotated regularly.

• Use roles with the minimum required permissions to separate development, deployment, and administrative responsibilities.
• Model repositories, cloud credentials, and API keys are all stored in Secret.
• Enable tokens at the production endpoint and restrict source IPs.
• Automation uses service accounts, without reusing employees’ personal tokens.
• Periodically revoke expired, retired, or unused tokens.
• Avoid including prompts, keys, and sensitive user data in the logs.
• Throttle, abuse, and cost attack tests are conducted before the public endpoint goes live.

The open-source boundaries of Python SDK, CLI, and cloud platforms

The open-source SDK allows developers to examine client code, submit issues, and enable automation in the build process; however, this does not mean that the scheduler, control interface, and cloud services of DGX Cloud Lepton are all open source. The ability to run certain Photon or client functions locally does not imply that the entire platform can be deployed offline in a private environment.

Product advantages

• Manage development, training, batch processing, and production inference under a unified platform.
• It connects to multiple GPU suppliers and supports various regions as well as NVIDIA architectures.
• Compatible with NVIDIA NIM, NeMo, and common open-source model workflows.
• It provides Endpoint, Dev Pod, Batch Job, and distributed clusters.
• It supports BYOC, facilitating the integration of existing GPU infrastructure.
• SDKs, CLI, and REST APIs address the needs of automation and platform engineering.
• The open-source Python SDK is licensed under the permissive Apache-2.0 license.
• Access control, keys, logging, and monitoring capabilities are quite comprehensive.
• Resource Shape helps to provide a unified description of GPUs, CPUs, memory, and storage.

Usage restrictions and precautions

• Pricing varies depending on the supplier, region, and GPU capacity, making it difficult to specify a single public price.
• The platform is intended for professional developers and requires knowledge of container, Linux, and GPU management.
• If the endpoint is set to public, anyone who has the address can call it and incur costs.
• Scaling down to zero results in a cold start, and production delays require actual performance testing.
• Dev Pods and clusters continuously consume resources, and failing to release them will increase costs.
• Model weights, datasets, and NIM may have separate licenses and usage restrictions.
• Cloudy environments and BYOC reduce differences between platforms, but they cannot eliminate differences in networking and hardware.
• Node Local Volume is not replicated across nodes, and therefore cannot serve as the sole backup.
• Strictly define permission boundaries for service accounts and workload identities.
• An open-source SDK does not mean that the entire hosting platform is open source.

Basic information

Recommendation score

4.7 / 5. DGX Cloud Lepton is suitable for professional teams that need centralized GPU resources for training, batch processing, and production-level inference; the NVIDIA ecosystem and open-source SDKs represent clear advantages. However, it comes with dynamic pricing, high technical requirements, and stringent demands regarding resource management.

Frequently Asked Questions

What is Lepton AI called now?

The current official name is NVIDIA DGX Cloud Lepton. Lepton AI can remain as the old name, while the official documentation and SDK repositories have been updated to use the new platform name.

Is it a model API platform or a GPU cloud?

Both are covered, but it has a broader scope. It allows for the deployment of model endpoints, as well as providing capabilities for developing Pods, handling batch processing, managing training clusters, offering GPU resources, and supporting BYOC management.

How much is DGX Cloud Lepton?

There is no fixed uniform price that applies to all regions. The costs depend on the GPU supplier, region, model, quantity, duration, storage, and network; these details need to be confirmed in the quote provided by the workspace or company.

Which GPUs are supported?

The specific model depends on the chosen Node Group and supplier; the official ecosystem includes architectures such as NVIDIA Blackwell as well as H100, H200, A100, etc. The actual available inventory is subject to what is available in the workspace.

Can endpoints be automatically scaled down?

Yes. The documentation states that it is possible to reduce the scale to zero after one hour of inactivity, but a cold start may occur when making another call.

Can one connect one’s own GPU?

Yes, it is possible to add nodes that meet the system, driver, CUDA, storage, and network requirements to the platform using BYOC. The user remains responsible for their own hardware as well as for the underlying operations and maintenance.

Is Lepton AI open source?

The Python SDK and lep CLI are open source and licensed under the Apache-2.0 license; however, the entire DGX Cloud Lepton platform as well as the GPU marketplace are not open source projects.

How to prevent endpoints from being abused?

Enable endpoint tokens, restrict IP ranges, apply the principle of least privilege, set limits on scaling, and monitor requests and costs. Do not leave production endpoints publicly accessible by default.