EN

Running OpenAI’s GPT-OSS-20B Locally with Open WebUI (Full Setup Guide)

#AI Tools
2025-12-05 18:288 min read

Content Introduction
Ask Questions

In this video, Rob walks viewers through a hands-on setup of the Open Web UI on an NVIDIA DGX Spark, designed for interaction with OpenAI's GBOSS20B model, which contains 20 billion parameters. The tutorial highlights the configuration processes that can be performed across various hardware platforms but is centered around a desktop Blackwell system. Rob explains how to utilize the NVIDIA Sync tool for creating SSH port mappings and managing Docker containers for running the Open Web UI application. He details the steps to download necessary Docker images, set up volume mounts for data storage, and access the GPU and memory usage while running the app. After establishing the admin account and installing the model, Rob demonstrates running a basic query. The video aims to provide a comprehensive guide for users interested in deploying large language models with NVIDIA hardware.

Key Information

  • Rob introduces a hands-on tutorial on setting up Open Web UI on a DGX Spark to interact with a local version of OpenAI's GBOSS20B model, which has 20 billion parameters.
  • The configuration can be done on various hardware, but the demonstration is specifically on an NVIDIA DGX Spark system.
  • The video covers the use of Nvidia Sync to manage GPU and memory utilization while setting up the configuration.
  • Rob showcases downloading Docker images, creating containers, and configuring them with port mappings for Open Web UI access via SSH.
  • The tutorial includes setting up custom applications in the Nvidia Sync UI for streamlined access to the Open Web UI.
  • Rob emphasizes observing GPU and RAM utilization during interactions with the model, noting spikes in usage when queries are processed.
  • He concludes by encouraging viewers to try setting up similar configurations on their systems with compatible GPUs.

Timeline Analysis

Content Keywords

Open Web UI

Rob introduces a hands-on walkthrough demonstrating how to set up Open Web UI on an NVIDIA DGX Spark to interact with a local version of OpenAI's GBOSS20B model, which has 20 billion parameters. The video shows various configurations that can be performed on different hardware.

NVIDIA DGX Spark

Details about the NVIDIA DGX Spark system used for this setup are covered, including its performance monitoring via the Nvidia sync utility and GPU utilization during different tasks.

Docker Container

The process to download and run the Open Web UI Docker container is explained, including creating the container with port mappings and ensuring it interacts correctly with the host system's ports.

Model Installation

Rob walks through the installation of the GPTOSS20 billion parameter model, emphasizing the expected performance increases in subsequent queries as the model is loaded into memory.

Response Testing

The video concludes with testing the model's capabilities by querying it to deliver jokes and more complex instructions, while keeping an eye on the GPU and RAM utilization throughout the process.

Nvidia Sync

Instructions on how to configure and utilize Nvidia Sync for managing and launching applications on the DGX Spark are provided, along with details on how to create a custom app within the interface.

Performance Monitoring

Users are encouraged to monitor GPU and RAM usage during operations to ensure optimal performance and to evaluate the responsiveness of the system as different queries are made.

Related questions&answers

Who is Rob?

Rob is the host of the video and is welcoming viewers to his lab.

What is the main topic of the video?

The video is a hands-on walkthrough showing how to set up Open Web UI on a DGX Spark to interact with OpenAI's GBOSS20B model.

What is OpenAI's GBOSS20B?

GBOSS20B is an OpenAI open weight model with 20 billion parameters.

Which hardware is being used in the demo?

The recording is being done on an NVIDIA DGX Spark desktop Blackwell system.

What tools are mentioned for accessing the open web UI?

Rob uses the Nvidia sync utility to access the open web UI app and the Spark terminal.

How does Rob monitor GPU utilization?

Rob opens a dashboard using Nvidia sync to monitor unified memory and GPU utilization on the Spark.

What needs to be done before using Open Web UI?

Rob downloads the Docker image and creates a Docker container to run the Open Web UI.

What is the purpose of the Docker container?

The Docker container is used to run the Open Web UI and enable connection via an SSH tunnel.

What is a custom app in the context of Nvidia sync?

A custom app in Nvidia sync allows users to manage and launch applications easily, like the Open Web UI.

What type of questions does Rob ask OpenAI's model?

Rob asks the model simple questions, such as a joke, to test its functionality.

How does Rob confirm that the app is running?

Rob uses the 'docker ps' command to check if the container is running.

What happens to GPU utilization during model interactions?

GPU utilization spikes when the model is processing a question and returns to zero once the response is delivered.

Can the setup be used on other systems?

Yes, the setup can be used on any system with a compatible GPU.

More video recommendations

Share to: