SkillCurrent
Estimate the GPU memory needed to run a Hugging Face model (weights, KV cache and inference overhead) and whether it fits on given GPUs, without downloading it. Use when asked how much VRAM a model needs, whether it fits on a GPU, the longest context that fits, or which GGUF quantization to pick.
0
Save adds this to your Playground favorites. Hearts show appreciation.
Set it up
Pick the app you use.
hf-mem
- 1
Open Plugins → Skills
Left sidebar → Plugins, then the Skills tab.
- 2
Add → Upload from your computer
Pick the SKILL.md (or Create with editor and paste the text), then Create.
- 3
Use it with /
In a chat, type / and pick the skill, then write your request.
Show the skill file
Codemd
# hf-mem ```bash uvx hf-mem <model> -o json [options] ``` `<model>` is a Hub ID (`org/name`, `org/name@revision`), a Hub URL, a local path, or `org/name-GGUF:Q4_K_M` for one GGUF quantization. Options: - `--gpu-memory 24` (GiB per GPU) or `8x80` (8 GPUs): adds `fit`. - `--max-model-len N`: context length, defaults to the model's maximum, which can dominate the total. - `--batch-size N`: concurrent sequences, defaults to 1. - `--kv-cache-dtype fp8` (vLLM names for Safetensors, llama.cpp cache types such as `q8_0` for GGUF). - `--file <file or variant>` and `--format safetensors|gguf|onnx` to pick the weights. - `--max-num-batched-tokens 2048` on A100s. - Gated or private models need `HF_TOKEN` or `--hf-token`. JSON (sizes in bytes, divide by 2^30 for GiB): - `total` is the GPU memory for one copy of the model: `weights` + `kv_cache` + `overhead` (vLLM for Safetensors, llama.cpp for GGUF, none for ONNX). - `offloaded` are weights the engine keeps in CPU memory: excluded from `weights` and `total`, but needed as free system RAM. - `fit` (with `--gpu-memory`): `fits`, `needed` GPUs, `max_model_len` (the longest context that fits), and the `required` vs `usable` memory across the given GPUs. Use `required` / GPUs for the memory per GPU, not `total` / GPUs, as each GPU also holds its own overhead and replicated KV cache. - GGUF and ONNX repositories with several files return a `files` list to compare quantizations; pick one with `:TAG` or `--file` to get `fit`. - Report `warnings` to the user, and errors explain how to fix them.
Read a skill before you install it: it tells the AI what to do. Promptu never installs or runs anything.

