Garlic LLM brings Qwen3.5-9B to an offline PS4

Garlic LLM brings Qwen3.5-9B to an offline PS4

A PlayStation 4 can run the Qwen3.5-9B language model without a PC or internet thanks to Garlic LLM, an open-source project by developer Leonardo Alegría. On a base or Slim PS4 with GoldHEN, the application generates between 31 and 39 tokens per second in a conversation and reaches 43 when writing code.

The proposal does not involve installing a different operating system or connecting the console to a computer to process responses. Garlic LLM is a homebrew application that runs on the PS4's own GPU. The project is distributed for free under the GPLv3 license; its package includes the application and the necessary models.

The GPU receives commands without going through a conventional driver

The PS4 does not provide homebrew applications with a compute interface or machine-learning libraries. Alegría addressed this shortcoming with custom compute kernels, written for AMD's Graphics Core Next (GCN) architecture. The application sends PM4 commands directly to the GPU instead of relying on a conventional graphics driver.

The project website explains that these kernels perform model operations such as matrix multiplications, attention, and normalization. The code also uses the OpenOrbis open-source toolchain. In this architecture, the CPU handles tasks including the tokenizer, interface, and text; the GPU runs the model computations.

The name Garlic comes from a term used for the memory path that the GPU accesses directly. Its sibling memory is called Onion. The project explains that the model weights are stored on the Garlic side, a decision tied to how the application makes use of the console's memory.

Quantization to fit in shared memory

The PS4 has 8 GB of shared GDDR5 memory between the CPU and GPU, according to Frandroid. To reduce the size of Qwen3.5-9B, Garlic LLM uses custom 4-bit and 3-bit quantization. Frandroid notes that this compression brings the model down to around 4.5 GB and entails a slight loss of accuracy.

The project identifies the quantizations as Q4 and Q3. AI News Today's technical specifications describe a mixed scheme: most weights use 4 bits, while the two largest weight matrices in each layer use 3 bits. According to the outlet, the model takes up 4.5 GiB, around 400 MB less than with pure Q4 quantization.

The memory must also accommodate the context cache and, if used, the voice models. AI News Today puts the total runtime memory usage at 5.23 GiB: 4.5 GiB for the main model, 0.33 GiB for speech recognition, and 0.38 GiB for synthesis. The project confirms a 16,000-token context window, meaning the amount of text it can keep in mind during an interaction.

ItemPublished figure
Chat modelQwen3.5-9B
QuantizationCustom Q4 and Q3
Conversation speed31–39 tokens per second
Code generation speedUp to 43 tokens per second
Context window16,000 tokens
Package 1.195.66 GB on Hugging Face
Garlic LLM lleva Qwen3.5-9B a una PS4 sin conexión
© Garlic LLM

Speculative prediction speeds up generation

Speed does not depend solely on reducing the model's size. Garlic LLM takes advantage of Qwen's Multi-Token Prediction (MTP) feature to use speculative decoding: the system predicts tokens, which the full model then verifies. The project website attributes the performance of 31 to 39 tokens per second in chat and up to 43 when writing code to this technique, together with custom Q4 kernels.

The maximum figure does not describe every conversation. According to the project, the usual range is 31 to 39 tokens per second; 43 is for code generation. To give a sense of the pace, its page estimates that a 60-token response appears in under two seconds and a 300-token response in under nine. Filling the 16,000-token context at that rate would take around eight minutes.

Frandroid offers another comparison: it notes that a PC with a graphics card with 10 GB of VRAM can run the same model without modification. It also cites a model rating based on a Qwen3.5-9B specification sheet: ranked 146th out of 262 models, with a score of 61 in coding. These figures put the model's capabilities in context, but do not change the main demonstration, which is running it on the console's hardware.

Voice in seven languages and dictation in 25

Garlic LLM is not limited to text chat. The application lets users speak to the model and hear its responses through Chatterbox, with voice in seven languages: Spanish, English, Portuguese, Russian, Italian, French, and German. For speech-to-text transcription, it includes Parakeet TDT 0.6B v3, an NVIDIA model that supports dictation in 25 European languages.

There is also a Python mode: the model writes programs in MicroPython, and the console itself runs them. The project also describes a Think mode, disabled by default, that lets the model reason before answering. The Garlic LLM page shows a demonstration recorded on the console in which the model writes and runs a turtle program, then answers a spoken question.

Inference, speech recognition, and voices run locally, according to the project. No network connection is needed while using the application, and the website says these components are included in the package. The cited demonstration was run with temperature set to 0 for reproducibility; the chat default is 0.7.

GoldHEN, free space, and compatibility

Installing Garlic LLM requires a PS4 with GoldHEN and a homebrew environment. GoldHEN allows community applications to run on modified consoles; the project page warns that it is installed at the user's own risk. The application itself is a standard homebrew package and does not contain an exploit, according to Hugging Face.

The Garlic-LLM-1.19.pkg package takes up 5.66 GB and includes the application and its models. Installation requires around 6 GB of free space; if the package is first copied to the console's internal drive, 12 GB is needed. The official website says it can be updated by installing the new package over the old one. The source code is available under GPLv3, and the included models retain their own licenses.

Compatibility requires some clarification: the project pages and initial coverage describe a base or Slim PS4 with firmware 11.52 as the tested hardware. The official website also includes reports from users who ran it on a Slim with 9.04 and 14.00. For the PS4 Pro, the main page still lists it as untested, but its FAQ records two user tests: one with version 1.19 and firmware 12.02, and another with a 1.20 build made from the source code and firmware 11.00.

Those tests on the Pro produced different results. The first user measured between 25 and 45 tokens per second; the second, between 35 and 41. The latter also tested a native PS4 Pro mode that reached up to 56 tokens per second, but that mode is still under review and is not included in the download. The project clarifies that the published speeds for the base or Slim PS4 were measured with firmware 11.52. The PS5 is not compatible: Garlic LLM is written for the PS4 GPU and software.

A free project with a paid guide

In addition to the free package, Alegría offers a 147-page guide titled Nine Billion Parameters on a PS4, for $19 or more. The website explains that the application and code remain under GPLv3 and that the guide helps fund the work. It also invites people to support the project to expand testing on other consoles and speed up the kernels.

Frandroid presents Garlic LLM as a way to give a console another use, noting that, according to its article, it has long been sidelined in game development in favor of the PS5. Alegría credits some of the project to the free tools he used:

«would not exist without them»

— Leonardo Alegría, developer of Garlic LLM

The next step proposed by the project itself is to test more consoles and improve performance. For now, the download available on Hugging Face is version 1.19, at 5.66 GB; the native PS4 Pro mode that reached 56 tokens per second is not yet part of it.

Sources: Tom's Hardware, HardZone, Garlic LLM, Hugging Face (project page), Frandroid, AI News Today