Kokoro: Researchers Took Over Its Hugging Face Account

Kokoro: Researchers Took Over Its Hugging Face Account

A team of researchers took control of the Kokoro account on Hugging Face by registering the developer’s domain listed in the project’s donation file. They then reset the password and, in a controlled test, modified the model’s weights in 87 seconds so it would produce audio different from the input text. They did not upload the altered version to the platform.

The case exposes a model supply chain risk: a compromised account could distribute manipulated weights through an apparently routine update. The researchers say Hugging Face would not detect such changes as suspicious. The model is used in projects associated with Google, NVIDIA, AMD, Intel, and Amazon Web Services.

The domain made it possible to reset the password

According to BleepingComputer, the researchers registered the hexgrad.com domain over the weekend; it was listed in Kokoro’s public DONATE.md file. They then used the domain to initiate a password reset for the account that publishes the model. The article does not identify the other team members; it does attribute the explanation of the test to security researcher Alon Hertz.

The method did not rely on exploiting the model or modifying the application that runs it. In the demonstration, the team adjusted the neural weights and ran the result through the usual inference pipeline. Hertz described the test as follows:

“We changed what this model says by fine-tuning its neural weights. Same input text. Same voice file. Same configuration. Same inference code — not a single line touched in the application that runs it,”

— Alon Hertz, security researcher

The researchers ran the test locally and used a gaming GPU for 87 seconds to modify the weights. The model generated a poisoned output, but the altered files were not sent to Hugging Face. Hertz said doing so would have put production systems that depend on the model at risk.

The risk described does not require the package to contain conventional malware. According to the team, the manipulated version would retain the characteristics of a legitimate release: the publisher, download path, format, and architecture would all remain the same. File sizes would be similar, and the package would contain no malware; because only the weights would be changed, malware scanners would have no malicious code or signature to detect.

A familiar voice in AI projects

Kokoro is an open-weight text-to-speech model with 82 million parameters, according to the project page on Hugging Face. It converts text to speech, supports multiple languages, and offers more than 50 predefined voices, according to BleepingComputer. Its weights are distributed under the Apache license, and the project says it can be used for everything from personal projects to production environments.

The model page lists two published versions. v1.0, dated January 27, 2025, includes eight languages and 54 voices; v0.19, dated December 25, 2024, includes one language and 10 voices. The model card also lists the SHA-256 identifier for v1.0, although BleepingComputer does not claim that the researchers changed a specific version from the list.

VersionRelease dateLanguages and voicesAbbreviated SHA-256
v1.0January 27, 20258 languages and 54 voices496dba11
v0.19December 25, 20241 language and 10 voices3b0c392f

The model records nearly 13 million downloads per month on Hugging Face, according to BleepingComputer. The article says it is used in tools from Google, NVIDIA, and Intel for voice agents and on-device speech synthesis. AMD and Amazon Web Services have also published guides showing its use in AI agent workflows.

The project, attributed to Hexgrad, had not received an update since April 2025, according to BleepingComputer. The GitHub repository and the Python package on PyPI also became inactive around that time. That silence did not stop Kokoro from continuing to accumulate downloads and a presence in other projects.

Kokoro: investigadores tomaron su cuenta de Hugging Face
© Hugging Face

The test highlighted automatic updates

The scenario that concerns the team is that systems which automatically download new revisions could incorporate altered weights without any change to the application code. If a compromised publishing account distributed such an update, it could reach installations that trust the project’s usual channel. BleepingComputer cites data theft or code execution in supply chain attacks as possible consequences; the demonstration described, however, changed the voice output and did not involve publishing the manipulated weights.

Hertz also warned of a problem broader than account hijacking. In his view, an account that has not been taken over by a third party could also publish files that appear legitimate for a time and later distribute a malicious update. Trust in the publisher’s name alone would not prove that the person controlling the account is still its original owner.

The researcher summarized the criticism of the platform’s controls: “Hugging Face does not vet publishers. It doesn’t verify whether an account is still held by its original, legitimate owner — or by whoever quietly took it over last week. It doesn’t vet the models or datasets those accounts upload,”. He added that unverified weights and datasets also circulate through mirrors, converted versions, GitHub releases, package registries, deployment guides, and direct downloads.

Hugging Face had not yet responded

Hertz said he reported the issue to Hugging Face and expected a quick response, but had not received one. BleepingComputer also contacted the platform for comment and had still not heard back when it published its report. The article does not detail any measures taken by Hugging Face or announce changes to its account or release verification process.

The project’s DONATE.md file is the document that made it possible to identify the domain used in the password reset. It includes sponsorship and donation channels, including an email address associated with Hexgrad. The model page, meanwhile, says Kokoro has been deployed in commercial projects and APIs, and warns about websites using “kokoro” in the root domain that are not affiliated with the author or the model page.

The researchers presented Kokoro as a prominent example of how a change to weights can retain the appearance of an ordinary release. The test remained local: the altered weights never reached Hugging Face, and the platform had not responded to BleepingComputer’s inquiries.

Sources: BleepingComputer, ATLABYTE, Hugging Face, Hugging Face