What is voice cloning?

Voice cloning is text-to-speech with one extra input. An ordinary text-to-speech system takes text and produces speech in a voice the model already knows. A cloning system takes text *and* a recording of a particular person, and produces speech in that person's voice.

That extra input is called the reference clip, and almost everything about the quality of the result traces back to it.

What the model does with your recording

Modern cloning is zero-shot: nothing is trained on you. The model was trained once, on thousands of speakers, to learn a general map of what human voices can sound like. When you supply a reference clip, it encodes that clip into a compact description of your voice — roughly, where you sit in that map — and conditions its output on that description.

This is why cloning takes seconds rather than hours. There is no per-user training step, no model file that belongs to you, and no waiting. It is also why the reference clip has more influence than any setting: the model is not learning your voice, it is *reading* it, once, from that one file.

The older approach — fine-tuning a model on 30 minutes of someone's speech — still produces slightly better results at the extreme end, and is what studio voice-replica projects use. For everyday creator work, zero-shot from a few seconds closed most of that gap between 2023 and 2026.

What a reference clip actually needs to contain

It needs to be a representative sample of you talking normally. Concretely, that means:

• Enough speech. A few seconds of actual talking, not a few seconds of file. Clips that are mostly silence fail.
• Clean signal. Not clipped, not distorted, not recorded at arm's length in a bathroom. Mild room noise is survivable; a hard-clipped recording is not.
• Your normal register. The clone inherits the read. Record over-projected and every script afterwards sounds over-projected.
• The right language. A reference in one language is a worse basis for output in another, and the difference is audible. Record in the language you intend to publish in.

How accurate is it, honestly

For a scripted read, close enough that regular listeners accept it. Timbre, accent and speaking rate transfer well. Someone who knows your voice intimately can usually still tell, particularly at the start of a sentence and at the ends of long phrases.

For performance, it is clearly weaker. Shouting, laughing mid-sentence, comic timing, a deliberate break in the voice — these are choices a performer makes, and the model has no reason to make them because nothing in your text told it to. If your project depends on that, hire a human. We would rather say so than sell you a subscription you will cancel.

Anyone claiming their clone is indistinguishable from you in all conditions is selling something. The useful question is not 'is it perfect' but 'is it good enough for this job', and for narration the answer has been yes for a while.

Voice cloning, voice conversion and text to speech

Three related things that get used interchangeably and should not be:

• Text to speech turns text into speech in a voice the system already has. Input: text.
• Voice cloning turns text into speech in a voice you supplied a sample of. Input: text plus a reference recording.
• Voice conversion takes an existing *recording* of someone speaking and re-renders it in another voice, keeping the original delivery and timing. Input: audio plus a target voice.

Voice conversion is the one that preserves performance, because a human performed it — which is why dubbing workflows reach for it. Mayantar does cloning and text to speech; the longer comparison is in voice cloning vs text to speech.

The part that matters more than the technology

Cloning is the AI capability with the most obvious misuse case, and the shortest distance between capability and harm. Three seconds of someone's voice from a public video is enough for a convincing fake. That is not a hypothetical — voice-based fraud against families and finance departments is an established pattern.

Which is why any serious platform gates it: explicit consent recorded at the point of cloning, screening of submitted reference clips, watermarking of generated audio, and a real route to report misuse. What Mayantar does specifically is on the safety and consent page, and if you are worried about your own voice, read how to protect your voice from cloning.