A pair of undergrads, neither with intensive AI experience, say that they’ve created an overtly out there AI mannequin that may generate podcast-style clips just like Google’s NotebookLM.
The marketplace for artificial speech instruments is huge and rising. ElevenLabs is among the largest gamers, however there’s no scarcity of challengers (see PlayAI, Sesame, and so forth). Traders imagine that these instruments have immense potential. In response to PitchBook, startups creating voice AI tech raised over $398 million in VC funding final yr.
Toby Kim, one of many Korea-based co-founders of Nari Labs, the group behind the newly launched mannequin, stated that he and his fellow co-founder began studying about speech AI three months in the past. Impressed by NotebookLM, they needed to create a mannequin that supplied extra management over generated voices and “freedom within the script.”
Kim says they used Google’s TPU Analysis Cloud program, which gives researchers with free entry to the corporate’s TPU AI chips, to coach Nari’s mannequin, Dia. Weighing in at 1.6 billion parameters, Dia can generate dialogue from a script, letting customers customise audio system’ tones and insert disfluencies, coughs, laughs, and different nonverbal cues.
Parameters are the inner variables fashions use to make predictions. Usually, fashions with extra parameters carry out higher.
Accessible from the AI dev platform Hugging Face and GitHub, Dia can run on most fashionable PCs with at the very least 10GB of VRAM. It generates a random voice except prompted with an outline of an meant type, however it might probably additionally clone an individual’s voice.
In TechCrunch’s transient testing of Dia by way of Nari’s net demo, Dia labored fairly nicely, uncomplaining producing two-way chats about any topic. The standard of the voices appears aggressive with different instruments on the market, and the voice cloning operate is among the many best this reporter has tried.
Right here’s a pattern:
Like many voice mills, Dia affords little in the best way of safeguards, nevertheless. It’d be trivially simple to craft disinformation or a scammy recording. On Dia’s challenge pages, Nari discourages abuse of the mannequin to impersonate, deceive, or in any other case interact in illicit campaigns, however the group says it “isn’t accountable” for misuse.
Nari additionally hasn’t disclosed which knowledge it scraped to coach Dia. It’s attainable Dia was developed utilizing copyrighted content material — a commenter on Hacker Information notes that one pattern sounds just like the hosts of NPR’s “Planet Cash” podcast. Coaching fashions on copyrighted content material is a widespread however legally doubtful observe. Some AI firms declare that truthful use shields them from legal responsibility, whereas rights holders assert that truthful use doesn’t apply to coaching.
In any occasion, Kim says Nari’s plan is to create an artificial voice platform with a “social side” on high of Dia and bigger, future fashions. Nari additionally intends to launch a technical report for Dia, and to increase the mannequin’s help to languages past English.
