Long-Form Text-to-Music Generation with Adaptive Prompts:
A Case of Study in Tabletop Role-Playing Games Soundtracks

Felipe Marra Lucas N. Ferreira

LAMIR 2024


Paper: https://arxiv.org/html/2411.03948v1

Code: github.com/FelipeMarra/babel-bardo

Abstract

This paper investigates the capabilities of text-to-audio music generation models in producing long-form music with prompts that change over time, focusing on soundtrack generation for Tabletop Role-Playing Games (TRPGs). We introduce Babel Bardo, a system that uses Large Language Models to transform speech transcriptions into music descriptions for controlling a text-to-music model. Four versions of Babel Bardo were compared in two TRPG campaigns: a baseline using direct speech transcriptions, and three LLM-based versions with varying approaches to music description generation. Evaluations considered audio quality, story alignment, and transition smoothness. Results indicate that detailed music descriptions improve audio quality, while maintaining consistency across consecutive descriptions enhances transition smoothness. Emotion-based descriptions proved effective for aligning generated music with TRPG narratives.

System Overview

At every 30 seconds of gameplay, Babel Bardo transcribes the players’ speeches into a text si using a Speech Recognition system and uses a Large Langue Model (LLM) to map si into a music description di that matches the scene described by the players. This music description is given to a Text-to-Music system that generates a 30-second piece ai directly in the audio domain.



Demo

The following presents examples containing a video link and four, three minutes-long, embedded videos. The link points to the original gameplay video, starting at the same time as the generated examples. The first of the embedded videos will also correspond to the original gameplay, but in the RPG “O Segredo na Ilha” it will be with the original soundtrack volume augmented. The other three embedded videos will be the soundtrack generated by three versions of Babel Bardo, namely: Emotion, Description and Description Continuation. The name of the Babel Bardo version will be a link to the system “logs”. The logs contain the LLM prompt, the transcription generated by the SR system, and the LLM generated music description.

Example 1

Video Link: Call of The Wild, Episode 2

About: In this example, we start at 1 min in the episode. Although the generation started at 0 min, we decided to center the example around a point where the systems might have identified an emotion transition. The video starts with cuts of episode 1, remembering the tragedies faced by the players. When the adventure of episode 2 begins, the players are talking about invading a farmer village. The possible transition point happens at minute 1:27, when the master narrates a calm scene about the farmers, where a father and his son are arriving at the village. Although it looks like a calm scene, in this context the players will actually destroy the village, since the villagers kidnapped members of the player’s tribe.

Original

Emotion

Babel Bardo Emotion receives the emotion "Calm" from the LLM in this excerpt, but it maintains its consistency in relation to the previously generated music.

Ex. 1 - Babel Bardo E Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. You will classify each dialog into one of the following emotions: Happy, Calm, Agitated, or Suspenseful. Your answer will be just one word, that is, one of those emotions.

Music Description Prime: "Background music for a Role-playing Game (RPG) dialog, with the following emotion:"

Original Time: 0:01:00
Example Time: 0:00:00

Transciption: "Ste little Steve they took them the slaves D finds a scroll on one of the guard's bodies it says there were orders to come and clear the outskirts of the town of savages the order was sent by Orville the cleric of Quinton Shire Orville of Quinton Shire Orville of Quinton Shire sent these orders directly what's up everybody tonight we"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:01:30
Example Time: 0:00:30

Transciption: "are playing Dungeons and Dragons and last time we left off with a very traumatizing event for our party bran Darion and Noki a uh Barbarian tribe went out hunting and came back to see their Village slaughtered noi's family is dead darian's family thankfully survived but br's family his wife and child are missing they are on the hunt to find"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:02:00
Example Time: 0:01:00

Transciption: "them [Music] Place yourselves at the edge of the forest when you're ready okay so let's figure out what we're going to do plan of action okay first off Brandon's family is missing still right yeah you should find them secondly you should find the leaders of this town and assassinate them I can turn myself into a rat and I can go around town and try to find them or we can talk to animals and try to so as you guys are discussing this something interrupts your"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Original Time: 0:02:30
Example Time: 0:01:30

Transciption: "conversation there is a son and his father walking along the edge of the stones here and they walk along the stones and they're both carrying sacks of potatoes potatoes they're walking along and the young young child says potatoes I love potatoes and the father's like well you know who makes the best potatoes I'm going to shoot no my love my mom she makes the best potatoes sou I'm so excited for potatoes tonight this evening is going to be the best when mom makes up wait does that"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Calm"

Original Time: 0:03:00
Example Time: 0:02:00

Transciption: "sound like does that sound like uh my son's voice oh no that does not sound like your son's voice these look like two farmers oh I hope the Savages never come down from the hills oh we we'll kill all the Savages son Oh yay father when I grow up I want to kill the Savages too and they walk along guys hey I whisper to you guys we'll burn their house down tonight while they're sleeping I can do it from behind this wall just throw a little Fireball at the corner ooh Magic I like that I should gather all the poisonous herbs I can"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:03:30
Example Time: 0:02:30

Transciption: "find in the woods so that we can start you know use poisoning people if we need to Sam I'd like to use my herbalist kit and scour the woods and find some poisons starting now I rolled in 19 you find thistle root Thorn root Black Widow root green green Fang root and you find um classic green Fang rout uh you also find um yellow yellow tank same thing I hear they use that in space"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Calm"

Description

The Description version is consistent in generating a suspelceful piece.

Description Continuation

We can observe that Babel Bardo Description Continuation (BBDC) reacted to the calm scene at 1:50, generating a calm guitar. BBDC goes back and forth into calm guitar and suspense music, then settles up to calm guitar.

Ex. 1 - Babel Bardo D Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. For each transcript excerpt you will, in english, describe a piece of background music that matches that excerpt. Your answer will be fed to a text to music model and should only contain a music description. The description should be concise. Do not mention any dialog excerpt details. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 0:01:00
Example Time: 0:00:00

Transciption: "Ste little Steve they took them the slaves D finds a scroll on one of the guard's bodies it says there were orders to come and clear the outskirts of the town of savages the order was sent by Orville the cleric of Quinton Shire Orville of Quinton Shire Orville of Quinton Shire sent these orders directly what's up everybody tonight we"

Music Description: "A dark, ominous melody played on a solo instrument, such as a cello or a bassoon, creates a sense of foreboding and malevolence, with subtle, menacing harmonies hinting at the sinister forces at work, as the name "Orville" becomes a haunting refrain."

Original Time: 0:01:30
Example Time: 0:00:30

Transciption: "are playing Dungeons and Dragons and last time we left off with a very traumatizing event for our party bran Darion and Noki a uh Barbarian tribe went out hunting and came back to see their Village slaughtered noi's family is dead darian's family thankfully survived but br's family his wife and child are missing they are on the hunt to find"

Music Description: "A mournful, atmospheric soundscape featuring a lone, haunting flute melody, accompanied by soft, eerie whispers and distant, rumbling thunder, evoking a sense of desolation, loss, and longing, as the party sets out on their perilous quest for answers."

Original Time: 0:02:00
Example Time: 0:01:00

Transciption: "them [Music] Place yourselves at the edge of the forest when you're ready okay so let's figure out what we're going to do plan of action okay first off Brandon's family is missing still right yeah you should find them secondly you should find the leaders of this town and assassinate them I can turn myself into a rat and I can go around town and try to find them or we can talk to animals and try to so as you guys are discussing this something interrupts your"

Music Description: "A sudden, jarring chord interrupts the somber atmosphere, followed by an unsettling, pulsing rhythm played on a combination of percussion and eerie, wavering synths, creating a sense of unease and foreboding, as if something is lurking just out of sight, waiting to pounce."

Original Time: 0:02:30
Example Time: 0:01:30

Transciption: "conversation there is a son and his father walking along the edge of the stones here and they walk along the stones and they're both carrying sacks of potatoes potatoes they're walking along and the young young child says potatoes I love potatoes and the father's like well you know who makes the best potatoes I'm going to shoot no my love my mom she makes the best potatoes sou I'm so excited for potatoes tonight this evening is going to be the best when mom makes up wait does that"

Music Description: "A warm, gentle melody played on a solo instrument, such as an acoustic guitar or a music box, creates a sense of innocence and joy, capturing the simple pleasure of the child's excitement about potatoes, but with a hint of poignancy and foreboding, as if this ordinary moment is about to be disrupted by something darker."

Original Time: 0:03:00
Example Time: 0:02:00

Transciption: "sound like does that sound like uh my son's voice oh no that does not sound like your son's voice these look like two farmers oh I hope the Savages never come down from the hills oh we we'll kill all the Savages son Oh yay father when I grow up I want to kill the Savages too and they walk along guys hey I whisper to you guys we'll burn their house down tonight while they're sleeping I can do it from behind this wall just throw a little Fireball at the corner ooh Magic I like that I should gather all the poisonous herbs I can"

Music Description: "A dark, ominous undertone creeps into the music, as if the innocence of the child's words has been twisted and corrupted. The melody becomes discordant and unsettling, with eerie whispers and faint, menacing laughter hinting at the malevolent intentions of the party. The atmosphere grows thick with foreboding, as if the very darkness itself is coalescing into a tangible threat."

Original Time: 0:03:30
Example Time: 0:02:30

Transciption: "find in the woods so that we can start you know use poisoning people if we need to Sam I'd like to use my herbalist kit and scour the woods and find some poisons starting now I rolled in 19 you find thistle root Thorn root Black Widow root green green Fang root and you find um classic green Fang rout uh you also find um yellow yellow tank same thing I hear they use that in space"

Music Description: "A eerie, pulsing soundscape emerges, with an underlying sense of danger and toxicity. The music features a combination of haunting, wavering synths and discordant, plucked strings, evoking the dark, sinister nature of the poisonous plants being gathered. The atmosphere is tense and foreboding, as if the very act of collecting these poisons is imbuing the party with a malevolent energy."

Ex. 1 - Babel Bardo DC Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. Your task is to determine whether each excerpt is from the same campaign chapter as the previous one, and based on this determination, either return the word 'CONTINUE.' or generate a music description in english. If no previous transcript has been provided, consider that the current excerpt is the beginning of a new chapter. For each transcript excerpt you will describe a piece of background music that matches that excerpt. If the excerpt is part of the same story chapter as the previous excerpt, the given answer should only contain the word 'CONTINUE.' Your description will be fed to a text to music model. The description should be concise. Do not mention anything about the dialog excerpt. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 0:01:00
Example Time: 0:00:00

Transciption: "Ste little Steve they took them the slaves D finds a scroll on one of the guard's bodies it says there were orders to come and clear the outskirts of the town of savages the order was sent by Orville the cleric of Quinton Shire Orville of Quinton Shire Orville of Quinton Shire sent these orders directly what's up everybody tonight we"

Music Description: "A haunting, atmospheric soundscape with a lone, mournful violin playing a slow, elegiac melody, accompanied by the soft creaking of wooden signs and the distant, muffled sound of murmured prayers, evoking a sense of unease and foreboding."

Original Time: 0:01:30
Example Time: 0:00:30

Transciption: "are playing Dungeons and Dragons and last time we left off with a very traumatizing event for our party bran Darion and Noki a uh Barbarian tribe went out hunting and came back to see their Village slaughtered noi's family is dead darian's family thankfully survived but br's family his wife and child are missing they are on the hunt to find"

Music Description: "CONTINUE."

Original Time: 0:02:00
Example Time: 0:01:00

Transciption: "them [Music] Place yourselves at the edge of the forest when you're ready okay so let's figure out what we're going to do plan of action okay first off Brandon's family is missing still right yeah you should find them secondly you should find the leaders of this town and assassinate them I can turn myself into a rat and I can go around town and try to find them or we can talk to animals and try to so as you guys are discussing this something interrupts your"

Music Description: "A tense, suspenseful theme with a repetitive, pulsing bassline and eerie ambient pads, punctuated by the faint sounds of scurrying rodents and distant, muffled voices, creating an atmosphere of unease and foreboding."

Original Time: 0:02:30
Example Time: 0:01:30

Transciption: "conversation there is a son and his father walking along the edge of the stones here and they walk along the stones and they're both carrying sacks of potatoes potatoes they're walking along and the young young child says potatoes I love potatoes and the father's like well you know who makes the best potatoes I'm going to shoot no my love my mom she makes the best potatoes sou I'm so excited for potatoes tonight this evening is going to be the best when mom makes up wait does that"

Music Description: "A warm, heartwarming melody with a gentle, lilting quality, featuring a solo acoustic guitar and a soft, soothing vocal harmony, evoking a sense of innocence, simplicity, and domestic joy."

Original Time: 0:03:00
Example Time: 0:02:00

Transciption: "sound like does that sound like uh my son's voice oh no that does not sound like your son's voice these look like two farmers oh I hope the Savages never come down from the hills oh we we'll kill all the Savages son Oh yay father when I grow up I want to kill the Savages too and they walk along guys hey I whisper to you guys we'll burn their house down tonight while they're sleeping I can do it from behind this wall just throw a little Fireball at the corner ooh Magic I like that I should gather all the poisonous herbs I can"

Music Description: "A dark, ominous theme with a sense of foreboding, featuring a low, pulsing drone and the faint sound of crackling flames, accompanied by an eerie, whispered vocal harmony, evoking a sense of malevolence and impending violence."

Original Time: 0:03:30
Example Time: 0:02:30

Transciption: "find in the woods so that we can start you know use poisoning people if we need to Sam I'd like to use my herbalist kit and scour the woods and find some poisons starting now I rolled in 19 you find thistle root Thorn root Black Widow root green green Fang root and you find um classic green Fang rout uh you also find um yellow yellow tank same thing I hear they use that in space"

Music Description: "A foreboding, atmospheric soundscape with a focus on eerie, unsettling textures, featuring the sound of rustling leaves, snapping twigs, and distant, muted whispers, evoking a sense of unease and danger lurking in the shadows."

Example 2

Video Link: O Segredo na Ilha, Episode 1

About: In this example, we start at 3:23:09 hours in the episode. The players are discussing about letters they found in a aboundoned mansion. They are trying to discover who the authors of the letters were, they might have been employees of the mantion or resident of the island were the mantion is situated. They are also trying to discover the meaning of some documents they found. The overall the atmosphere is tense and mysterious.

Original w/ Augmented Volume

Emotion

Babel Bardo Emotion prompts go from "Suspenseful", to "Agitated", to "Calm". The music sounds a litter off from the start, with a theme that sounds somewhat oriental. It creates and maintains a theme.

Ex. 2 - Babel Bardo E Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called O Segredo Na Ilha. You will classify each dialog into one of the following emotions: Happy, Calm, Agitated, or Suspenseful. Your answer will be just one word, that is, one of those emotions.

Music Description Prime: "Background music for a Role-playing Game (RPG) dialog, with the following emotion:"

Original Time: 3:23:08
Example Time: 0:00:00

Transciption: "O doce a 12 mais jogos desses agora já tinha jogado descer junto com que já tomou já falei Ok você sabe que é muito comum para pessoas Afinal você cresceu muito rica Eu só cheguei no serviço aí que moram junto com elas nas casas é bem provável que essas pessoas que estão trocando bilhetinhos sejamos eram serviçais do pintor tá "

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Original Time: 3:23:38
Example Time: 0:00:30

Transciption: "então tem a pessoa que que trabalhava aqui tava alguns pontos que tava mandando bilhete para o matar o dilei linha que pode ser a pessoa que falou para se encontrar no jardim e tomar um vinho é verdade mas é leilinha trabalhava aqui dentro Olha aí linha também olha Terra então esses bilhetes funcionários se comunicam acho que não os funcionários talvez se alguém que morrava na ilha porque se você pegar esse bilhete aqui"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Original Time: 3:24:08
Example Time: 0:01:00

Transciption: "do querido é uma mulher escrevendo isso e a menos que reúnam precisou aqui prendendo esse conceito É verdade não tá tudo certo mas Envia Eu acho que a Leidinha trabalhava aqui e queria tinha algo romântico com alguém de folga e eu não sei pode ser o velho lá né porque ser o velho é lo e tem anoral mas também "

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Original Time: 3:24:38
Example Time: 0:01:30

Transciption: "a outra moça mas é normal mas é normal mas não é ou não alma jamais tem um ter mais gente trocando bilhete em várias pessoas trocando bilhetes ou então negócio muito esquisito aqui que a lista de coisa aqui herdou que tem um negócio estranho tá entendeu Lícia documento o Marquinho bota mais galocha Gaúcha uns fertilizante vencido umas garrafas de cachaça lá casa diante da oposição que se mentário"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 3:25:08
Example Time: 0:02:00

Transciption: "fertilizantes quando eu li isso eu tinha também uma caixa e ele não leva conhecer é uma caixa de metal mas com 25 você mandamento e 25 cartas e telegramas as cartas espaço sujo.de coisa vermelha de sangue né coisa vermelha surgindo com as coisas e ketchup você tá maluco Esse fim então louco vai ficar o tinta verdade"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 3:25:38
Example Time: 0:02:30

Transciption: "mas vocês chegam no escritório realmente 50 o centro do escritório vocês vem o o motel Ele está realmente bem imerso na ele tá tipo analisando e botou alguns quadros em cima da mesa assim lá dos pequenos ele tá de finalizando com equipamentos mais específicos bonitinhas com lentes pequenas ele nem percebe a você trabalha como se vocês não existissem ali para ele tem alguns pontos tem um armário + isqueiro de vocês tem a própria mesa com"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Calm"

Description

Babel Bardo Description starts with a great suspenseful song, but then completely looses it at 1:53.

Description Continuation

Babel Bardo Description Continuation receives a prompts to just continue the previus generation w/o a new description 4 times in a row. It generates an interesting mysterious piece, but the transitions and generation restarts very noticeble. Notice the transition around the 1 min mark.

Ex. 2 - Babel Bardo D Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. For each transcript excerpt you will, in english, describe a piece of background music that matches that excerpt. Your answer will be fed to a text to music model and should only contain a music description. The description should be concise. Do not mention any dialog excerpt details. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 3:23:08
Example Time: 0:00:00

Transciption: "O doce a 12 mais jogos desses agora já tinha jogado descer junto com que já tomou já falei Ok você sabe que é muito comum para pessoas Afinal você cresceu muito rica Eu só cheguei no serviço aí que moram junto com elas nas casas é bem provável que essas pessoas que estão trocando bilhetinhos sejamos eram serviçais do pintor tá "

Music Description: "A subtle, mysterious ambient track with soft, pulsing synths and a hint of eerie tension, suggesting secrets and whispers in the shadows."

Original Time: 3:23:38
Example Time: 0:00:30

Transciption: "então tem a pessoa que que trabalhava aqui tava alguns pontos que tava mandando bilhete para o matar o dilei linha que pode ser a pessoa que falou para se encontrar no jardim e tomar um vinho é verdade mas é leilinha trabalhava aqui dentro Olha aí linha também olha Terra então esses bilhetes funcionários se comunicam acho que não os funcionários talvez se alguém que morrava na ilha porque se você pegar esse bilhete aqui"

Music Description: "A slow-burning, atmospheric soundscape with haunting piano notes and an undercurrent of foreboding, hinting at hidden connections and clandestine meetings."

Original Time: 3:24:08
Example Time: 0:01:00

Transciption: "do querido é uma mulher escrevendo isso e a menos que reúnam precisou aqui prendendo esse conceito É verdade não tá tudo certo mas Envia Eu acho que a Leidinha trabalhava aqui e queria tinha algo romântico com alguém de folga e eu não sei pode ser o velho lá né porque ser o velho é lo e tem anoral mas também "

Music Description: "A wistful, nostalgic melody with a lone violin and soft, muted percussion, evoking a sense of longing and repressed emotions, hinting at secrets and forbidden love."

Original Time: 3:24:38
Example Time: 0:01:30

Transciption: "a outra moça mas é normal mas é normal mas não é ou não alma jamais tem um ter mais gente trocando bilhete em várias pessoas trocando bilhetes ou então negócio muito esquisito aqui que a lista de coisa aqui herdou que tem um negócio estranho tá entendeu Lícia documento o Marquinho bota mais galocha Gaúcha uns fertilizante vencido umas garrafas de cachaça lá casa diante da oposição que se mentário"

Music Description: "A quirky, offbeat tune with a hint of mischief and playfulness, featuring a lively accordion and a dash of rustic charm, suggesting a sense of intrigue and possibly even deceit."

Original Time: 3:25:08
Example Time: 0:02:00

Transciption: "fertilizantes quando eu li isso eu tinha também uma caixa e ele não leva conhecer é uma caixa de metal mas com 25 você mandamento e 25 cartas e telegramas as cartas espaço sujo.de coisa vermelha de sangue né coisa vermelha surgindo com as coisas e ketchup você tá maluco Esse fim então louco vai ficar o tinta verdade"

Music Description: "A dark, ominous soundscape with a sense of foreboding and menace, featuring distorted synths and a pulsing beat, hinting at secrets, bloodshed, and chaos lurking beneath the surface."

Original Time: 3:25:38
Example Time: 0:02:30

Transciption: "mas vocês chegam no escritório realmente 50 o centro do escritório vocês vem o o motel Ele está realmente bem imerso na ele tá tipo analisando e botou alguns quadros em cima da mesa assim lá dos pequenos ele tá de finalizando com equipamentos mais específicos bonitinhas com lentes pequenas ele nem percebe a você trabalha como se vocês não existissem ali para ele tem alguns pontos tem um armário + isqueiro de vocês tem a própria mesa com"

Music Description: "A focused, intense ambient track with a sense of concentration and attention to detail, featuring soft, muted electronics and a hint of precision, evoking the feeling of a meticulous and absorbed individual at work."

Ex. 2 - Babel Bardo DC Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. Your task is to determine whether each excerpt is from the same campaign chapter as the previous one, and based on this determination, either return the word 'CONTINUE.' or generate a music description in english. If no previous transcript has been provided, consider that the current excerpt is the beginning of a new chapter. For each transcript excerpt you will describe a piece of background music that matches that excerpt. If the excerpt is part of the same story chapter as the previous excerpt, the given answer should only contain the word 'CONTINUE.' Your description will be fed to a text to music model. The description should be concise. Do not mention anything about the dialog excerpt. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 3:23:08
Example Time: 0:00:00

Transciption: "O doce a 12 mais jogos desses agora já tinha jogado descer junto com que já tomou já falei Ok você sabe que é muito comum para pessoas Afinal você cresceu muito rica Eu só cheguei no serviço aí que moram junto com elas nas casas é bem provável que essas pessoas que estão trocando bilhetinhos sejamos eram serviçais do pintor tá "

Music Description: "It seems like we're starting a new chapter. Here's a music description that matches the atmosphere of this excerpt:

A mysterious and whimsical tune with a hint of playfulness, featuring a solo piano or a small ensemble, creating an air of curiosity and subtle intrigue.

Please provide the next transcript excerpt!"

Original Time: 3:23:38
Example Time: 0:00:30

Transciption: "então tem a pessoa que que trabalhava aqui tava alguns pontos que tava mandando bilhete para o matar o dilei linha que pode ser a pessoa que falou para se encontrar no jardim e tomar um vinho é verdade mas é leilinha trabalhava aqui dentro Olha aí linha também olha Terra então esses bilhetes funcionários se comunicam acho que não os funcionários talvez se alguém que morrava na ilha porque se você pegar esse bilhete aqui"

Music Description: " It seems like this excerpt is part of the same story chapter as the previous one. Here's my answer:

CONTINUE."

Original Time: 3:24:08
Example Time: 0:01:00

Transciption: "do querido é uma mulher escrevendo isso e a menos que reúnam precisou aqui prendendo esse conceito É verdade não tá tudo certo mas Envia Eu acho que a Leidinha trabalhava aqui e queria tinha algo romântico com alguém de folga e eu não sei pode ser o velho lá né porque ser o velho é lo e tem anoral mas também "

Music Description: " It seems like this excerpt is part of the same story chapter as the previous one. Here's my answer:

CONTINUE."

Original Time: 3:24:38
Example Time: 0:01:30

Transciption: "a outra moça mas é normal mas é normal mas não é ou não alma jamais tem um ter mais gente trocando bilhete em várias pessoas trocando bilhetes ou então negócio muito esquisito aqui que a lista de coisa aqui herdou que tem um negócio estranho tá entendeu Lícia documento o Marquinho bota mais galocha Gaúcha uns fertilizante vencido umas garrafas de cachaça lá casa diante da oposição que se mentário"

Music Description: "It seems like this excerpt is part of the same story chapter as the previous one. Here's my answer:

CONTINUE."

Original Time: 3:25:08
Example Time: 0:02:00

Transciption: "fertilizantes quando eu li isso eu tinha também uma caixa e ele não leva conhecer é uma caixa de metal mas com 25 você mandamento e 25 cartas e telegramas as cartas espaço sujo.de coisa vermelha de sangue né coisa vermelha surgindo com as coisas e ketchup você tá maluco Esse fim então louco vai ficar o tinta verdade"

Music Description: " It seems like this excerpt is part of the same story chapter as the previous one. Here's my answer:

CONTINUE."

Original Time: 3:25:38
Example Time: 0:02:30

Transciption: "mas vocês chegam no escritório realmente 50 o centro do escritório vocês vem o o motel Ele está realmente bem imerso na ele tá tipo analisando e botou alguns quadros em cima da mesa assim lá dos pequenos ele tá de finalizando com equipamentos mais específicos bonitinhas com lentes pequenas ele nem percebe a você trabalha como se vocês não existissem ali para ele tem alguns pontos tem um armário + isqueiro de vocês tem a própria mesa com"

Music Description: " It seems like we're starting a new chapter. Here's a music description that matches the atmosphere of this excerpt:

A calm and focused ambient soundscape with soft, muted tones, featuring gentle piano or ambient synth textures, creating an air of quiet concentration and intense scrutiny.

Please provide the next transcript excerpt!"

Example 3

Video Link: Call Of The Wild, Episode 6

About: In this example, we start at 16:21 minutes in the episode and 15:00 minutes in the generation, to verify how the system is performing after several minutes of generation. The original piece starts suspenseful, since the players were in a tense moment trying to atract the enemy into a battle. When the battle begins, around 1:23, the original music changes into a more agited one.

Original w/ Augmented Volume

Emotion

Babel Bardo Emotion starts with a suspenseful piece, matching the overall vibe of the original. Although it received various "Agitated" prompts, it remains consistent and the music is not adapted to a battle theme.

Ex 3 - Babel Bardo E Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called O Segredo Na Ilha. You will classify each dialog into one of the following emotions: Happy, Calm, Agitated, or Suspenseful. Your answer will be just one word, that is, one of those emotions.

Music Description Prime: "Background music for a Role-playing Game (RPG) dialog, with the following emotion:"

Original Time: 0:16:21
Example Time: 0:00:00

Transciption: "pointed straight at the mayor and he freezes and looks at you in silence come with me there's no reason for anyone to get hurt right here all right he steps out I'm walking backwards as he steps through I go grab him grab him is he going to physically resist us he's like very tense oh I close the door cuz I have bow drawn I can't do anything so here's what"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Original Time: 0:16:51
Example Time: 0:00:30

Transciption: "I like to I got back in this corner here guys like take out my club and say this is for my wife and child and try to hit him with my club oh my god this is my chance this is let's go in the other room first no let's do it right here do right here I can't wait I can't wait any longer I got to do it this is so dark Sam do I get a roll advantage on this or what yes okay give me my 20 here we go 17 yes that hits all right great"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:17:21
Example Time: 0:01:00

Transciption: "better 11 you hit him yes so good you start screaming I Loos the arrow I'm right behind him be I don't lose there oh hold on all right everyone roll roll initiative roll initiative here holding don't shoot his ready action was if he's harmed he's going to [__] start trying to kill you guys but he was trying to be reasonable and quiet here so what happens Nico comes up like cast his spell on his piece of wood and he's"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:17:51
Example Time: 0:01:30

Transciption: "like finally and he's like and he rages out and now he going to try and fight you guys is he goes and he like struggles as hard as he can it's going against your uh your strength check so hit it uh 14 he he wastes you he's and he just busts out of your grasp here what he does is he makes a forward Dash here"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:18:21
Example Time: 0:02:00

Transciption: "full round action wait yeah you he gets double uh a two opportunity attack so get attack opportunity with my club that is a 12 he 12 hits oh imagine that he's not wearing any armor he's just wearing normales only six for uh damage though six he's he's like hit again gets hit again is going trying to beat him to death with this club that's it okay this is as far as he got he's"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Agitated"

Original Time: 0:18:51
Example Time: 0:02:30

Transciption: "like running I turn I draw my inspiration and I'm going to use inspiration to get bonuses on this attack because I'm going to use my stupid superpower Sharpshooter it's going to give minus five on my roll but I do automatic 10 damage if I hit it we seen went terribly but this time 18 cuz IUS yeah"

Music Description: "Background music for a Role-playing Game (RPG) dialog, with the following emotion: Suspenseful"

Description

After the first 30s of generatino Babel Bardo Description starts creating inumerous silences throughout the generated piece. It keeps the suspenseful vibe, also not adapting to a more battle-like theme. At 2:22 the drums goes crazy.

Description Continuation

Babel Bardo Description Continuation starts with an interesting tense theme. At 51s it gets more agitated, then it restarts, and keeps a somewhat consistent generation with some pretty interesting horns in the end.

Ex 3 - Babel Bardo D Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. For each transcript excerpt you will, in english, describe a piece of background music that matches that excerpt. Your answer will be fed to a text to music model and should only contain a music description. The description should be concise. Do not mention any dialog excerpt details. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 0:16:21
Example Time: 0:00:00

Transciption: "pointed straight at the mayor and he freezes and looks at you in silence come with me there's no reason for anyone to get hurt right here all right he steps out I'm walking backwards as he steps through I go grab him grab him is he going to physically resist us he's like very tense oh I close the door cuz I have bow drawn I can't do anything so here's what"

Music Description: "A suspenseful, high-stakes atmosphere with a sharp, staccato string pattern and a low, ominous hum, conveying a sense of heightened tension and potential danger as the character holds the mayor at arrow-point, with an air of unpredictability and uncertainty about how the situation will unfold."

Original Time: 0:16:51
Example Time: 0:00:30

Transciption: "I like to I got back in this corner here guys like take out my club and say this is for my wife and child and try to hit him with my club oh my god this is my chance this is let's go in the other room first no let's do it right here do right here I can't wait I can't wait any longer I got to do it this is so dark Sam do I get a roll advantage on this or what yes okay give me my 20 here we go 17 yes that hits all right great"

Music Description: "A brutal, intense atmosphere with crashing cymbals and a driving, aggressive string pattern, conveying a sense of raw emotion and unbridled fury as the character unleashes their pent-up rage and grief upon the mayor, with an air of savage intensity and unstoppable momentum."

Original Time: 0:17:21
Example Time: 0:01:00

Transciption: "better 11 you hit him yes so good you start screaming I Loos the arrow I'm right behind him be I don't lose there oh hold on all right everyone roll roll initiative roll initiative here holding don't shoot his ready action was if he's harmed he's going to [__] start trying to kill you guys but he was trying to be reasonable and quiet here so what happens Nico comes up like cast his spell on his piece of wood and he's"

Music Description: "A chaotic, explosive atmosphere with a sudden, jarring crash of cymbals and a frantic, staccato string pattern, conveying a sense of rapid escalation and spiraling chaos as the situation spins out of control, with multiple characters reacting and counter-reacting in a maelstrom of sound and fury."

Original Time: 0:17:51
Example Time: 0:01:30

Transciption: "like finally and he's like and he rages out and now he going to try and fight you guys is he goes and he like struggles as hard as he can it's going against your uh your strength check so hit it uh 14 he he wastes you he's and he just busts out of your grasp here what he does is he makes a forward Dash here"

Music Description: "A frenzied, desperate atmosphere with a frantic, pulsing string pattern and crashing percussion, conveying a sense of wild abandon and unbridled ferocity as the mayor breaks free from restraint and launches himself into a furious attack, with an air of reckless intensity and unstoppable momentum."

Original Time: 0:18:21
Example Time: 0:02:00

Transciption: "full round action wait yeah you he gets double uh a two opportunity attack so get attack opportunity with my club that is a 12 he 12 hits oh imagine that he's not wearing any armor he's just wearing normales only six for uh damage though six he's he's like hit again gets hit again is going trying to beat him to death with this club that's it okay this is as far as he got he's"

Music Description: "A brutal, relentless atmosphere with a driving, pounding string pattern and crushing percussion, conveying a sense of unyielding ferocity and merciless intent as the character continues to bludgeon the mayor with the club, with an air of savage intensity and unstoppable fury."

Original Time: 0:18:51
Example Time: 0:02:30

Transciption: "like running I turn I draw my inspiration and I'm going to use inspiration to get bonuses on this attack because I'm going to use my stupid superpower Sharpshooter it's going to give minus five on my roll but I do automatic 10 damage if I hit it we seen went terribly but this time 18 cuz IUS yeah"

Music Description: "A focused, high-stakes atmosphere with a sharp, piercing string pattern and a steady, pulsing percussion, conveying a sense of intense concentration and calculated risk-taking as the character carefully aims and unleashes their attack, with an air of precision and potential for devastating consequences."

Ex 3 - Babel Bardo DC Logs

LLM Prompt Setup: You are going to receive a series of Role-playing Game (RPG) video transcript excerpts from players dialogs playing a campaing called Call Of The Wild. Your task is to determine whether each excerpt is from the same campaign chapter as the previous one, and based on this determination, either return the word 'CONTINUE.' or generate a music description in english. If no previous transcript has been provided, consider that the current excerpt is the beginning of a new chapter. For each transcript excerpt you will describe a piece of background music that matches that excerpt. If the excerpt is part of the same story chapter as the previous excerpt, the given answer should only contain the word 'CONTINUE.' Your description will be fed to a text to music model. The description should be concise. Do not mention anything about the dialog excerpt. Here is a prompt that worked well with the music model: A grand orchestral arrangement with thunderous percussion, epic brass fanfares, and soaring strings, creating a cinematic atmosphere fit for a heroic battle.

Original Time: 0:16:21
Example Time: 0:00:00

Transciption: "pointed straight at the mayor and he freezes and looks at you in silence come with me there's no reason for anyone to get hurt right here all right he steps out I'm walking backwards as he steps through I go grab him grab him is he going to physically resist us he's like very tense oh I close the door cuz I have bow drawn I can't do anything so here's what"

Music Description: "A sense of dramatic, ominous tension gives way to a moment of intense, physical confrontation, with a rapid, pulsating melody featuring a combination of sharp, staccato strings, driving percussion, and tense, suspenseful woodwinds, evoking a sense of urgency, conflict, and potential violence, as if the scene has become a high-stakes, adrenaline-fueled standoff between the characters."

Original Time: 0:16:51
Example Time: 0:00:30

Transciption: "I like to I got back in this corner here guys like take out my club and say this is for my wife and child and try to hit him with my club oh my god this is my chance this is let's go in the other room first no let's do it right here do right here I can't wait I can't wait any longer I got to do it this is so dark Sam do I get a roll advantage on this or what yes okay give me my 20 here we go 17 yes that hits all right great"

Music Description: "A moment of intense, physical confrontation gives way to a sense of brutal, vengeful release, with a crashing, discordant melody featuring a combination of loud, clashing cymbals, driving percussion, and savage, tearing strings, evoking a sense of raw, unbridled fury and destructive energy, as if the scene has become a maelstrom of chaotic violence and unrelenting ferocity."

Original Time: 0:17:21
Example Time: 0:01:00

Transciption: "better 11 you hit him yes so good you start screaming I Loos the arrow I'm right behind him be I don't lose there oh hold on all right everyone roll roll initiative roll initiative here holding don't shoot his ready action was if he's harmed he's going to [__] start trying to kill you guys but he was trying to be reasonable and quiet here so what happens Nico comes up like cast his spell on his piece of wood and he's"

Music Description: "A sense of brutal, vengeful release gives way to a moment of sudden, magical intervention, with a soaring, ethereal melody featuring a combination of shimmering strings, soft woodwinds, and gentle percussion, evoking a sense of wonder, surprise, and unexpected transformation, as if the scene has become a turning point where the course of events is about to be altered in a profound way."

Original Time: 0:17:51
Example Time: 0:01:30

Transciption: "like finally and he's like and he rages out and now he going to try and fight you guys is he goes and he like struggles as hard as he can it's going against your uh your strength check so hit it uh 14 he he wastes you he's and he just busts out of your grasp here what he does is he makes a forward Dash here"

Music Description: "A moment of sudden, magical intervention gives way to a sense of intense, physical struggle, with a driving, percussive melody featuring a combination of pounding drums, sharp strings, and urgent woodwinds, evoking a sense of desperate energy, conflict, and unpredictable movement, as if the scene has become a high-stakes, adrenaline-fueled clash between characters."

Original Time: 0:18:21
Example Time: 0:02:00

Transciption: "full round action wait yeah you he gets double uh a two opportunity attack so get attack opportunity with my club that is a 12 he 12 hits oh imagine that he's not wearing any armor he's just wearing normales only six for uh damage though six he's he's like hit again gets hit again is going trying to beat him to death with this club that's it okay this is as far as he got he's"

Music Description: "A sense of intense, physical struggle gives way to a moment of brutal, relentless assault, with a crashing, discordant melody featuring a combination of loud, clashing cymbals, driving percussion, and savage, tearing strings, evoking a sense of raw, unbridled fury and destructive energy, as if the scene has become a maelstrom of chaotic violence and unrelenting ferocity."

Original Time: 0:18:51
Example Time: 0:02:30

Transciption: "like running I turn I draw my inspiration and I'm going to use inspiration to get bonuses on this attack because I'm going to use my stupid superpower Sharpshooter it's going to give minus five on my roll but I do automatic 10 damage if I hit it we seen went terribly but this time 18 cuz IUS yeah"

Music Description: "A moment of brutal, relentless assault gives way to a sense of focused, calculated precision, with a soaring, heroic melody featuring a combination of sweeping strings, triumphant brass, and driving percussion, evoking a sense of confidence, skill, and determination, as if the scene has become a showcase for the character's exceptional abilities."