Show HN: Mofi – Content-aware fill for audio to change a song to any duration

[jaflo]

I worked on a web service that allows you to import a song and define a target length that the song will be shortened or lengthened to. It does this by analyzing the song and finding repeating audio patterns. This is helpful for making any song match a video or performance with a set duration. You can also specify areas of the song to prefer or avoid. An example is available here: https://mofi.loud.red/edit/8bd3fdf780f8c3927e41029f3b957f8a7...

The cool thing is that after the song is analyzed on the server, the client can recompute and preview the results completely client-side through an implementation that uses Web Workers and WebAssembly. The audio previewing uses Tone.js. I am thinking of writing up some more details about the implementation in the future.

I'm still working on a way to explain this easily, but I like the idea of carrying over the concept of content-aware fill from images to audio.

Please let me know if you have any comments or questions!

[Philip-J-Fry]

There used to be something called the eternal jukebox I think. But you could give it a song and it would endlessly loop the audio like you have here. But it would also switch it up and change the parts it looped. And it would have a nice visualisation showing how the parts of the song connect and the different paths it takes.

I think it was https://eternalbox.dev/ since that's all I can find on Google. But that site is down.

[gffrd]

Yes! It was "Infinite Jukebox," created by Paul Lamere … it was awesome because it would analyse a track, then visualize its "components" and you could watch as the new "infinite" track looped back on itself and jumped from point-to-point in the original track to create an everlasting one.

He created some excellent products from the Rdio API, and later Spotify … and I believe his analysis engine ended up being the foundation upon which Spotify's _play more tracks like these_ capability is based.

Looks like he's moved over to publish on Substack—there's a recent(ish) post reflecting on 10 years of Infinite Jukebox: https://musicmachinery.substack.com/p/the-infinite-jukebox-1...

[jaflo]

This project actually builds on the same analysis techniques as the Infinite/Eternal Jukebox! I really liked it when the original was up. I think they also had a version just for Gangnam Style along with a matching infinite video!

There is this version now, but it does not allow custom uploads anymore: https://eternalboxmirror.xyz/jukebox_index.html

[npteljes]

It was Infinite Jukebox. The subreddit is still there: https://www.reddit.com/r/infinitejukebox/

[pfd1986]

Neat. Went down a rabbit hole and ended up finding this out. Quite interesting

https://publicinfrastructure.org/podcast/80-nick-seaver/

[josuepeq]

This is neat, I have wondered if anything of this nature existed, in the past, as a child of the 80s/90s attempting to master the art of the perfect mixtape… 30 minutes a side down to the second the tape runs out, would be a win.

But for today’s music, shortening the 2010s/2020 already shorter lengths would mean a song might not be more than a minute in length. On average, full unedited tracks today end up being a bit shorter than they used to be, solely due to the economics of streaming. Rather than paying for the content second by second, it is done by paying per track play. The result is a lot of 2 minute tracks, which were produced with the “verse” parts getting jammed together into the “chorus” with no break in vocals, which also uses pitch adjustments, “the “bridge” is an afterthought that is terrible, or more recently, nonexistent……… Instrumental solo? Anyone? Bueller?

Music is no longer anticipated, budgeted for, and purchased on launch day with great fanfare. We have grown accustomed to the idea that we should have everything available at our fingertips, and as a consequence of this we get exactly what we pay for.

[robinsonb5]

You're not wrong about modern songs - but that's not really the use case for this tool:

A few days ago I was given the task of creating a corporate video - just a rolling slideshow for a shop-window display. Then suddenly it was going to go on YouTube as well - so needed some music! I found a suitable track but needed to edit it for length so its closing chord coincided with the credits card at the end of the video.

This tool might have saved me the bother of splicing the music in Audacity.

[asdff]

Music is a self eating beast these days. Any new artist is competing with not only all contemporary artists but all past artists too, since its so trivial to just go on spotify and find a playlist with the "best alternative albums of all time" and find all the usual suspects of the last 25 years or so. It's really unprecedented for old material to be at an even pedestal as new material. These older already monied acts tend to get the bigger venues in town and tour all the time too, so it extends beyond the digital realm. Not to mention how many small act sized venues have closed and no longer exist so there are even fewer potential opportunities for such bands compared to the arena filling acts.

[efdee]

I've rewritten this comment ten times already and I guess I don't really know what I _want_ to say, but this really irks me the wrong way. What you are describing is not Music (with a capital M) but rather the very similar product that is available in high abundance and nearly worthless. Real artists still have a considerable following that will absolutely anticipate and purchase physical media on launch day.

I'm sorry, that's the best I could do.

[Gordonjcp]

> as a child of the 80s/90s attempting to master the art of the perfect mixtape

I hit upon the plan of taping stuff off the radio onto 1/4", and then I could splice a not-talked-over beginning onto a not-talked-over ending.

Later, I worked out that I could extend or shorten tracks, particularly if I could get a tape of the instrumental version, using the same trick.

No-one uses tape and razors these days, but it was good fun.

[seanthemon]

I agree, I think another part of it is accessibility to tools and content. Same thing happens with gaming to an extent, when tools became more accessible, so did the content, but the quality dropped significantly, even among AAA games.

I think music suffers even more so because we're all so tuned into having the best at our fingertips that if a single moment in a song isn't to our liking we can skip and forget about it completely - i think this fuels the fast-short song market, easier to saturate with many short songs and get listens rather than to work/slave on a longer more intricate piece.

Back in the day, mixtapes with songs were slaved on and cherished, today slaving over something is seen as a negative quality.

[hedora]

In this world of algorithmic content distribution, we have more genres of music than any one person can enumerate.

I don't see any of the effects you describe on my feeds (tidal; previously apple music). Perhaps you need to switch to a service with a better recommendation algorithm, or nuke your personalization profile and start over.

[_glass]

Meanwhile the ghost of the bridge lives on as a sample in contemporary songs. Future generations will watch these ruins in awe of what the masters in the past were able to construct. Will we ever be able to surpass the classics?

[jollyllama]

The way this works out in practice is a DJ makes their mix by blending these songs together at the start/end via remixes with similar beats.

[lelanthran]

My word is this impressive.

I uploaded Wuthering Heights[1], selected just the trailing guitar solo (3:09 to just before the fade-out starts), and got a bunch of seemless sounding 3m tracks of just that guitar shredding away.

All the ones it generated were perfectly cut and pasted, with one exception in track 2 where I could tell where it was cut/pasted.

Can't wait to try it with the guitar at the end of Brothers in Arms (Dire Straits), or the flute solo in Locomotive Breath.

If anyone is looking for a good use of AI, I wouldn't mind a webpage that lets me say "complete this solo that was abruptly cut short" and get something great.

[1] This one is especially sad because the sound engineer/producer later lamented that he faded the solo out so quickly because the player was still improving like mad and what he had left after the fadeout was apparently better than what he had done while recording.

[InCityDreams]

Thanks! I took up guitar because recorded solos were never long enough.

[cheschire]

There's a specific sect of deadmau5 fans that would love to use this to generate multi-hour continuous mixes of a song called Strobe.

[jermeh]

I feel attacked

[mmh0000]

This is the coolest thing ever!

I like to have multiple hour continuations of songs which I use to help me fall asleep. In the past I've made my own, but I'm terrible at audio editing. It is a ton of work and, for me, really hard to get right.

After a little bit of experimenting with Mofi, it seems to do a very good job and selecting when to repeat a sections.

[edit]

Here's my first attempt for a 30 minute song:

https://mofi.loud.red/edit/ebbf4b410181aa767152945cbb6a2d679...

[jtbayly]

I wish this worked. I wanted to listen to an example. But all I got was “You haven't imported this file yet. Import it first to use it.”

[BonoboIO]

This gives me

"Not Imported You haven't imported this file yet. Import it first to use it."

[mmh0000]

After spending all afternoon playing with this. I have two inputs.

1] A lot of the heavy lifting is done client side, which is excellent; I have a super fast processor! But, watching the CPU utilization, it seems to spike the CPU for a few milliseconds, then waits for a second or two and, spikes the CPU again, then repeats. I suspect there may be some significant performance improvements to be had on client recomputes.

2] While it is doing recomputes, it would be great to get a progress bar or, at least, a message saying, "This will take a long time, be patient." It took me multiple attempts to determine that the service was indeed working, just that my client was taking its sweet time recomputing.

[spaceman_2020]

Can’t wait to use this to stretch my 19 minute prog rock tracks into 190 minutes :)

[diggum]

I was fortunate to help bring similar tech to life as a PM for Adobe Audition and Premiere Pro in a feature dubbed Remix. Since we were designing to help music fit the duration of a scene or video project, our goals resulted in a slightly different experience from those of music fans wanting to recut a song for general purpose listening, but it worked like magic for most tracks. Still one of my favorite projects that I worked on.

[diggum]

And I just realized you acknowledged it at the bottom of the page! ;)

[Solvency]

How did you make the animation in the homepage? It's definitely not a screen recording...it feels like something animated in After Effects at 60fps. Is that true? Or did you use something else?

[jasonjmcghee]

Looks a bit like https://www.screen.studio/

[zedadex]

As a fan of tools like this and Eternal Jukebox, I cannot wait for the ones that have an option to request a cut specifically, like "from approximately this point, jump back to around this other point", while using the 'smart' features to line up the audio cuts (I'm sure stuff like that is built into the CC suite but I'm not springing that kind of money to recut a few of my tunes for personal listening ).

With Eternal Jukebox I was able to emulate it by setting the jump percent very high manually at the right time (but had to be paying attention + had to get lucky with which branch it took), whereas here I'll see if I can get it going by calculating what the cut I want would set the song length to, and requesting that length and fewest cuts. (Edit: didn't work)

Sometimes the simplest features are the easiest to overlook but most useful

[MatthewWilkes]

Surprising that the only mention of intellectual property in the FAQ and the legal terms page is about copyright that the service claims in the outputs, not restrictions on the inputs.

[liminalsunset]

The ToS for this claims that you have to pay for it to use the results commercially, but I couldn't find the pricing for it anywhere or any way to sign up.

Interesting, as the primary use case for this seems to be something along the lines of editing music to fit into video projects.

[naltroc]

since there is no login mechanism, I used the anonymous feedback section to request their agreements with BMI and ASCAP for creating liscenced mechanical renditions of copyrighted materials.

[sergiotapia]

Thank you, now I can remove Dababy from Dua Lipa - Levitating. https://www.youtube.com/watch?v=TUVcZfQe-Kw&pp=ygUTZHVhIGxpc...

He just butchers an otherwise perfect song!

Will report back with results.

Edit: Results are great! https://voca.ro/13ar1g88LSKK

[brennopost]

Can't you just listen to the version he's not featured on?

[lcnPylGDnU4H9OF]

Seeing the domain has me curious, is it a reference to a Pokemon[0] or is it something else? (Sorry for off-topic!)

> On repeat: Make an extended version of your song's favorite part by choosing the catchy part and seamlessly repeating part of it!

John Mulaney's "Best Meal I Ever Had" story[1] had me wanting to do this for Tom Jones' What's New Pussycat so I could play that as an inside joke with some friends. This would definitely have made that easier. (Ultimately, I had a version of the song which started in the middle and "ended" right before it "started" so the effect could be mimicked if the file was played on loop. This had a short silence manifest as the media player "loaded a new song" from its perspective; didn't have that issue in Audacity.)

[0] https://www.pokemon.com/us/pokedex/loudred

[1] https://www.youtube.com/watch?v=Tv1l1eUhN-E (Usually called "The Salt and Pepper Diner")

[sd9]

Infinite roll: https://mofi.loud.red/edit/517b150b00ca475016cd64c2ed0730266...

[codetrotter]

> This is helpful for making any song match a video or performance with a set duration.

This is perfect for TikTok videos!

Will definitely be using your tool. Very cool!

> I am thinking of writing up some more details about the implementation in the future.

Please do :D

[personjerry]

I'm enjoying the results, I can make my favourite tunes 10 minutes long and often it's pretty seamless! I can't wait to get sick of all my songs :)

Is the project open-source?

[jaflo]

Glad you like it! Sadly this version is not, but this project is a rewrite from an old version I created that is open-source! https://github.com/jaflo/snipsnip

If you are interested in audio analysis in general, I would also recommend checking out librosa: https://librosa.org/doc/latest/index.html

[stu2010]

What's the maximum length? I can't find it documented anywhere and keep running into it when trying to try this with Tool tracks.

[jaflo]

There is no enforced maximum length for either input or output right now. In practice, the remote machine times out trying to analyze a song after 10 or so minutes. For output, your machine will take forever to generate really long results.

[jedberg]

The way this works is interesting. I asked it to cut 30 seconds from the middle of a song (a spoken word portion without music) and then shorten the track by 30 seconds. I figured it would just cut that out and do some fancy blending of the music before and after the cut to make it sound good. Instead I got:

    at 00:00 a 00:16 segment (00:00–00:16)
    at 00:16 a 00:51 segment (02:05–02:57)
    at 01:07 a 02:31 segment (01:34–04:05)
    at 03:39 a 00:11 segment (04:03–04:14)

It's adding and repeating chunks of the song and cutting other parts and I'm not sure why.

[quickthrower2]

I can finally see and hear what Bohemian Rapsody would have sounded like had they caved in to make a radio-friendly cut.

[redox99]

Does this create "new" music for the filled parts, like some AIs (MusicLM, MusicGen, etc) do? Or just repeat in a more seamless way?

[jaflo]

It does not create new music from scratch using one of the techniques you mentioned, but just reuses parts from the song, blending the transitions between them. That is a cool idea though!