YouTube actually had something like this available for a while (or still does this?) through Caption Search.[1] If a video had captions available (either user provided or automated), one could search based on the captions on that video. Results would return the video and the time associated with that clip in question.
I am not exactly sure why they discontinued the user experience, since it seems like it could be really useful for finding relevant parts in potentially long videos.
Yeah, good point. We knew about this but the accuracy wasn't all that great. Our algorithm is fuzzy and can find results even if they are transcribed incorrectly (even today, every speech transcription technology out there still makes plenty of mistakes).
I would love to contact you regarding using this for sales phone calls. Often times, my employees don't remember an important portion of some conversation. This makes it really easy to find that!
Will this work with videos that aren't already subtitled? I haven't gotten one without subtitles to work yet maybe I'm just not waiting long enough though.
We do two things simultaneously. If there is no closed caption, then the video is submitted to our API to be processed (which takes longer on longer videos - a link shows up on the page to let you know the future URL). If there is a CC, then we parse that right away (which gives ok search results) but still submit to our API for processing, so later the search results will purely be from our indexing.
You are right that you need to input the particular video for now, but we are expanding to include multi-video search (constantly indexing in the background). Right now the use case is particularly nice for finding phrases in long videos (look for Donald Trump saying random stuff {wall, tremendous, many many, women}, it is pretty comical - e.g. http://www.lexika.io/?s=https%3A%2F%2Fwww.youtube.com%2Fwatc...).
A heads up on the homepage, the video you featured Steve Carrell and Jimmy Fallon with the keyword "pantaloons" is captioned as "the wrong kind pantaloons ah", when the spoken phrase was "the wrong kind of pantaloons on" - maybe this is me being paranoid and pedantic, but I feel like having a featured video with a pre-selected keyword that returns flawed captions gives a worse impression than just having the video flags without any captions at all. I understand you guys are still refining the product but highlighting one of your flaws (however small) so early on can rub people the wrong way.
On a separate note, when I google "lexika search engine" I get your old homepage lexika.co instead of the one linked here, lexika.io. Consider setting up a redirect from .co to .io?
Thanks for the transcription feedback. You can look at this as a problem but we think it's a cute demonstration of our search ability. We didn't really set out to make transcriptions perfect (that's very, very hard), we wanted to make search very accurate. Having a better transcription model would be great though (we are working on that!).
We'll definitely work on the site. (We should have done that earlier.) Thanks again!
Nice service! May I ask what machine learning technology you're using? I'm experimenting with neural nets these days, in order to learn the basics of image & speech recognition and would love to hear about your experiences if your'e usikg ANN's.
YouTube is a huge domain - the potential for audio indexing could bring vast improvements to search relevancy. What other audio domains are you experimenting with?
The first thing that comes to mind is podcasts. That are a slightly different direction, but a great fit for our service. They're long, have well spoken words, and are regularly released, so there is a mound of searchable audio data out there just waiting to be liberated.
Yeah for sure! This site is basically a demo/test for our backend.
We built an API that processes audio and forms an index for that file. The API search function then goes into that index to look for queries. It doesn't just look for words that match the text you see, but also the way it sounds.
You should be able to hit enter in the search field on your phone's soft keyboard (or the button might just say search). Which phone/operating system is this?
[+] [-] tsurantino|10 years ago|reply
I am not exactly sure why they discontinued the user experience, since it seems like it could be really useful for finding relevant parts in potentially long videos.
[1] http://youtube-global.blogspot.ca/2012/02/captions-for-all-m...
[+] [-] stephensonsco|10 years ago|reply
[+] [-] tonydiv|10 years ago|reply
tonydiepenbrock[@at]gmail.com
[+] [-] mkorfmann|10 years ago|reply
manu <at> korfmann <dot> info
[+] [-] pmorici|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] fiatmoney|10 years ago|reply
Scraping the content would be more useful, but I'm guessing that gets shut down quick.
[+] [-] stephensonsco|10 years ago|reply
[+] [-] jjzhang|10 years ago|reply
On a separate note, when I google "lexika search engine" I get your old homepage lexika.co instead of the one linked here, lexika.io. Consider setting up a redirect from .co to .io?
[+] [-] stephensonsco|10 years ago|reply
We'll definitely work on the site. (We should have done that earlier.) Thanks again!
[+] [-] mrborgen|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] chejazi|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] xasos|10 years ago|reply
[+] [-] dmcswain|10 years ago|reply
https://youtube.com/treycent/search?query=youtube
[+] [-] bedeho|10 years ago|reply
[+] [-] misiti3780|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
We built an API that processes audio and forms an index for that file. The API search function then goes into that index to look for queries. It doesn't just look for words that match the text you see, but also the way it sounds.
[+] [-] duaneb|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] avodonosov|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] Hydraulix989|10 years ago|reply
[+] [-] stephensonsco|10 years ago|reply
[+] [-] Kinnard|10 years ago|reply