WingNews logo WingNews
top | item 30969501

(no title)

mzur | 3 years ago

Is it open source or are there plans to make it so? I'd be very interested how you do frame-accurate video annotations in the browser, as I found the currentTime attribute of HTML video elements to be too unreliable for this. (Context: I maintain a video annotation tool that is mostly used for marine imaging.)

discuss

order

zackoverflow|3 years ago

For Anno we are using HTML video elements / YouTube embeds which have the same frame inaccuracy from currentTime. For our editor we have a WebGL-based solution that seems to get better frame granularity but still has some floating point rounding errors converting frames <-> seconds

There's also requestVideoFrameCallback() and seekToNextFrame() in the Web APIs but they are still experimental/not supported by all browser. The WebCodecs API is also experimental/not supported to all browsers but would allow you to decode and grab the individual frames from a video and draw them to a canvas

bjano|3 years ago

In Chrome and Edge you can use WebCodecs to decode and display video frames one-by-one. (this is what I used for the video editor at https://vidmix.app )

In other browsers you could build FFmpeg with webassembly and use that for frame-by-frame decoding but it's not going to be nearly as performant.

mzur|3 years ago

Decoding the video manually seems so much overkill when all that's needed would essentially be a currentFrame property. But it seems I have to explore this avenue as well. I only aim to support Chrome and Firefox but as much as I hate to admit it, Firefox is somewhat lacking on the video side anyway (as there is a bug that lets it display different video frames for the same currentTime than other browsers or FFmpeg [1]). Therefore I already discourage using Firefox for video annotation.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1735300

SpectralName|3 years ago

> Context: I maintain a video annotation tool that is mostly used for marine imaging.

Which one, out of curiosity? I'm just starting to look into this space and interested in knowing what's out there.