The creator of that repo has a blog with some posts on building a wasm version of FFmpeg, I mostly followed those: https://jeromewus.medium.com/
I also used some of their scripts for building the 3rd-party libraries.
It took a bit of trial and error to see which versions of the different libraries and build tools work together.
You don't need to keep the video files in memory, it's enough to have the javascript File object and then you can read the packets you need on the fly. This way the app's memory usage should be okay.
This is what I did in my similar project (vidmix.app) and seems to be what OP's project is doing, except that they also copy the file to indexedDB to keep it around after the browser tab is closed.
It seems to me that the demonstration is calculated in sRGB space, so with non-linear brightness values and I suspect that most of the unnaturalness of the smear is due to that. To simulate the physics this would need to be done with linear brightness values and only at the end converted to sRGB.
(unless some non-linear effects in human visual perception cancel all that out, but it should at least be mentioned if so)
I've been having the same issue with an app. Random rejections with a single line copy-pasted from the (genuine) feature list in the app description. In this case the lines just start with a regular dash ("-").
This together with the sometimes 10+ day review times makes it impossible to maintain an app on the Play Store now.
I use a WebAssembly build of FFmpeg to extract the streams and packets from the video files and as a fallback for some codecs. But most of the decoding/encoding is done with WebCodecs which in most cases has access to hardware acceleration so can be much faster than ffmpeg.
I've built the UI library myself, it's written in C++ and is drawing with OpenGL. (translated to WebGL by emscripten) Javascript is only used for the browser-related parts.
The point of this is that the javascript layer can be easily replaced and the project built for other platforms. (for example it runs on iOS natively: https://apple.co/3QVAYAq)
For people who don't want to sign up but still like the idea of a video editor running in a browser (that doesn't upload anything), I am working on an alternative:
Yes, the shortcomings of the html video tag are quite unfortunate. Just some very small improvements would make it a lot more useful. One other issue I found at the time I was looking at it (not sure if it's still the case) was that stepping one frame forward wasn't any faster than seeking from a random location. Seemed like the video was decoded from the last keyframe every time, making it inefficient to iterate through the frames.
In Chrome and Edge you can use WebCodecs to decode and display video frames one-by-one. (this is what I used for the video editor at https://vidmix.app )
In other browsers you could build FFmpeg with webassembly and use that for frame-by-frame decoding but it's not going to be nearly as performant.
Couple years ago there was a similar site with the same idea about listing all private keys. If you "found" an active address it was immediately emptied. (I am not sure if it had a search bar or you had to edit the URL)
The biggest problem is not supporting h264 encoding for the output.
Also as the preview playback is done with html video elements and those can't be started at a precise enough time, the timing of these videos are imprecise and may not align with other elements. This only happens at playback or scrubbing, not when staying on a single frame and the exported result should not be affected.
At the moment I don't have plans to make it open source.
I suspect that the drag and drop limit of 5 may be a browser/OS limitation, now I couldn't reproduce this but will look out for it. And yes, probably it would make more sense to put them on the same track after each other so I will change that. Thanks for pointing this out!
mp3 files not loading was an embarrassing bug, I just fixed it, should work now. (only remaining issue I just realised is that the volume panel doesn't show up when the track is first selected, I'll look into that later)
Hello HN! WebCodecs landed in chromium recently opening new possibilities for browser-based video editing: instead of relying on cloud-based processing, everything can now be done locally. As I never had a good answer when app developer friends asked me how to quickly edit their screen recordings for appstore listings, I decided to build something with this: a no download, no sign-up, no BS video editor.
pros:
- Everything is processed locally, nothing is sent to the cloud. This means privacy, but also speed: if you want to extract a 20 seconds part of a 800 MB video file, you don't have to wait for the upload.
- No limits on frame size or video length. It isn't using any cloud resources so no need to put artificial limits on that.
- Precise timing. Seems like a small thing, but surprisingly difficult to get it right with html <video> tags.
- Preview matches exactly the final result: the exact same code generates the preview and the exported video, not some ugly hack attempting to replicate what you see in the browser to the cloud renderer.
cons:
- It only works properly in Chrome and Edge, with some acceptable fallbacks in Firefox.
- HEVC is not supported by any browser, so for that FFmpeg is needed but that's only fast enough on recent computers
- Everything needs to be loaded into memory. Compared to the cloud-based editors this isn't much of an issue as those typically limit you to 100-200 MB files anyway, but eventually I'd like to use the Filesystem API to make it possible to edit videos of any size.
Also there are plenty of more advanced features: unlimited number of layers, motion tracking, blending modes, using LUT files, a not-so-basic painting tool, etc.
Let me know if you have any questions or feedback!