Maybe a bit outdated now, but reminds me of LSTMs from the recurrent update of a memory / hidden state with gating. I remember one of the biggest problems with such RNNs being vanishing gradients as a result of the long context, which vanilla transformers presumably avoided by parallellizing over the context instead of processing them individually. I wonder how this is avoided here?
Love the site, but the one thing I am missing is instantly seeing what papers are trending right now, instead of having to manually select a timeframe. Think hacker news frontpage. Would be interesting to add a "hot" filter, or similar. Average weighting by exponential decay over time? Not sure how you usually do that with pagerank.
Can relate to this problem a lot. I have considered starting using a Docker dev container and making a base image for shared dependencies which I then can customize in a dockerfile for each new project, not sure if there's a better alternative though.
I saw that Spotify recently released a similar feature for selected podcasts, could anyone recommend some recent research papers related to voice translation? Would be really interested in reading!
This study was done at my uni, and the legitimacy and rigor of it was questioned and investigated by the university. However, it was ultimately not deemed to be inappropriate. Some interesting info is that the single picture that was shown to the "jury" of a student was handpicked by the author from Facebook, and that the jury consisted of high school students.
I'm doing my thesis right now in diffusion models (for audio) and have experienced a lot of the same things you mention. One paper which I found illuminating was this one: https://proceedings.mlr.press/v139/nichol21a.html
Particularly relevant to the noisy training you mentioned earlier is their alternative timestep sampling procedure they propose which seem to reduce gradient noise significantly judging from their experiments. Would love to hear or discuss if you have found any other design changes which have improved training / sample qualities :)
http://nobelprizes.com/nobel/why_no_math.html#story