Shameless plug, for anyone who's interested in "self-improvement" agent check out StreamBench[1] where we benchmark and try out what's essential for improvements in online settings. Basically we find feedback signal is vital and the stronger the signal the more improvement you can get if you were able to feed it back to the agent in terms of weights (LoRA) or in-context examples.
Do beware on some reasoning task, our recent work[0] actually found it may cause some performance degradation as well as possible reasoning weakening in JSON. I really hope they fix this in the latest GPT-4o version.
I have been following the vector database trend back in 2020 and I ended up with the conclusion: vector search features are a nice to have features which adds more value on existing database (postgres) or text search services (elasticsearch) than using an entirely new framework full of hidden bugs. You could get way higher speedup when you are using the right embedding models and encoding way than just using the vector database with the best underlying optimization. And the bonus side is that you are using a stack which was battle tested (postgres, elasticsearch) vs new kids (pinecone, milvus ... )
This looks pretty interesting! But the landing page has only one sentence : Understand and implement research papers faster, followed by a get in touch button. Care to extrapolate more? The blog button doesn't work as well?
Note that stability have been funding freelance researcher by providing compute resource such as RWKV[1], Open Assistant, some works by LAION[2] and lucidrains[3]
On the other hand, meta AI research should be renamed as OpenAI. They are the only few big institute who open almost every models they train (galactica, OPT, M2M, wav2vec ...)
Recently just migrate a project from pypoetry away to the traditional setup method. Poetry works great for simple package, but once you started to add in complexities, it just falls apart due to everything was abstract away and simplified into config files and command line.
When chatGPT first came out, my first thought was to replicate it myself. But then, I have too many missing skills or lack the time for backend, frontend and deployment. So I found LAION started an initiative for open-assistant.
https://github.com/LAION-AI/Open-Assistant
So I just join them, so far its quite an active community working on pushing out the initial 0.1 version.
You can use sin as activation function, but that would require careful initialization to avoid gradient explosion as you would ended up with a lot of points where gradient is simply zero. You can refer to Implicit Neural Representations with Periodic Activation Functions for more details.
What I ended up doing was crawling news from major news services and do aggregation myself. My algorithm was simple : cluster news that was reported by multple different services or multiple reports within 36 hours. By showing multiple news titles in a same box and order them by cluster size, I could just skipped those which isn't important at all.
I agree having a good vector is important to start with. However this is not very hard to make it work, you only need to finetune some of the clip models[1] to run it well.
Disclose: I have built a vector search engine to proof this idea[2]
Do anyone know any existing effort on converting these scanned image to text corpus ( probably a new OCR model needed to be developed on these old text ) ? I think it would be more usable if they are in text form in terms of search and research purpose.
I created edgedict [0] a year ago part of my side projects. At that time this is the only open source STT with streaming capabilities. If anyone is interested the pretrained weights for english and chinese is available.
I am surprised to see Comma Two in a consumer report not to mention out perform all other automative company including Tesla (probably the older version?). I remember in one of Geohot twitch stream bragging about their driver monitoring being really hard to duped.
Same in chinese search results for programming related question, most of the time I get was some bad translation from stackoverflow or github issue. Now I had to rely on browser plugins to automatically remove these domains from search results.
For reference, Boeing 747 average speed is 909 km/h (565 mph) with a top speed 986 km/h (613 mph). However maglev train saves time on landing and coordination makes it faster compare to air travel.
If anyone is interested in building 10Gb router on a budget you can buy Mellanox OCP NIC with an adapter from OCP to PCIe for around 5~30 USD ( depend on your location )[0]. I recently build one 10G router with 4x10Gb, 2x1Gb ports for around 200 USD using second hand PC.
In terms of setup and ease of use jaided OCR[1] is the best one I have used out there. I can't stress how many times I tried other alternative (tesseract, paddle OCR) out there and kept going back to use their OCR library. Mainly because it supports a large variety of languages out of the box while provide nice results.
Disclaimer : I am not affiliated to jaidedAI, just a satisfied user.
[1] https://arxiv.org/abs/2406.08747