regen7253 · HackerTrans

2 jaar geledendiscuss

it would be interesting to analyze hacker news's threads in this same fashion. not only comments but also upvotes. in the past few months I have noticed that my experience of reading through them has changed significantly, with a lot less comments that are actually informative or add real new takes. plus, a lot more skewed towards more mainstream discourse and less hacker-ish...

regen7253

2 jaar geledendiscuss

And duplicated, two submissions got similar ~20ish upvotes about the same subject...

regen7253

2 jaar geledendiscuss

"Owning ideas" is a relatively new concept -- intellectual property is a few centuries old.

"Ownership" is completely opposite to a natural state of things, and can only be enforced with force. Mix that abstract concept with the abstract of information for extra complexity. Add to it that Information can't be "owned" in the same sense that physical property can, and the fact that (if not copyrighed), it's a non-excludable, non-rivalrous good (it can benefit everyone, equally, at the same time).

On the other hand, it's nice to incentivize innovation, competition and creativity, or at least, not hinder it.

It would be very interesting to go to 2124 and check if these concepts are still alive, or if somehow we have evolved our social norms to be closer to the natural world. Once you convey information to any entity outside of your brain/body, it naturally belongs to the world.

regen7253

2 jaar geledendiscuss

Instead of "almost", maybe "Open Source Code (and free for non-commercial use due to model licencing)" since your contributions are free as in freedom,and that is amazing!

Thank you for building this, I have been using a web interface connected to a local server for inference but the latency was about 1 second, too much for my taste!

regen7253

2 jaar geledendiscuss

Thank you!

I've been using llama.cpp for about a year now, mostly implementing some RAG and React related papers to stay up to date. I mostly used llama.cpp, but since a few months, I started to use both Ollama and Llama.cpp.

If you added grammars I wouldn't have to be running the two servers, I think you're doing an excellent job out of maintaining Ollama. Every update is like Christmas. They also don't seem to have the server as a priority (it's still literally just an example of how you'd use their C api).

So, I understand your position, since their server API has been quite unstable, and the grammar validation didn't work at all until February. I also still can't get their multiple model loading to work reliably right now.

Having said that, GBNF is a godsend for my daily use cases. I even prefer using phi3b with a grammar than deal with the hallucinations of a 70b without it. Fine tuning helps a lot, but can't solve the problem fully (you still need to validate the generation), and it's a lot less agile when implementing ideas. Crating some synthetic data sets is easier if you have support for grammars.

I think many like me are in the same spot. Thank you for being considerate about the stability and support that it would require. But please, take a look at the current state of their grammar validation, which is pretty good right now.