Getting Started with Orca-2-13B(secondstate.io)

68 pointsby 3Sophonsvor 3 Jahren17 comments

https://www.secondstate.io/articles/orca-2-13b/

4 comments

riquitovor 3 Jahren

I see that they recently changed the intro of WasmEdge on Github [1] to " WasmEdge is the easiest and fastest way to run LLMs on your own devices. "

Since it's a wasm runtime capable of many things I find bizarre that they now start describing it with a ultra-specific use case

- [1] https://github.com/WasmEdge/WasmEdge

[deleted]vor 3 Jahren

anuraagavor 3 Jahren

I'm assuming they're both VC companies in the same portfolio and are cross-marketing. If not, the the descriptions on both sides seem quite excessive indeed...

crimsoneervor 3 Jahren

for those who want a less command line focused approach, I'd really recommend https://lmstudio.ai/

mannyvvor 3 Jahren

Amazing enough, there are a couple of orca models in LM Studio/huggingface.

Not sure what the differences are between them.

Are these like AMIs, where you can stuff all kinds of odd shit into them?

mark_l_watsonvor 3 Jahren

I am not at my computer and won’t be for several days, so I can’t check this myself: is Orca-2-13B available in their model library? How about Ollama?

pmarreckvor 3 Jahren

I for one cannot stand projects that depend on Python due to the multitenant and dependency management problems, so this is welcome

bambaxvor 3 Jahren

Yeah. If I understand this correctly[0], this would work with any model in GGUF format.

[0] https://www.secondstate.io/articles/convert-pytorch-to-gguf/

ekianjovor 3 Jahren

Which means any model derived from llama2

3Sophonsvor 3 Jahren

Indeed. There are the others like LlamaCode, Yi and so many more

tanvirulhudavor 3 Jahren

(4)