Show HN: Tabby – AI Coding Assistant Runs on Apple M1/M2 GPU(github.com)

Show HN: Tabby – AI Coding Assistant Runs on Apple M1/M2 GPU

https://github.com/TabbyML/tabby/releases/tag/v0.1.1

Hi HN, after five months of hard work since Tabby's previous Show HN (https://news.ycombinator.com/item?id=35470915), I'm thrilled to share you the release of Tabby v0.1.1.

Previously, Tabby ran exclusively on CUDA devices, posing a significant barrier for developers looking to effectively utilize LLMs in their day-to-day coding.

The Tabby team made a significant contribution by enhancing support for the StarCoder series models (1B/3B/7B) in llama.cpp. This enhancement allows these models to run on Metal, providing comparable performance to that of an NVIDIA GPU.

I eagerly look forward to receiving feedback from the community and witnessing the enhanced on-edge deployment experience!

References:

[1] Apple Installation Instructions: https://tabby.tabbyml.com/docs/installation/apple/

[2] Launch blog: https://tabby.tabbyml.com/blog/2023/09/18/release-0-1-1-meta...

2 comments

gyxlucy3 yıl önce

This is quite cool. Have you compared performance vs latency tradeoff between 1B/3B/7B models? What would you recommend as the ideal choice?

myin3 yıl önce

Cool to be able to run locally