also its pretty big model inference costs are high even with margins running a 2.8T model costs a lot. if they release oss may be it goes down to $10-12 per million tokens.
I built https://meow.gs as a side project. It gives you a cloud shell via SSH, authenticated with Face ID / Touch ID instead of SSH keys.
ssh meow.gs from any terminal iPad, iPhone, laptop. Easy to work while remote and have ai coding tools on private vps and access from anywhere.
Though claude/codex now has remote access, this was created 1 month back but still helpful if you need own private tools on vps and accessibility without managing ssh keys.
codex/gpt is a stubborn model, doubt it would accept claude reviews or counter it. have seen cases where claude is more willing to comply if shared feedback though its just sycophancy too.
By looking at the paper, memory needed per layer seems to be higher than transformer architecture. Pretty sure that would be blowing up the vram of gpu at scale.
Its so useful to use Cerebras api for other tasks too not just coding with qwen coder but even simpler things like lets say analysing with gpt-120 oss or llama.
Just plug it in with normal chat interface like Jan or Cherry studio and its incredibly fast.
Ideally when we are writing agents we need mcp to support auth, custom headers because by design when deploying for saas we need to pass around client params to be able to isolate client connections.
We do token optimisation and other smart stuff to save token money. Looking forward to try this as well if this solves similar problems as well
true, I was using Cline/Roocode from almost an year and it always made sure to read things from memory-bank which i really liked. Claude has gone downhill from August mid for me and often it doesn't follow instructions from claude.md or forget things mid-way.
Mostly agree with article, though what happens in few years when juniors will eventually become senior.
Personally seeing trend juniors are relying so much on AI that they can't even explain what they wrote even in interview or coding assignments or even PR. Its like blackbox to them.
I believe then we would see the higher impact or may be by then its solved problem already.
Also from my experience you need more power to get some significant result. Mostly fine tuning would work if base model is very close to what you are trying to achieve and you won't be much happy with the results though.
Also context length becomes an issue trying to fit in with gpu with lesser ram.
We do quite similar but through Debezium/Kafka CDC pipeline to clickhouse. This way primary database is protected. Directly querying from files from postgres might get slower eventually IMO compare.
we are actually trying something similar but possible kinesis + clickhouse or kafka + clickhouse. Currently kinesis seems easier to deal with but not a good intergration or sink connector available to process records at scale for kinesis to put into clickhouse. Were you ever felt into similar problems where you had to process records at huge scale to be able to insert into clickhouse without much delay.
One more thing is kinesis can have duplicates while kafka is exactly once delivery.