amboo7 · HackerTrans

11 dagen geledendiscuss

I have my own. One Telegram bot for reading, summarizing etc. (using a local Qwen 3.6), another for improving it (OpenCode Telegram, free DeepSeek V4). I might put it on github soon.

amboo7

11 dagen geledendiscuss

I have always liked coding (started in the 80s). But I got bored having to deal with trivialities (spending life on fixing commas) and increasing accidental complexity (having to memorize tons of APIs). Now LLMs deal with that crap and I can apply stuff I read in research papers and such, with much lower threshold.

amboo7

2 maanden geledendiscuss

Trying to recall...

# 27B GGUF # Benchmark results: # - TG speed: ~5 tok/s (vs baseline ~4.5 tok/s, +10% improvement) # - PP speed: ~60 tok/s (stable across context sizes) # - With parallel=4: total throughput ~28 tok/s

llama-server \ -m /Users/*/models/hf/models--unsloth--Qwen3.6-27B-GGUF/snapshots/82d411acf4a06cfb8d9b073a5211bf410bfc29bf/Qwen3.6-27B-Q6_K.gguf \ --alias "qwen3.6-27b" \ -ngl -1 \ --n-cpu-moe 0 \ -fa off \ -ctk q4_0 \ -ctv f16 \ -c 131071 \ -b 512 \ -ub 256 \ --spec-type ngram-cache \ --jinja \ --cache-ram -1 \ --parallel 4 \ --kv-unified \ --no-context-shift \ --mlock \ --slot-save-path ~/qwen_slots \ --reasoning-budget 512 \...

# 27B MTP GGUF # Benchmark results (from bench_conversation.sh, ~500tok prompts + multi-turn): # Config | PP (tok/s) | TG (tok/s) | Draft accept # ---------------------------------------|------------|------------|------------- # Baseline (ngram-cache, non-MTP) | 48.0 | 5.2 | N/A # MTP --no-mmap, dn=4, fa on, ctk q4_0 | 45.7 | 6.9 | 60% # MTP --no-mmap, dn=3, fa on, ctk q4_0 | 46.3 | 7.5 | 73% ← winner

MODEL="/Users/bale/models/hf/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/ac393bc3d23fd5a929a85e2f33c7c4fd5be02d43/Qwen3.6-27B-Q6_K.gguf"

llama-server \ -m "$MODEL" \ --alias "qwen3.6-27b-mtp" \ -ngl -1 \ --n-cpu-moe 0 \ --no-mmap \ -fa on \ -ctk q4_0 \ -ctv f16 \ -c 131071 \ -b 512 \ -ub 256 \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ --jinja \ --cache-ram -1 \ --parallel 1 \ --kv-unified \ --no-context-shift \ --slot-save-path ~/qwen_slots \ --reasoning-budget 512 \...

# 35B GGUF # Benchmark results: # - TG speed: ~27 tok/s (vs baseline ~21 tok/s, +30% improvement) # - PP speed: ~370-350 tok/s (slight decrease with larger context) # - Parallel=4 gives best throughput at ~152 tok/s total

llama-server \ -m /Users/*/models/hf/models--unsloth--Qwen3.6-35B-A3B-GGUF/snapshots/9280dd353ab587157920d5bd391ada414d84e552/Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf \ --alias "qwen3.6-35b" \ -ngl -1 \ --n-cpu-moe 0 \ -fa on \ -ctk f16 \ -ctv f16 \ -c 262144 \ -b 2048 \ -ub 512 \ --spec-type ngram-cache \ --jinja \ --cache-ram -1 \ --parallel 4 \ --kv-unified \ --no-context-shift \ --mlock \ --threads 4 \ --threads-batch 8 \ --slot-save-path ~/qwen_slots \ --reasoning-budget 512 \...

# 35B MTP GGUF # Benchmark results (128K context, verified 2026-05-18): # TG: 30.7 tok/s, Draft accept: 65%, no OOM at 128K

MODEL="/Users/**/models/hf/models--unsloth--Qwen3.6-35B-A3B-MTP-GGUF/snapshots/e28512781649329c5b37cbf55029355a48d158d4/Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf"

export GGML_METAL_BF16_DISABLE=1

llama-server \ -m "$MODEL" \ --alias "qwen3.6-35b-mtp" \ -ngl -1 \ --n-cpu-moe 0 \ --no-mmap \ -fa on \ -ctk f16 \ -ctv f16 \ -c 262144 \ -b 512 \ -ub 256 \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ --jinja \ --cache-ram -1 \ --parallel 1 \ --no-context-shift \ --threads 4 \ --threads-batch 8 \ --slot-save-path ~/qwen_slots \ --reasoning-budget 512 \...

amboo7

2 maanden geledendiscuss

Very cool. Anything publicly available?

amboo7

2 maanden geledendiscuss

Same happened to me. I only had to ask the service where to tap exactly.

amboo7

2 maanden geledendiscuss

For me the combination of agentic search and deep wiki creation for legacy code bases is a killer app/game changer. Not for the surviving authors of the legacy code bases.

amboo7

2 maanden geledendiscuss

Whay about of the tons of caches that just pile up until you notice that you must delete them manually?

amboo7

2 maanden geledendiscuss

I also have an M1 Max 64GB: Qwen 3.6 benefits from MTP (after rounds of parameter optimization). MLX was unstable (haven't tried it recently), faster at TG but slower at PP, so inconclusive.

amboo7

3 maanden geledendiscuss

https://github.com/microsoft/skills/tree/main/.github/plugin... is similar, works for Claude Code, too. I ported it to Pi: https://pi.dev/packages/@amb007/deep-wiki?name=deep-wiki Elsewhere, I added deep-wiki:lookup that's like deep-wiki:ask but giving precedence to the wiki instead of the code.

amboo7

vorig jaardiscuss

Essentials of Programming Languages 1st ed. is special

amboo7

2 jaar geledendiscuss

About deployment to cloud: https://news.ycombinator.com/item?id=38988238

amboo7

2 jaar geledendiscuss

https://icfp24.sigplan.org/home/minikanren-2024#event-overvi... is related. There has been work on turning slow bidirectional code into faster functional code (in 2023, reachable from this url).

amboo7

2 jaar geledendiscuss

Something like (in software, trying to understand) a spaghetti monolith?

amboo7

2 jaar geledendiscuss

Code rusts via the loss of information about it (memory, documentation...). If it cannot be maintained, it is a zombie.

amboo7

2 jaar geledendiscuss

OK, I realized it was about something else. I don't follow Uncle Bob, and was referring to some people justifying "unclean"/incorrect shortcuts to gain performance.

amboo7

2 jaar geledendiscuss

Nicely put. I also spent decades in the SW industry, tried about a dozen things, but chose to stay out of management. For me, business and engineering are two completely different areas, often conflicting, and my training is extensively in math and engineering. Trying to do both ends up with mediocrity almost certainly.

amboo7

2 jaar geledendiscuss

Wrong. Performance and correctness/cleanness go together. Perhaps you have never tried generating correct code that is ways faster than what you can write by hand.

amboo7

2 jaar geledendiscuss

https://www.cs.tufts.edu/~nr/cs257/archive/norman-ramsey/mat... by the current Microsoft CTO

amboo7

3 jaar geledendiscuss

https://www.mg.edu.rs/en#gsc.tab=0

amboo7

3 jaar geledendiscuss

Plain entropy. I use an extension for killing duplicate tabs, helps a lot.

comments