Mergen v1.0: Running 9B and 27B LLMs Across Reclaimed Phones Over USB
When I first published Mergen, the goal was modest: prove that a C++20 / GGML runtime could split a GGUF language model into consecutive transformer layer shards, ship those shards to reclaimed phones...
Read post