The headline claim from Swiftlet is striking: fit an 80B-parameter Qwen model into 4.3 GB of RAM on a Mac, and squeeze a 35B model onto an iPhone. If those numbers hold up in your own testing, it reframes what "local LLM" can mean on consumer Apple hardware, where memory has traditionally been the hard ceiling.
The practical trick behind numbers like these is almost always aggressive quantization combined with smart memory management—loading weights in low-bit formats and streaming or mapping them rather than holding the full model in RAM at once. That's how a model whose full-precision weights would need tens of gigabytes can report a runtime footprint of a few gigabytes. The tradeoff to watch for is speed and output quality: memory savings often come at the cost of tokens-per-second and some precision loss.
Why this matters: on-device inference means no API bills, no network latency, and no data leaving the device—useful for privacy-sensitive apps, offline use, and prototyping without cloud dependencies. For iOS developers specifically, a 35B model running locally opens the door to app features that previously required a server round-trip.
What you can do with it: clone the repo, run the provided models, and benchmark the two things the headline doesn't tell you—actual generation speed and response quality under real prompts. Check whether the RAM figure reflects peak usage or a steady state, and test on the exact device you plan to ship on, since Apple silicon generations vary widely in memory bandwidth.
As with any Show HN project making bold efficiency claims, treat the numbers as a starting point rather than a spec sheet. The community discussion on Hacker News (180 points, 78 comments) is worth reading for independent results and caveats before you build anything production-facing on top of it.