RE: LeoThread 2026-01-01 19-56
You are viewing a single comment's thread:
One of the best things to see is how fast the existing small models are improving. In order to make LLMs accessible, we need theme to run on low resource requirements.
0
0
0.000
Yeah, they are still pretty dumb unless you can get to a 14B level, that's when they become usable, at least for me, lol. Anything less is like talking to the dumb blond bimbo secretary, lol, she only got the job because she was hot kind of thing, lol. But yes, I agree. The Apple Silicon is really the way consumers can get to a near 'super computer' level for this stuff. That unified memory is the way. I am about to buy a Macbook Pro M4 Max with 128GB of unified memory and 1TB ssd for like just under $5K and will be able to run llama 3.3 70B 4-bit and Qwen3-coder 30B full 16Bit at the same time. I'll even be able to expand and get a Mac Studio and cluster them in the future, lol.
There is a version of Linux available for Macbooks. It is a work in progress: https://asahilinux.org
I could use Bootcamp on the mac now and install Ubuntu if I wanted and run it on a different partition, but I'll run with MacOS as my daily driver because of how it works with the hardware. I am a former Apple senior support engineer, so I am pretty heavy in that ecosystem and can do things most people don't know how to, lol.