RE: LeoThread 2025-08-07 04:14
You are viewing a single comment's thread:
I don't know how much RAM it needs to run, but 20 stands for 20 Billion parameters, (not 20GB,) and quantized versions of models usually cost significantly less in RAM.
0
0
0.000
That's correct, the model needs 16Gb of RAM to run unquantised. I imagine that Apple will quantise it and market it as their foundation model at the WWDC 26 next year. More here.
If they quantize it heavily it'll only cost 2 RAM but the quality will be bad, so... Who knows?
That's my point: Apple can't afford to release a substandard model anymore. Google is much more advanced than Apple in this area!
The next question is how big the context windows should be for us to consider the model smart enough. Because, nowadays, Siri seems very incompetent.
They'll figure it out by then.~