Rendered at 13:30:47 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Gerard22Aug 2 days ago [-]
Yeah, there is latest and greatest trend: 0-bit quant. Absolutely outstanding performance, can produce as many tokens per second as you terminal may sustain with zero load on unified memory / GPU. Real winner!
kamranjon 2 days ago [-]
Hmm, makes the 2 bit quants actually seem pretty reasonable…