Towards Data Science AI

Wat kost het echt om een lokaal AI-model te draaien? Gemeten op RTX 3090

Back to overview
AISummary generated by AI from the original source

Running local language models on consumer hardware like an RTX 3090 reveals surprising cost patterns, with electricity expenses per million tokens varying unpredictably across different model sizes. A recent analysis of eight models showed that smaller systems don't necessarily cost less to operate, while larger ones don't always demand the highest power consumption.