Ask HN: Climate Concerns Using ChatGPT
Am I the only one who thinks twice before pressing "Send" on ChatGPT because of how much resources it needs to give me the answer?
2 comments
Can someone check my math on this? Supposedly ChatGPT uses 8x A100's per request. If we say each request takes 5 seconds, and a A100 uses 300watts.
(300w * 8) * 5 = 12,000 watt/sec or 12,000 joules
12,000/3600s = 3.3 watt/hour ?
This is just over the amount of power in a lithium AA battery according to wolframalpha.
(300w * 8) * 5 = 12,000 watt/sec or 12,000 joules
12,000/3600s = 3.3 watt/hour ?
This is just over the amount of power in a lithium AA battery according to wolframalpha.
To me that's still a lot. A whole AA battery for a single response. Maybe I'm just built this way.
They do add up a bit. I'm concerned about a future where we do this for _everything_ because that'll truly be a massive scale unless we get smaller models. If we go off 10M queries a day that's
0.0033KW/H / 24 = 0.0001375/D
0.0001375KW/D * 10M = 1375KW/D or 1.375MW/D
(if I did that right)
0.0033KW/H / 24 = 0.0001375/D
0.0001375KW/D * 10M = 1375KW/D or 1.375MW/D
(if I did that right)
This is a crazy amount of power for a single request.
Unfortunately, yes.