GPUs for Running Falcon-40B
| GPU | Inference speed relative to 2x H100s (est) | Speed / $ (relative est) | Cost at Runpod | Cost at FluidStack | Cost at Lambda Labs |
|---|---|---|---|---|---|
| 2x H100s | 100% | Not available | Not available | Not available in an on-demand 2x instance | Not available in an on-demand 2x instance |
| 2x 6000 Ada | 48% | 0.20 | ✅ $2.38 | Not available | Not available |
| 2x L40 | 43% | 0.18 | $2.38 | Not available | Not available |
| 2x A100 80GB | 43% | 0.12 | $3.58 | $4.99 | Not available |
| 2x A6000 | 19% | 0.12 | ✅ $1.58 | ✅ $1.60 | ✅ $1.60 |
| 2x A40 | 19% | 0.12 | $1.58 | $3.19 | Not available |