GPU Clouds for Falcon-40B
To run Falcon-40B, 85GB+ of GPU ram is preferred. 8-bit mode should be runnable on an 80GB card, but we’re going to focus on setups with 85GB+ ram for better performance, and assuming using 16-bit mode.
| Provider | System | Overall | Access | Total price/hr |
|---|---|---|---|---|
| Vastai | 2x A6000 | Sometimes good, sometimes bad | Instant | $1.35 |
| FluidStack | 2x A6000 | Best | Instant | $1.60 |
| Lambda | 2x A6000 | Best | Instant | $1.60 |
| Runpod | 2x A40 or 2x A6000 | Best | Instant | $1.62 |
| Crusoe | 2x A6000 | Not recommended | Waitlist | $1.84 |
| Oracle | 4x A10 | Not recommended | My account failed verification | $2.44 |
| Oblivus | 2x A40 or 2xA6000 | Good but higher pricing | Instant | $3.03 |
| CoreWeave | 2x A40 or 2x A6000 | Not recommended | Manual approval | $3.50 |
| AWS | 4x A10 | Not recommended | Instant | $5.70 |
| Azure | 2x A100 80GB | Not recommended | Instant | $7.35 |
| Google Cloud | 2x A100 80GB | Not recommended | Instant | $10.09 |
View the full table with additional details on Google Sheets here and add your comments.