Training Infrastructure & Compute.
Where to train, how much it costs, and how to avoid burning money.
After this lesson you'll know
- Cloud GPU options: pricing, performance, and availability for fine-tuning
- How to estimate compute costs before you start training
- Multi-GPU training with DeepSpeed and FSDP
- Local training options for budget-conscious practitioners
GPU Landscape for Fine-Tuning
Fine-tuning compute comes from three sources: cloud GPU rental, managed fine-tuning APIs, and local hardware. **Cloud GPU pricing (as of 2026):** | GPU | VRAM | LoRA 7B | QLoRA 70B | $/hour (spot) | $/hour (on-demand) | |-----|------|---------|-----------|---------------|-------------------| | RTX 4090 | 24GB | Yes | No | $0.30-0.50 | $0.70-1.00 | | A100 40GB | 40GB | Yes | Yes (tight) | $1.00-1.50 | $2.50-3.50 | | A100 80GB | 80GB | Yes | Yes | $1.50-2.00 | $3.50-5.00 | | H100 80GB | 80GB | Yes | Yes | $2.00-3.00 | $4.00-8.00 | | L40S | 48GB | Yes | Yes | $0.80-1.20 | $1.80-2.50 | **Cloud providers:** ``` RunPod — Best spot pricing, easy setup, good for experimentation Lambda Labs — Dedicated instances, reliable availability Vast.ai — Cheapest rates, community GPUs, variable reliability AWS (p4d/p5) — Enterprise, most expensive, most reliable GCP (a2/a3) — Enterprise, good availability Azure (NC) — Enterprise, MSFT ecosystem integration Modal — Serverless GPU, pay per second, great for scripts ``` **Managed fine-tuning APIs (no infrastructure to manage):** ``` OpenAI Fine-Tuning API — GPT-4o-mini from $0.30/1M tokens Together AI — Open models, $0.50-2.00/1M tokens Fireworks AI — Fast, open models, competitive pricing Anyscale — Ray-based, scalable ```
For your first fine-tuning run, use a managed API (OpenAI or Together AI). Zero infrastructure setup. Once you understand the process and have proven data quality, switch to cloud GPUs for cost savings on larger or repeated training runs.
This lesson is for Pro members
Unlock all 518+ lessons across 52 courses with Academy Pro.
Already a member? Sign in to access your lessons.