Open-Weight AI Models: A Practical Guide
Open-weight models let you run capable AI on your own hardware. Here is what "open weight" means, the trade-offs, and how to get started.
Open-weight models publish their trained parameters — the "weights" — for anyone to download and run. They are not the same as open-source software: the weights are available, but the training data and code usually are not.
Why they matter
Open weights give you control. You can run them on your own hardware, fine-tune them on your own data, and avoid sending sensitive information to a third-party API. They also create competition, pushing down prices across the industry.
The trade-offs
Running a capable model locally requires hardware. A 7-billion-parameter model runs comfortably on a modern laptop; a 70-billion-parameter model typically needs a GPU with 48GB of memory or more. Quantisation — compressing the weights — makes larger models practical on smaller hardware at a small cost in quality.
Getting started
Pick a well-regarded open-weight model that fits your hardware, download a quantised version, and run it with a local inference tool. Start with a small model to learn the workflow, then scale up as needed.
The bottom line
Open weights are the closest thing AI has to a public utility: inspectable, controllable and increasingly capable.