AI gateway
Also called LLM gateway, LLM proxy, orchestration middleware
Software that sits between an organisation's apps and the AI models, holding the provider keys and routing, logging and limiting every request.
When one app calls an AI model, it can hold the provider's key itself. When 20 apps do, the keys end up scattered and nobody can say what the organisation spends or sends. An AI gateway fixes that by putting one piece of software in the middle. Apps talk to the gateway, and the gateway talks to the model providers.
That makes it useful and worth attacking, for the same reason. The gateway holds the provider keys, and every prompt passes through it. Someone who takes it over can read the prompts and use the keys.
So treat a gateway as a server that holds credentials. Keep its admin page off the open internet, put it behind sign-in, and patch it early. Only a handful of the flaws found in this kind of software are known to have been used in attacks, and those few handed over servers and keys.
For example
12 internal apps call AI models. None holds a provider key: each sends its requests to the gateway, which adds the key, records the cost and passes the request on.
Put it to use
Where you'll meet AI gateway in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.