08 / Projects
Self-hosted AI Infrastructure
GPU infrastructure, vLLM, local models and an API layer for real applications.
The project is shown at the level of problem, architecture and capabilities; some implementation details remain reserved.
01 · The problem
Some AI workloads need to run locally for reasons of cost, privacy or control, particularly when the model becomes a permanent part of the system.
02 · The idea
Own the inference path: GPU → LLM → vLLM → API → application.
03 · The system
Linux hosts, GPU runtime, vLLM serving Qwen and LLaMA-class models, FastAPI as the application boundary.
04 · The implementation
Used as the inference backbone for internal assistants, document systems and product experiments.
05 · The next step
Keep the serving path boring and reliable. Capacity follows the products that need it.
The architecture
- 01 GPU
- 02 LLM
- 03 vLLM
- 04 API
- 05 AI application
Technologies
- Linux
- vLLM
- Qwen
- LLaMA
- FastAPI
- Python
Categories
AI · Infrastructure