The Adapter Is Not the Model

Ten fine-tuned models means ten copies of the base model. Each copy occupies GPU memory. Each copy idles when the others are busy. That is the naive architecture — and it is what most self-hosted deployments do.

It is the wrong architecture.

What LoRA Actually Is

LoRA does not modify a model. It adds two small matrices — a down-projection A and an up-projection B — at each weight layer it targets. The forward pass computes the base weight result, then adds the low-rank correction: W_final = W_base + BA. During inference, the base computation runs unchanged. The adapter is a delta.

That structure is a deployment hint, not just a training trick. The base model does not need to change between users. Only the deltas do.

The Merge-and-Serve Trap

The default instinct is to bake the adapter into the weights before serving. Merge W_base + BA once, then serve the merged model. This works for one adapter. It breaks for ten.

Merged serving requires one model instance per adapter. GPU memory scales linearly with the number of adapters. A 70B model running ten fine-tuned variants needs ten times the VRAM. The weights that are identical across all ten variants — the entire base model — are duplicated ten times.

The Right Architecture: One Base, Many Deltas

S-LoRA (Sheng et al., 2023) makes the inversion explicit. Keep the base model in GPU memory once. Store all adapters in CPU RAM. When a request arrives, fetch the adapter for that request — only for the layers active in the current batch — and run the addition on-device.

The key mechanisms:

Unified Paging. S-LoRA uses a single memory pool that manages both KV cache blocks and adapter weight blocks under the same paging scheme. Adapters are not loaded wholesale — they are fetched in fixed-size pages, on-demand, from CPU to GPU. The same mechanism that prevents KV cache fragmentation prevents adapter fragmentation.

Heterogeneous Batching. Requests in the same batch may use different adapters with different LoRA ranks. S-LoRA's custom CUDA kernels compute the BA additions for all active adapters simultaneously, operating on non-contiguous memory without serializing across adapter boundaries.

Two-tier memory. Adapters not in the current batch live in CPU RAM. Warm adapters — recently used — are paged back to GPU in milliseconds via DMA transfer. Cold adapters load from disk. The base model never moves.

The Numbers

S-LoRA scales to thousands of adapters on a single GPU. Against HuggingFace PEFT, it delivers up to 30x throughput improvement. Against naive multi-instance vLLM (the merge-and-serve approach), it improves throughput up to 4x and serves orders-of-magnitude more adapters concurrently.

vLLM now supports multi-LoRA natively — a direct consequence of the S-LoRA work. As of vLLM 0.15.0, multi-LoRA serving works across dense models (Llama, Qwen) and MoE architectures (DeepSeek, Qwen-MoE).

The Engineering Principle

An adapter is not a model. It is a residual — a correction on top of a shared foundation. Serving infrastructure that treats adapters as models pays the full weight cost for every variant. Serving infrastructure that treats adapters as residuals pays the base cost once, then the delta cost per request.

The distinction is not a performance optimization. It is an architecture decision that determines whether fine-tuned serving is economically viable at all.

If you are running more than one fine-tuned variant of the same base model, your deployment architecture should reflect what LoRA actually is.

References

1. Sheng, Y., Cao, S., Li, D., Hooper, C., Lee, N., Yang, S., Chou, C., Zhu, B., Zheng, L., Keutzer, K., Gonzalez, J. E., & Stoica, I. (2023). S-LoRA: Serving Thousands of Concurrent LoRA Adapters. arXiv:2311.03285. https://arxiv.org/abs/2311.03285

2. LMSYS Org. (2023). Recipe for Serving Thousands of Concurrent LoRA Adapters. LMSYS Blog. https://www.lmsys.org/blog/2023-11-15-slora/

3. vLLM Team. (2024). LoRA Adapters — vLLM Documentation. https://docs.vllm.ai/en/stable/features/lora/

4. vLLM Team. (2026). Efficiently serve dozens of fine-tuned models with vLLM on Amazon SageMaker AI and Amazon Bedrock. vLLM Blog. https://vllm.ai/blog/2026-02-26-multi-lora

5. UC Berkeley Sky Computing Lab. (2023). S-LoRA Project Page. https://sky.cs.berkeley.edu/project/s-lora/

devinfo.dev. (2026). "The Adapter Is Not the Model." devinfo.dev:2026.0064. https://devinfo.dev/d/2026.0064