Local AI on a VPS: choose the workload before the server

Start with the workload, not a VPS label

“Local AI” can describe very different operating patterns: one person querying an interactive model, a scheduled batch that processes a bounded folder, an API serving several approved clients, or a stack that also embeds documents and retrieves passages. The words do not say how much work arrives, when it arrives, what must remain available, or what happens when the process is busy. A product name cannot answer those questions for you.

Describe the job before comparing plans. Name the real task and the people or workflows that will call it. Separate a private deployment objective from a guarantee: keeping inference on a server you administer may reduce which external services receive prompts, but it does not by itself secure that server, establish model rights, or prove that every adjacent service keeps data local. Record the intended boundary and validate each component against it.

Record the job and caller

Complete one row per materially different workload. Use units and a source for each estimate; if you do not know a value, mark it unknown and assign someone to resolve it. Do not turn a guessed peak into a guaranteed capacity requirement, or omit the peak because average use looks small.

Keep these as requirements, not claims that a candidate server meets them. Later, compare the requirements to published host fields and a controlled observation on the exact candidate. A result from another model, runtime build, region or time window is not a substitute. Where no observation exists yet, leave the field open.

Keep optional architectures optional

Mark every component selected, rejected or undecided. A simple path may need only inference and a caller. Retrieval-augmented generation (RAG) may add an embedding model, source corpus, vector database and backup. A user interface or orchestration service may add its own process, data and access boundary. A remote API is a different placement choice. These are alternatives or additions only when your design actually selects them.

Do not add every conceivable component into one imagined bill or capacity total. Equally, do not call an unselected component free if it later becomes necessary. For every selected component, record a placement and owner; for an undecided one, record the decision needed and leave its quantities unpriced. If the intended workflow needs a service that is missing from the plan, the plan is incomplete.

Build the dependency ledger

Follow the chain from the exact artifact through the program that serves it, the workload and neighboring processes, the caller and network controls, then the state that must survive an update or failure. The chain ends with provider terms and exit needs. A break in any link can invalidate a purchase even when the headline CPU, RAM or accelerator figure looks attractive.

Keep quantities distinct. Artifact files occupy storage; system RAM, GPU VRAM and runtime allocation describe different resources. Activations and key/value (KV) cache depend on execution choices, while downloads, build caches, RAG data and recovery copies have their own disk paths. A model file size does not imply a universal RAM multiplier. Do not use a single “AI-ready” label to merge these fields.

Identify artifact, runtime, and target host

For each artifact, save its repository or source URL, exact revision or immutable identifier, file format, quantization/tensor metadata when supplied, file names and byte sizes, shard count, and associated projector, adapter or other sidecar. A repository name or a shorthand such as “7B Q4” is not a complete inventory: repositories can contain variants and GGUF can use mixed tensor encodings. Verify the files you would actually retrieve and the rights attached to them.

Then record the runtime, exact release or commit, build method, operating system and architecture, and whether the candidate exposes the required CPU instructions or accelerator and driver path. Compatibility is a gate to check against the chosen release documentation; it is not established by an API being described as OpenAI-compatible. That phrase does not promise complete feature fidelity, equivalent behavior or security controls. Runtime documentation changes, so capture the version and the documentation date alongside the decision.

For the host candidate, record provider, product and region as selected by you, published CPU architecture, RAM, storage allocation and accelerator details if disclosed. Mark unavailable specifications as unknown. Do not infer a GPU, VRAM amount, driver access, stock, support entitlement or suitability from an “AI” product label. Detailed artifact and memory fields go to the model and memory worksheet; processor requirements go to the CPU/GPU procurement checklist.

Identify data, network, and exit needs

List input sources, derived data, indexes, logs and application state separately. For each, note where it lives, who owns it, its retention period, access rules, backup method and acceptable loss. If you use a vector database, count its source corpus, indexes, replicas, exports and recovery copies as separate structures; a database label is not a complete storage inventory. The RAG and vector-data page owns that detailed inventory.

Draw the caller path in words: caller location, destination service, name or address used, network boundary crossed, and control that authorizes access. A process bind, container port publication, host firewall, provider network policy and application authentication are different layers. A successful local request does not demonstrate that a remote caller can reach the service safely. The endpoint reachability guide owns the per-topology review.

Finally, record what must be recoverable and what must be removable when you leave. Distinguish a model that can be retrieved again from prompts, private documents, adapters, configuration and indexes that may not be reproducible. Identify the person who will export and verify a copy, the destination, retention, and the recipient or deletion authority. Runtime unload, stopping a host, destroying a host, deleting files and canceling a subscription are separate actions. Keep this hub at the handoff level: the lifecycle and repull page details those states.

Decide whether the operating model fits

Fit is more than whether a model might start. It includes whether you can operate the service, maintain the selected software path, protect the intended caller boundary, retain the required data, and absorb the costs and recovery work. This is a decision framework, not a promise that a VPS or any vendor will meet an unmeasured target.

Usually worth evaluating

Continue to a candidate comparison when the workload has a named operator; inputs, outputs, peak window and caller are bounded well enough to investigate; you can identify the exact artifact and runtime release; and a plausible host path exists for verification. You should also know what state persists, how it is recovered, what rights and provider terms need checking, and who will review exposure before a service is reachable beyond its intended boundary.

A limited trial may be a sensible next decision if it has a defined scope, authorized data, a time window, stop criteria and an owner. Decide in advance what evidence would change your mind: for example, a required build feature unavailable on the candidate, a resource observation outside your stated limit, an unacceptable caller path, or unresolved terms. A trial is not automatically representative of production, and this article does not claim to have run one.

Pause or choose another path

Pause procurement if the required model or its rights are unclear; a runtime/build or host capability is unknown and decisive; required availability has no operator or recovery plan; a caller needs an unreviewed public endpoint; the provider terms are unresolved; or you cannot estimate the selected data and recurring obligations well enough to approve them. An empty requirement is a reason to investigate, not evidence that the requirement is zero.

Possible alternatives include reducing the task to a bounded batch, using a remote service after reviewing its data and terms, or postponing until the requirements are known. None is universally cheaper, safer or more suitable. Compare only the alternatives that meet the same workload and data boundary. If the unresolved issue is legal or contractual, collect the relevant text and obtain appropriate review; this page is not a legal verdict. Product permission, provider acceptable-use terms, model and dataset licenses, and rights in derived material are separate questions.

Walk the procurement sequence

Move in dependency order and carry the same workload ledger forward. Each destination returns information for this decision; it does not make a purchase recommendation without your requirements. Keep dates, versions and evidence with the fields they support.

Select artifact and memory worksheet

First identify the files and execution context. The artifact, quantization and VRAM worksheet separates file bytes, system RAM, VRAM, runtime allocation, activations, KV cache, concurrency, context, download cache and temporary/recovery disk. Return the actual file inventory and unresolved memory fields here. Do not choose a host from parameter count alone. Next, the runtime comparison checks whether a selected runtime and release support the needed format, platform, API behavior and operating requirements.

Context guidance is version-sensitive. The captured Ollama context page describes VRAM-based context defaults while its FAQ describes a 4096-token default with an override. Treat this discrepancy as a version and configuration check, not a universal default. Likewise, documented q8 or q4 KV-cache ratios apply to KV cache only, not model weights or total VRAM, and can have accuracy tradeoffs. These details belong in the relevant worksheet, not a one-number shortcut here.

Choose CPU/GPU and calculate cost

With a runtime path in view, use the CPU versus GPU checklist to compare disclosed architecture, device access, driver/toolkit responsibility, runtime build and security boundary. It does not rank speed or infer that basic CPU support equals GPU performance. Keep a candidate open if the required hardware or software details are missing.

Then pass selected host, disk, traffic, copies and period to the storage and egress cost worksheet, which alone owns the aggregate cost roll-up. Verify current units, included quantities, rates, taxes and terms directly for the chosen product and date. Unknown prices remain unknown; there is no general assumption of free or unlimited ingress or egress. Do not add mutually exclusive host options to the same total.

Check terms, reachability, and exit

Review the actual provider acceptable-use and service terms, the selected model and data licenses, and the intended activity separately. Captured Contabo material dated December 2025 and Vultr material dated March 11, 2026 are bounded historical examples, not current approvals or universal rules. Confirm the exact current terms for the chosen service and date. Fair use depends on context; an editorial checklist cannot decide it for you. The rights and terms checklist tracks those distinct questions.

Before exposing a service, enumerate callers and intended destinations, then test only from authorized vantage points under an approved plan. Process bind is not network authorization, and CORS is not authentication. Docker publication, host firewall rules and provider controls can affect different paths. See the private-network and bind guide for the topology worksheet and stop conditions. Separately, model retrieval, retained disks and recurring subscriptions can have different exit consequences. See the stop, deletion and repull guide before deciding what can be removed and who may approve it.

Add RAG and n8n only when selected

If retrieval is actually required, send collection, embedding, vector-index, payload, replica, snapshot and recovery requirements to the RAG and vector database storage guide. Its structures have separate disk and memory behavior; do not count them as a single “database size.” Keep the original source corpus and recoverable exports visible.

If n8n is part of the selected service, decide where its resident processes and AI/data services will run using the same-box versus split placement guide. This hub does not cover workflow construction, payload handling, runners or n8n security settings; those remain with the n8n decision pages. Once a child review is complete, bring its candidate values and still-open questions back into the fit ledger. Do not treat a link click as verification.

Fit worksheet and handoff

Copy one row per selected component and one separate row for each undecided component that could change the architecture. “Exact version/source” should be a URL or immutable identifier plus date; “evidence needed” names a document, field or authorized observation. Give every open value an owner and due decision. Preserve unknowns through handoff instead of filling blanks with zeroes.

Acceptance gate: every selected component has an owner, placement and next check; every network caller has an intended path and review; every persistent item has a retention and recovery or deletion decision; provider and artifact terms have a dated source; and undecided components remain visibly undecided and unpriced. The handoff is not complete while a decision-critical unknown is ownerless. Attach the completed ledger to your procurement review, then ask the cost page to calculate from the selected values only.

For the candidate comparison, state the requirement and evidence side by side: requirement ______; candidate field ______; source and date ______; observation still needed ______; acceptance threshold and owner ______. If the candidate misses a threshold or the evidence cannot be obtained, stop and revise the architecture or the requirement with its owner. Do not quietly relax a privacy, availability or rights requirement to make a server appear to fit.

Route map: choose the next open decision

Evidence and updates

This selection hub preserves the existing Local AI route while replacing generic model-volume assumptions with a workload-led decision and child-page handoffs. It uses captured primary documentation for bounded technical context: Ollama context, FAQ, hardware, Linux and Docker documentation; vLLM installation, quantization, cache, CPU and API documentation; llama.cpp build and server documentation; Qdrant storage, memory tiers and snapshots; Docker publishing/firewall guidance; and provider/model rights materials. Sources establish only their stated scope and capture date.

Version-sensitive defaults, release compatibility, provider terms, product availability, billing units, price and user-specific capacity remain open until checked for the selected release, region and date. The captured Ollama context page and FAQ differ on their default-context wording; resolve against the deployed release and explicit configuration. llama.cpp commit material records source lineage, not a broadly released behavior. A source’s API examples do not establish security or deployment suitability.

Community issue captures are individual reports or requests. They are useful as prompts to inspect disk, context or recovery behavior in the reader’s chosen version; they do not establish frequency, root cause, support commitments or a general product defect. AI22 and AI28 were rejected captions and are not used as positive evidence. The lineage records AI46 and AI50 do not prove a release. Duplicate reacquisitions AI52–AI54 are not independent confirmation.

• Ollama context length documentation — captured 2026-11-06; compare with the versioned FAQ and explicit configuration.
• Ollama FAQ — captured 2026-11-06; includes context, model storage, concurrency and network behavior whose current version must be checked.
• vLLM documentation — captured documentation set 2026-11-06; select exact release and platform before relying on a feature.
• llama.cpp project documentation — capture and commit lineage are not a release guarantee.
• Qdrant documentation — version and configuration scope apply to storage and recovery guidance.
• Docker port publishing and Docker firewall behavior — host and provider controls still require a topology-specific review.
• Hugging Face repository documentation and license metadata documentation — inspect the exact artifact and its terms.

Source capture(s) checked 2026-11-06. This is an editorial source and structure review, not a deployment, benchmark, purchase, network test or legal review. Recheck mutable documentation and provider terms before acting. For evidence method and limitations, see Methodology.