In-House LLM Cost & Budgeting — A 2026 Sizing Guide by Company Scale

The Question That Started in a Boardroom

Last week, in a boardroom in Gangnam, a CEO asked me point blank: "So how much will it actually cost us?" I paused for a beat and answered: "As things stand, I can't give you an exact number. But I can narrow it down to a range for you before the day is out."

To be honest, in-house LLM cost is not the kind of number that pops out the moment someone asks for a quote. It swings by a factor of five depending on how many GB of data you have, how many people ask how many questions per day, and how many existing internal systems you need to hook into. In the previous piece, Choosing Your In-House LLM Stack, we laid out the three tracks — commercial API, open source, and on-prem. This piece is about moving that choice into a real cost sheet.

Four Variables That Move the Quote

Why does the price of "just one chatbot" swing so wildly? Four variables explain roughly 80 percent of any LLM build budget.

First, the volume and cleanliness of your data. Indexing 200 pages of internal manuals is a different beast from indexing 200,000 pages. Second, users and traffic. Thirty in-house users asking ten questions a day is a completely different infrastructure tier from 30,000 customers hitting the system around the clock. Third, integration scope. Wiring into ERP, CRM, and groupware all at once is effectively an SI project on top of the LLM work. Fourth, whether operations and maintenance are included. If the twelve-month operating period is missing from the quote, you always have to redo the math.

What the Build Cost Actually Consists Of

Initial build cost breaks into four buckets: the RAG pipeline (data collection, cleaning, indexing, retrieval integration), fine-tuning if needed, internal system integration, and initial data cleanup. As a rough Korea-market gauge, the RAG pipeline itself typically lands in the $4,000–$12,000 range. LoRA fine-tuning of a 7B–14B open-weight model costs only $400–$2,400 in raw GPU time, but engineer hours on top of that (roughly $6,000–$12,000 per month, usually one to three months) are what actually drive the total. If you're seriously considering a full 70B-class fine-tune, it's safer to assume you're starting from $24,000.

The axis that swings enterprise AI quotes the hardest is actually integration development. Every time someone in a boardroom says "can't we just plug it into our groupware?", the first thing I do is ask about the state of that groupware's API documentation.

Operating Costs — The Money That Goes Out Every Month

More budgets collapse here than during the build. Three line items dominate: token spend, infrastructure, and the maintenance SLA. A monthly operating bill for a chatbot layered on a commercial LLM API typically runs $250–$1,600, driven by usage. Once you add a RAG stack — API, vector DB, and application servers — that climbs to $400–$2,400 per month. If you self-host, model serving infrastructure alone is a fixed $800–$4,000 per month before anything else. And if the SLA and change-request rates aren't written into the contract, expect a dispute six months in. Every time.

A Sizing Guide by Scale — Narrowing to Three Tiers

In actual client meetings, the intuitive ranges we walk CEOs through generally fall into three tiers. These are judgment brackets, not quotes.

Subscription (API) vs. Self-Hosting — The LLM TCO Break-Even

If the previous piece was about picking the stack, this one is about projecting that stack across twelve months of real cost. Pulling together domestic and international data, the range where a managed API wins is surprisingly wide. Below several million to a few tens of millions of tokens per month, the API is almost always cheaper. The self-hosting break-even for frontier-class models sits somewhere around 100M–200M tokens per month, and can drop to 5M–10M tokens per month if you narrow the comparison to premium tiers. That said, self-hosting always carries a hidden cost: 20–30% of a senior engineer's time on standby. That's another $4,000–$6,000 a month, separately.

That's why the 2026 consensus on AI adoption cost is hybrid: keep predictable baseline traffic on your own infrastructure, and route spikes and frontier-class requests through the API.

Five Lines to Insist On in the Contract Before Budgeting

Reviewing quotes from a lot of companies, these are the items that were repeatedly missing. Get them documented before signing.

• How monthly API usage is billed separately, and where the ceiling is set
• Hourly rate and turnaround time for change requests
• Maintenance SLA — incident response time and regular release cadence
• Whether source code and model weights can be transferred to you, and under what conditions
• Initial data cleanup priced as its own line item (bundling it into build cost invites disputes later)

One more thing: if you're a budget-constrained SME, it's worth running government support programs in parallel. Domestic surveys repeatedly identify the same three gaps — budget, headcount, and data cleanup — and support programs meaningfully close the first two.

Wrapping Up — Start with a Rough Estimate

At 5years+, we've supported SME and mid-market in-house LLM adoption in Korea and Japan, from initial diagnosis through operations. Narrowing that "how much?" question into a range before the meeting ends is one part of what we do. If you have a specific scale or approach in mind, feel free to reach out via Rough estimate — reply within 24 hours.

In the next piece, we'll cover the security and governance items that absolutely have to sit inside the quote — PII masking, audit logs, guardrails, RAG permissions. Skip them at the start, and roughly six months in, the rework invoice always comes back.