Enterprise Architecture Framework for LLM Model Sourcing and Hosting Decisions

The purpose of this framework is to select the best-fit sourcing strategy for a specific AI use case, rather than to identify a universally “best” model.

A sound decision should separately evaluate three related choices:

This distinction is important. For example, the same open-weight model could be hosted on-premises, on public-cloud GPUs, or through a managed cloud AI service. Those options have materially different cost, sovereignty, operations, security, and lock-in characteristics.

The governing principle should be:

Apply mandatory constraints first. Optimize quality, economics and strategic fit only among eligible options.

1. Decision dimensions

The following should form the standard enterprise assessment taxonomy.

NIST’s Generative AI Profile similarly treats generative-AI risk management as a lifecycle activity involving governance, measurement and management rather than a single model evaluation. ENISA also emphasizes lifecycle and AI supply-chain risks, which supports treating provenance and infrastructure as part of the selection rather than evaluating only model outputs.

References: “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile” “Artificial Intelligence Cybersecurity Challenges | ENISA”

2. Hard constraints versus optimization criteria

Use three classes.

A. Mandatory gates

A failed gate makes an option ineligible, independent of its aggregate score.

Unknown evidence for a mandatory gate should be treated as “not demonstrated,” not as “pass.”

B. Weighted optimization criteria

Once an option passes the gates, compare candidates using factors such as:

• benchmark quality
• latency
• throughput
• TCO
• time to market
• operational effort
• customization
• portability
• provider stability
• sustainability
• cost predictability
• strategic flexibility

An option may compensate for weakness in one of these areas through strength elsewhere.

C. Context-dependent criteria

These may be either gates or weights depending on the use case.

Examples include context size, version pinning, multilingual coverage, explainability, fine-tuning, edge capability, active-active resilience, response determinism and cloud portability.

For a casual internal summarizer, 500-ms additional latency may be irrelevant. For a real-time voice bot, it could be decisive.

3. Structured decision process

A standard architecture review can use the following 13-stage process.

The most important discipline is that benchmarking and TCO occur before final scoring. Architecture teams should not award “quality = 5” because a model is considered frontier-class, nor “cost = 5” because a model is open source.

4. Reusable scoring model

4.1 Scoring scale

Use a consistent five-point scale.

Avoid words such as “good model.” Each score should relate to a measurable requirement.

For example:

• Quality 4 = ≥95% successful task completion on enterprise evaluation set.
• P95 latency 4 = ≤1.2 s against 1.5-s requirement.
• Cost 4 = second-lowest risk-adjusted three-year TCO.
• Portability 5 = standardized interface plus demonstrated alternate runtime/provider.

4.2 Recommended category hierarchy

An enterprise scoring sheet should contain underlying criteria beneath the category score.

Individual criterion weights should roll up to the category weight.

4.3 Calculation

If category weights total 100:

where:

• $W_i$ = criterion weight
• $S_i$ = score from 1–5

This produces a normalized 0–100 score.

Do not use the score alone

The decision should require all four conditions:

Eligibility gates = PASS

AND

Critical category floors = PASS

AND

Residual risks within risk appetite

AND

Weighted score competitive

For example:

Candidate A: score 87, Security 2 → reject

Candidate B: score 81, all critical dimensions ≥3 → eligible

This prevents averaging from hiding a critical weakness.

4.4 Uncertainty and evidence

Add a second dimension: evidence confidence.

Rules:

• Gate with D evidence → pending/fail.
• Critical criterion with C/D evidence → risk review required.
• Do not simply replace missing scores with averages.
• For financial/capacity uncertainty, calculate best/base/worst-case scenarios.
• Report a separate decision-confidence rating alongside the weighted score.

5. Example weights for three use-case profiles

Weights are illustrative, enterprise architecture should maintain reusable profiles and permit controlled overrides.

Sensitive internal use case

Privacy, security and compliance dominate because the consequences of inappropriate disclosure may outweigh incremental model quality.

Additional floors might be:

• privacy ≥ 4
• security ≥ 4
• compliance ≥ 4
• capability ≥ 3

High-volume customer-facing conversational AI

Performance, model quality and economics become more prominent because small per-request differences multiply across millions of transactions.

Typical additional requirements:

• availability ≥ 4
• latency ≥ 3
• capability ≥ 4
• safety ≥ 4
• economics ≥ 3

Low-risk productivity/copilot

Capability, cost and operational simplicity tend to dominate. Rapid experimentation may be more valuable than extreme infrastructure control.

Data restrictions must still operate as gates: “low risk” must not become an excuse to send unsuitable data to an unapproved service.

6. Comparison of sourcing patterns

6.1 Proprietary managed LLM API

Typical model: proprietary frontier/general-purpose model exposed directly by its developer.

Advantages

• fastest access to leading capabilities
• minimal inference infrastructure
• strong elasticity
• rapid release cadence
• advanced reasoning/multimodal/tool capabilities often arrive early
• excellent prototyping speed

Disadvantages/risks

• external inference boundary
• pricing tied to provider
• API or model behavior can change
• reduced weight/architecture visibility
• limited low-level customization
• model/version deprecation
• provider concentration
• potential network dependency

Organizational requirements

Relatively low infrastructure expertise, but strong API governance, security, AI evaluation, contract management and FinOps are still required.

Economics

Excellent for uncertain, intermittent or modest workloads. Can become expensive at high sustained token volumes.

Best suited to

Rapid product development, complex reasoning, high-value low/medium-volume workloads, frontier multimodal capabilities.

Generally avoid when

Offline inference, absolute local execution or controls unavailable from the provider are mandatory.

6.2 Hyperscaler-managed foundation-model platform

Here the enterprise consumes one or several models through a cloud platform’s control plane.

Advantages

• integration with enterprise IAM/network/security services
• private networking options
• centralized governance
• multiple model families may be available
• easier cloud application integration
• managed elasticity and operations
• consolidated cloud contracting may simplify procurement.

Risks

• potentially two layers of dependency: cloud platform plus underlying model
• model availability/features can differ by region
• pricing may include additional platform/infrastructure dimensions
• abstraction does not necessarily imply portability.

Economics

Strong for organizations already operating significant workloads on that cloud. Reserved/provisioned capacity can improve economics for stable demand.

Best suited to

Enterprise applications already integrated deeply into a cloud platform, especially when security/network controls need to align with existing landing zones.

Avoid where

The cloud itself is outside the allowed sovereignty boundary or the additional platform dependency is unacceptable.

6.3 Self-hosted open-weight model

Weights and inference infrastructure remain under enterprise control.

Advantages

• maximum control over inference environment
• offline operation
• deterministic version control
• strong customization
• ability to inspect and govern artifacts
• potential data sovereignty benefits
• no per-token external API dependency
• potential edge deployment

Disadvantages

The enterprise assumes substantial responsibility for:

• GPU infrastructure
• capacity
• serving stack
• scaling
• optimization
• vulnerabilities
• model provenance
• patching
• evaluation
• safety mechanisms
• HA/DR
• lifecycle management

Open weights should therefore not automatically be equated with lower risk.

Economics

Can be attractive where there is:

• high sustained utilization
• predictable workload
• adequate accelerator capacity
• an appropriately sized model

It is frequently unattractive for low-utilization workloads because idle GPUs dominate TCO.

Best suited to

Sovereign/private workloads, disconnected environments, edge scenarios, specialized models and stable high-volume workloads where an open model achieves sufficient quality.

Avoid where

The enterprise lacks the operational maturity required to run production model serving securely and reliably.

7. Cloud-hosted open-weight models

This occupies the important middle ground between full self-hosting and proprietary APIs.

The enterprise uses open weights but executes them on public-cloud GPUs or through a managed model-serving environment.

Advantages

• open-model flexibility
• cloud elasticity
• easier experimentation
• reduced hardware procurement
• version control
• greater deployment portability than a proprietary API
• access to accelerator generations as they become available

Disadvantages

• cloud dependency remains
• GPU capacity may be constrained
• accelerator economics can be complex
• enterprise still owns more of the model lifecycle
• managed implementations may introduce proprietary serving features

Economics

Particularly attractive when workloads exceed simple API experimentation but do not justify dedicated data-center GPU infrastructure.

Best suited to

Organizations seeking control over models without operating physical accelerator estates.

8. Hybrid and multi-model architectures

For many enterprises, this should be considered a first-class sourcing pattern rather than an exception.

Examples:

• private model for confidential data
• frontier commercial model for difficult reasoning
• small model for classification
• specialized code model for development
• fallback model/provider for resilience

The trade-off is that the enterprise exchanges provider dependency for orchestration complexity.

Multi-model architecture requires:

• common evaluation
• routing policy
• prompt compatibility
• identity/policy enforcement
• observability
• cost attribution
• fallback testing
• consistent security guardrails

9. Practical sourcing decision matrix

Ratings are archetypal, not product assessments. The actual outcome depends on the specific model, provider, service, region, contract, runtime and use case.

10. Model evaluation and benchmarking framework

No model/provider should enter production solely because of leaderboard performance.

Evaluation hierarchy

Level 1 — Technical screening

Determine whether candidates satisfy:

• context requirement
• language requirements
• modalities
• structured output
• tool calling
• deployment environment
• required throughput

This cheaply eliminates unsuitable models.

Level 2 — Enterprise task evaluation

Create a golden evaluation dataset derived from realistic tasks.

Examples:

Sensitive enterprise data should be appropriately minimized, anonymized, synthetic or otherwise handled under approved evaluation controls.

Measure more than answer quality

A useful benchmark suite should cover:

The last metric is especially important.

A cheaper model that requires repeated calls, produces longer answers and succeeds less frequently may have a higher business cost per successful transaction than an expensive model.

Human evaluation

Human evaluation remains necessary for qualities such as:

• completeness
• tone
• nuanced reasoning
• domain correctness
• harmful ambiguity
• customer appropriateness

Use blinded pairwise evaluation where possible rather than asking evaluators whether a named model is “good.”

Why public benchmarks are insufficient

Public benchmarks are useful for candidate discovery but poor production proxies because they may differ from enterprise workloads in:

• domain
• language
• prompts
• context length
• retrieved data
• tool definitions
• output constraints
• safety configuration
• quantization
• serving infrastructure

There is also a risk of benchmark contamination and over-optimization.

The correct question is not:

“Which model has the highest benchmark score?”

It is:

“Which eligible architecture produces the highest acceptable task success, at the required risk, latency and total cost?”

11. Benchmark regression and version management

Treat every significant model version change as a software dependency upgrade.

Maintain:

Current model → candidate version → shadow evaluation → regression analysis → approval → rollout → rollback capability.

Regression gates might state:

• no >2% degradation in primary task score
• zero significant security-control regressions
• no material multilingual degradation
• cost/task increase <10% unless justified
• P95 latency remains within SLA

“Latest model” should therefore not automatically mean production model.

12. Economics and TCO

Evaluate economics over at least:

• current workload
• expected 12-month workload
• high-growth scenario
• stress/peak scenario

Note: The following TCO calculations are synthesized cost-accounting models, grounded in established FinOps/TCO principles. They are presented in this context as a proposed enterprise calculation model, not as standardized published formulae.

We start with two cost functions, for a given analysis period (normally one year):

where:

• $V$ = workload volume during the period
• $F_m$ = annual fixed managed hosting cost
• $F_s$ = annual fixed self-hosting cost
• $C_m$ = managed variable cost per workload unit
• $C_s$ = self-host variable cost per workload unit

Managed LLM Service TCO

As we have seen the managed TCO can be calculated with the formula:

Now we need to identify the $F_m$ and $F_s$ quantities.

Conceptually:

The above formula is correct for $V=1$ (for the selected period).

To define a more accurate formula, we need to split the various components into fixed and variable costs.

The variable cost can calculated as:

Note: Some components can have fixed and variable costs at the same time (the elasticity of the managed services allows to start a fixed size hardware, and scale-out if workload demands it), this is why they appear in both $F_m$ and $F_s$ .

Based on information from FinOps Foundation, “FinOps for Data Center: Applying the FinOps Framework”.

Self-hosted TCO

Similarly, for the self-hosted TCO we have the conceptual calculation for $V=1$ :

Most of the components in a self-hosted scenario are fixed costs. So,

The variable cost can calculated as:

Note: Do not compare API token price with GPU acquisition price alone.

Based on information from FinOps Foundation, “Token Economics: The Atomic Unit of AI Value” and “GenAI FinOps: How Token Pricing Really Works”.

Utilization is often decisive

Consider a GPU estate sized for 100 units of capacity:

• average demand = 15
• peak demand = 85

Even if inference on fully utilized hardware is inexpensive, the enterprise pays for substantial idle capacity.

Conversely:

• average demand = 80
• peak = 95
• stable 24×7 workload

Dedicated serving may become economically attractive.

Approximate break-even

The break-even point is where:

Therefore:

so:

assuming $C_m<>C_s$ .

Often it is difficult to calculate the fixed managed hosting cost, or it is considered negligible. Consequently, the formula can be simplified to this:

This is only a first approximation. Reviews should also model:

• capacity headroom
• growth
• accelerator refresh
• hardware depreciation
• redundant sites
• operations
• model changes that alter GPU requirements

13. Architecture patterns beyond one model

14. RAG, prompting and fine-tuning decision

A useful default hierarchy is:

flowchart TD
    A{Can prompt/system instructions achieve the requirement?}
    A -->|Yes| B[Use prompting first]
    A -->|No| C{Is the issue missing/current enterprise knowledge?}
    C -->|Yes| D[RAG]
    C -->|No| E{Is specialized behavior, format or task needed?}
    E -->|Yes| F[Fine-tune/adapters]

Prefer RAG when

Knowledge:

• changes frequently
• is enterprise-specific
• needs citations
• must be access-controlled dynamically.

Prefer fine-tuning when

The objective is more about behavior than knowledge:

• repeated structured transformations
• specialized terminology
• style
• classification
• task specialization.

They are not mutually exclusive.

15. Model-routing hierarchy

An enterprise routing architecture can implement progressively stronger controls:

flowchart TD
    A[Incoming AI request] --> B[Policy classification]

    B --> C[Restricted/private]
    B --> D[Standard workload]

    C --> E[Sovereign/private model]

    D --> F[Small/cheap model]
    F --> G{Confidence sufficient?}

    G -->|Yes| H[Return]
    G -->|No| I[Frontier model]

    I --> J{Provider failure?}

    J -->|No| K[Return]
    J -->|Yes| L[Fallback]

Possible routing dimensions include:

• data classification
• use-case type
• required language
• complexity
• expected context
• user geography
• cost budget
• latency
• provider availability

Do not route sensitive requests using a model that itself must inspect the unredacted sensitive prompt unless that router is within the appropriate trust boundary.

16. Abstraction and lock-in

An abstraction layer or AI gateway can provide:

• consistent authentication
• provider/model registry
• cost accounting
• data classification
• policy enforcement
• quotas
• logging
• routing
• retries/failover
• prompt management
• evaluation telemetry

But abstraction has limits.

A supposedly generic interface rarely makes models completely interchangeable because providers differ in:

• tool semantics
• multimodality
• tokenization
• context management
• safety behavior
• structured output
• reasoning behavior

The goal should therefore be bounded portability, not lowest-common-denominator abstraction.

17. Decision heuristics

These are rules of thumb, not mandatory architecture standards.

Commercial managed models are generally favored when

• frontier reasoning provides material business value
• demand is uncertain or bursty
• time-to-market matters strongly
• workload is not subject to incompatible sovereignty restrictions
• provider security/privacy controls satisfy enterprise requirements
• internal GPU operations would create disproportionate overhead

Locally hosted open models are generally favored when

• disconnected/offline operation is mandatory
• data must remain within an enterprise-controlled execution environment
• strict model-version control is required
• the workload is stable enough to economically utilize infrastructure
• an open model achieves the required task quality
• the enterprise possesses strong GPU/MLOps/security operations

Cloud-hosted open models are generally favored when

• open-model control/customization is valuable
• physical GPU ownership is undesirable
• cloud elasticity is useful
• data is permitted in the selected cloud boundary
• portability across inference stacks has strategic value

Hybrid architectures become compelling when

• data sensitivity varies materially between interactions
• expensive reasoning is needed for only a minority of requests
• customer-facing availability warrants provider fallback
• different tasks have different optimal models
• strategic dependence on one provider is considered material

18. Anti-patterns the framework should prevent

19. Governance and lifecycle management

The framework becomes an enterprise capability when decisions are managed as a portfolio.

Approved model/provider catalog

Maintain a catalog containing:

Approval should apply to the combination, not simply a model name.

Model A via provider B in region C may be approved while Model A via provider D is not.

20. Architecture Decision Record

Every significant production decision should generate a Model Sourcing Decision Record (MSDR).

Required record

21. Reassessment triggers

Do not wait for the scheduled review if one of these occurs:

A useful policy is:

planned review every 6–12 months + event-driven reassessment.

High-risk or rapidly changing applications may justify shorter cycles.

22. Recommended enterprise operating model

The decision is multidisciplinary.

The architecture board should approve exceptions to requirements, not manually select models on behalf of engineering.

23. Recommended selection funnel

The whole framework can be summarized as:

flowchart TD
    A[AI use case]
    B[Characterize workload/value]
    C[Data + regulatory classification]
    D[Mandatory gates]

    E{Gate result}
    F[Reject option]
    G[Capability threshold]
    H[Candidate shortlist]
    I[Enterprise benchmarking]
    J[TCO + scalability]
    K[Security/legal/ops assessment]
    L[Weighted scoring]
    M[Sensitivity analysis]

    N[Preferred option]
    O[Fallback option]
    P[Model Sourcing Decision Record]
    Q[Production monitoring]
    R[Reassessment triggers]

    A --> B
    B --> C
    C --> D
    D --> E

    E -->|FAIL| F
    E -->|PASS| G

    G --> H
    H --> I
    I --> J
    J --> K
    K --> L
    L --> M

    M --> N
    M --> O

    N --> P
    O --> P

    P --> Q
    Q --> R

24. A useful overall decision principle

For governance purposes, the selection question can be expressed as:

subject to:

That distinction is fundamental.

An enterprise should not choose:

“the strongest available LLM.”

Nor should it establish a blanket rule that:

“proprietary models are better,” “open models are safer,” or “on-premises is cheaper.”

Instead, it should choose the smallest, least complex, economically defensible architecture that meets the use case’s quality requirements and all mandatory risk constraints, while retaining an acceptable migration path.

This also naturally leads to a portfolio rather than a single enterprise model: private models where control is decisive, managed frontier models where capability is decisive, smaller models where economics and latency dominate, and routing/fallback architectures where no single option optimizes all requirements. NIST’s risk-management approach and ENISA’s lifecycle/supply-chain perspective are consistent with this use-case-specific, evidence-driven governance model.

Next: apply this framework with the interactive Model Sourcing Scorecard Worksheet.