Skip to content

Self-hosted LLMs and private AI systems

Run AI in your own environment when data can’t go to public AI services

Some data can’t go to a public AI service: client files, health records, contracts, source code. We build AI assistants on open-weight models (ones you can download and run yourself, such as Llama, Mistral or Qwen) inside your cloud account or on your servers. They answer from your documents and databases with a link to the source, and each person sees only what their role allows. If an EU-hosted business API already meets your rules, we’ll say so, because it’s cheaper and quicker to start.

  • Models running in your cloud or on your servers
  • Answers from your documents, with the source linked
  • The same access rights as your existing systems
  • Every question and answer logged on your side

Does this sound familiar?

The situations where teams usually call us.

  1. Your security or legal team has said no to public AI tools for anything that contains client data.

  2. Staff paste internal documents into public chatbots anyway, because nothing approved does the job.

  3. You want to ask questions about contracts, policies or case files without uploading them anywhere.

  4. A client contract or a regulator requires that data stays in your infrastructure or in one country.

  5. You ran an AI pilot on a public API and now need something that passes your security review.

What happens when someone asks a question.

Everything here runs inside your environment. The model is only one part. Most of the work is finding the right documents and respecting permissions.

  1. An employee asks In a chat window, in Teams or inside your own tool
  2. Permissions are checked Only sources this person is allowed to open
  3. The right passages are found Search across documents and databases by meaning
  4. The model writes an answer On your server, using only those passages
  5. Answer with sources Links to the documents used, and a log entry
This pattern is called RAG (retrieval-augmented generation). The model isn’t trained on your data: it reads the relevant passages each time, so new documents count as soon as they’re indexed.

What we’ve learned.

What we tell teams before they buy a GPU server.

  • 01

    You may not need self-hosting.

    Many data rules are met by a business API with EU data hosting, a data processing agreement and no training on your data. It’s cheaper and gives you the strongest models. Self-host when contracts, regulation or your own risk assessment say data must not leave your control, or when your volume is high and steady.

  • 02

    Open models are good enough for most internal tasks.

    Open-weight models handle summaries, answers from documents, data extraction and drafts well. The best commercial models are still ahead on hard reasoning and long, complex tasks. Test both on your real questions, not on public benchmarks, before you decide.

  • 03

    Retrieval matters more than the model.

    Most poor answers come from the search step: scanned PDFs with no text layer, outdated versions, documents split in the wrong places. Cleaning up and structuring the sources usually improves answers more than switching to a bigger model.

  • 04

    Count the running costs, not only the hardware.

    A GPU server costs money whether it’s busy or not, and someone has to update the model, monitor it and patch the system. Renting GPUs in an EU cloud is often a better start than buying hardware. Buy once usage is steady and you know the size of model you need.

What we can build.

  • Private deployment

    • Open-weight models (Llama, Mistral, Qwen and similar) on your servers or in your cloud account
    • Rented GPUs in an EU cloud or your own hardware, sized to your users
    • Model servers such as vLLM or Ollama, behind your firewall
    • Hybrid setups that send only non-sensitive work to an EU-hosted API
  • Answers from your data

    • Search and Q&A over SharePoint, Google Drive, file shares and wikis
    • Read access to databases, CRM and ERP through their APIs
    • Answers with links to the source document and passage
    • Text extraction from scans and PDFs, including tables
  • Access and control

    • Single sign-on with your Microsoft or Google accounts
    • Permissions taken over from the source systems
    • Logs of questions, answers and sources, kept in your environment
    • Rules for topics the assistant must not answer
  • Running it

    • Test sets of real questions to measure answer quality
    • Monitoring of usage, speed and load
    • Model updates tested before they go live
    • A chat window, a Teams app or AI inside your own tools

Public AI API, EU-hosted API or a self-hosted model?

All three can be used safely for the right data. The difference is who processes it, and what you give up for control.

Public AI APIEU-hosted business APISelf-hosted model
Data stays in your infrastructure No No Yes
Data stays in the EU Partly Yes Yes
Strongest model quality Yes Yes Partly
Time to start Days Days to weeks Weeks
How cost behaves Pay per use Pay per use Mostly fixed
Who maintains the model Provider Provider You or us
Works without internet access No No Yes
If an EU-hosted API meets your rules, start there: it’s cheaper and quicker. Self-host when data must stay under your control, or when volumes are high enough to keep your own hardware busy.

How the first version comes together.

A typical timeline. We confirm yours after the project workshop.

  1. Week 1

    Data rules and one use case

    We go through the data involved and what your rules and contracts require, then pick one use case with a clear owner, such as questions about contracts or internal policies. You get a fixed price.

  2. Weeks 2–3

    Test models on your questions

    We run open models on a set of your real questions next to an EU-hosted API, so you see the quality difference before choosing.

  3. Weeks 4–6

    Build and connect

    Deployment in your cloud or on your servers, document sources, permissions and logging. A working demo every week.

  4. Then

    Pilot and widen

    One team uses it daily while we measure answer quality, then we add sources and teams. You own the code and the setup.

A first version usually takes 6 weeks to 3 months. Buying your own hardware can add weeks, so a pilot can start on rented GPUs and move later.

Case studies.

Systems we’ve built that are close to this one.

  • Shovels.ai

    Construction Intelligence · Lafayette, CA

    Charlie: Ask questions about construction data

    An AI assistant that answers questions about building permits and contractors using current database records.

    Read the case study
  • Datair

    Data analytics · Bratislava, Slovakia

    AI data analyst

    Datair uses AI to help people analyze data and create reports without technical knowledge.

    Read the case study

Questions people ask.

Do we need our own hardware?

Not to start. A pilot can run on GPUs rented in an EU cloud, in your own account. Buying hardware makes sense once usage is steady and you know what size of model you need.

Is a self-hosted model as good as ChatGPT or Claude?

For summaries, answers from documents, extraction and drafts, current open-weight models are good enough for most internal use. On hard reasoning and long, complex tasks the best commercial models are still ahead. We test both on your real questions so you can see the difference before you decide.

What does it cost?

A private assistant with several sources and permissions usually fits our connected-system range of €10–30k. A pilot on a single source can fit the €5–10k range. Hardware or rented GPUs are a separate running cost, which we estimate before you commit, and the build price is fixed after the project workshop.

Does this solve GDPR and the EU AI Act?

It removes one big question: which outside company processes your data. GDPR still applies to what you store and for how long, and the EU AI Act adds duties depending on what the system is used for. We help document the setup, but it isn’t legal advice, so your data protection officer or lawyer should confirm it.

Can the assistant respect our existing permissions?

Yes. It checks who is asking and searches only the documents and records that person can open in the source system. Someone in sales won’t get answers from HR files.

Will you train the model on our data?

Usually not. The model reads the relevant passages when a question is asked (RAG), which is cheaper, keeps answers current and makes deleting data simple. Fine-tuning pays off only in narrow cases, such as a fixed output format or a specialised classification task at high volume.

Who runs and updates it after launch?

You own the code, configuration and accounts. Bug fixes are free for 3 months. After that we can handle updates and monitoring on monthly support, or hand everything over to your IT team.

How long does a pilot take?

A first version usually takes 6 weeks to 3 months, depending on how many sources are involved and whether hardware is already in place. Starting on rented GPUs avoids waiting for hardware delivery.

Which data can’t leave your company?

Tell us what data is involved and what your rules require. We’ll tell you whether you need self-hosting at all, and what a first use case would cost.

JurajCEO, QikBuild

You leave the call with

  • What we would build first
  • A rough price range
  • Whether custom software makes sense for you
Book a 30-minute call

Or write to hello@qikbuild.com