Choose a private AI model

Private AI apps offer hundreds of models with confusing names, and new ones appear every month. You don’t need to understand them all. This guide shows you how to narrow the list to two or three that fit your computer and your work, test them side by side on real tasks, and keep the one that does the job.

Before you start

• A private AI app installed, such as LM Studio, Ollama or AnythingLLM. The “Install private AI on your computer” guide covers this.
• Your computer’s memory (RAM) in GB. The “Check if your computer can run private AI” guide shows you where to find it.
• Three real tasks from your work, such as a customer email to answer, a page of notes to summarise and a paragraph to rewrite.
• About 45 minutes, plus download time.

Want to do it yourself?

Difficulty

6 out of 10

Estimated completion time

1–2 hours, plus model downloads

Skills you’ll need: Check memory limits and licences, install candidate models, and compare their answers on the same real tasks.

I’ll do it myselfI’d rather get help

How this helps your brand

Testing models on your own work helps you choose a private assistant that is usable at a realistic speed. Checking its output and licence supports a dependable business workflow rather than relying on model size alone.

1. 1

   Step 1: Write down what you need it for

   A model that’s great at writing may be poor at reading long documents, and the other way round. Before you browse, write one line for each:

   • The main job: drafting emails and posts, summarising, answering questions about documents, or something else.
   • How long your typical input is: a short email, a few pages, or a long contract.
   • How fast it needs to be: a few seconds for quick replies, or happy to wait a minute for a summary.
2. 2

   Step 2: Learn the three labels that matter

   Model names look like code, but only three parts matter for choosing:

   • The maker and family, such as Llama from Meta, Gemma from Google, Mistral, Qwen or Phi from Microsoft. Stick with well-known makers to start.
   • The size, shown as a number and a B, like 3B, 8B or 14B. Bigger is usually smarter but needs more memory and runs slower.
   • The version, sometimes with words like “instruct” or “chat”. These versions are tuned to follow instructions, which is what you want for everyday work.

   Tip: You may also see labels like Q4 or Q8. These are compressed versions that use less memory. A smaller number means smaller and faster, with a small drop in quality. Apps like LM Studio often pick a sensible one for you.

   Illustrated exampleRead a model name in four parts

   Example model name

   Llama 3.18BInstructQ4

   1 Family + release

   Llama is the family; 3.1 identifies this release.

   2 Model size

   8B means about 8 billion parameters. Check whether it fits your memory.

   3 Instructions

   Instruct is tuned to follow prompts for everyday tasks.

   4 Compression

   Q4 is a compressed copy. The download page gives its actual requirements.

   Read the parts separately. A longer name does not mean a better fit.

3. 3

   Step 3: Shortlist two or three that fit your memory

   In your app’s model search, look for popular, recent models from well-known makers in a size your computer can handle. As rough starting points: around 8 GB of memory suits small models, 16 GB suits small to mid-size, and 32 GB or more opens up larger ones.

   Many apps show whether a model is likely to fit your computer. Pick two or three candidates, for example one small and one a size up, and download them.

   Tip: Requirements change as new models come out. Check each model’s page in the app or on its maker’s official site for its current memory needs.

4. 4

   Step 4: Check the licence if you use it for business

   Each model comes with a licence that says how it can be used. Many allow business use, but some have conditions or restrictions. The licence is usually linked from the model’s page.

   If anything isn’t clear, ask the AI tool you normally use to explain the licence in plain English, then check the original wording. For anything important, ask a professional adviser.

5. 5

   Step 5: Test them on the same three tasks

   Give each model exactly the same three tasks from your work, in a fresh chat each time. For each answer, note:

   • Was it usable with small edits, or did it need rewriting?
   • Did it follow your instructions, like length and tone?
   • Did it invent anything that wasn’t in what you gave it?
   • How long did it take?

   Tip: Use the prompt at the end of this guide for each test so the comparison is fair.

   Illustrated exampleCompare useful answers, not impressive names

   =

   Same prompt. Same source. Fresh chat.

   Keep the audience, length and tone identical for both models.

   Fictional test notes · not model benchmarks

   Your real task	Candidate A · small	Candidate B · larger
   Customer emailFriendly reply, under 100 words	Candidate A · smallUsable · 8 sec	Candidate B · largerUsable · 21 sec
   Meeting notesFive bullets, no added facts	Candidate A · smallUsable · 12 sec	Candidate B · largerAdded a date · 29 sec
   Rewrite a paragraphPlain language, keep the meaning	Candidate A · smallSmall edit · 7 sec	Candidate B · largerUsable · 18 sec

   ✓

   In this example, keep A. It gives usable answers quickly and adds no facts in these three tests. Record its full name, size and your reason.

6. 6

   Step 6: Keep one and write down why

   Pick the model that gave usable answers at a speed you can live with. Write down its full name and size, and the date you chose it. Delete the ones you don’t need to free up disk space.

   Check back every few months. Newer models often do better on the same computer, and your three test tasks make it quick to compare.

[The task, for example: Summarise the notes below in five bullet points.]

Who it’s for: [for example, my team]
Length: [for example, under 100 words]
Tone: [for example, plain and friendly]
Use only the information below. Don’t add facts. If something is missing, say so.

[Paste the email, notes or paragraph]