Get better answers from private AI

Private AI on your own computer can feel slow, or give answers that ramble or miss the point. Usually that’s fixable in a few minutes. This guide walks you through the changes that make the biggest difference, in the order to try them, so you end up with answers that are quick enough and good enough for everyday work.

Before you start

• A private AI app already installed with a model downloaded, such as LM Studio, Ollama or AnythingLLM.
• One everyday task you want it to do well, like drafting a reply or summarising a page of notes.
• Your computer’s memory (RAM). The “Check if your computer can run private AI” guide shows you where to find it.
• About 20 minutes.

Want to do it yourself?

Difficulty

6 out of 10

Estimated completion time

1–2 hours, plus replacement model downloads

Skills you’ll need: Compare speed and answer quality, change model settings, and retest the same task without changing several variables at once.

I’ll do it myselfI’d rather get help

How this helps your brand

A repeatable speed and quality check helps make private AI useful for everyday business work. Saving the model and prompt that passed gives you a more reliable starting point for the next task.

1. 1

   Step 1: Time one test task so you can compare

   Pick one real task and keep it the same for every test, for example “Reply to this customer email” with the same email each time. Note how long the answer takes and whether it’s usable.

   Without a fixed test, it’s hard to tell whether a change helped or you just got lucky.

   Tip: Save the test prompt in a note so you can paste it in again after each change.

2. 2

   Step 2: Close other apps while it works

   The AI model has to fit in your computer’s memory. Web browsers with lots of tabs, video calls, photo editors and games all compete for that same memory. When memory runs short, answers slow to a crawl.

   Close what you don’t need, then run your test again. On a computer with 8 or 16 GB of memory, this alone often makes a clear difference.

3. 3

   Step 3: Try a smaller model if it’s slow

   If answers take ages or the app struggles, the model is probably too big for your computer. Models are usually labelled with a size like 3B, 8B or 14B. A smaller number means a smaller, faster model.

   Download a smaller version of the same model, or a different small model, and run your test again. Small models are often fine for short drafts, rewording and simple summaries.

   Tip: Many apps, including LM Studio, show whether a model is likely to fit your computer before you download it. Check each model’s page for its current memory needs.

4. 4

   Step 4: Try a bigger model if it’s weak

   If answers come quickly but are vague, repetitive or get facts muddled, and your computer has memory to spare, try one size up. Bigger models follow instructions and handle longer documents better.

   Go up one step at a time. If the bigger model is too slow to be useful, go back. The right model is the biggest one that still feels quick enough for your task.

   Illustrated exampleCompare speed and usefulness on the same task

   Same task in every trialDraft a customer refund reply, under 100 words.

   Same email, same prompt, fresh chat. Other apps closed.

   Model sizeTime to complete an answerRead the answer too

   1. 14B
      78 seconds

      Accurate, but too slow for this task

   2. 8BKeep this one
      34 seconds

      Usable reply; includes the stated refund timing

   3. 3B
      16 seconds

      Quick, but leaves out the refund timing

   Keep the largest model that is accurate and quick enough for your task.

5. 5

   Step 5: Give it one job at a time, in fewer words

   Local models are smaller than the big online AI tools, so they lose track more easily. Long, rambling prompts with several requests at once are the most common cause of weak answers.

   • Ask for one thing per message. “Summarise this email” first, then “Now draft a reply”.
   • Put the instruction first and the material after it.
   • Say exactly what you want back, such as “three bullet points” or “under 100 words”.
   • Paste only the part that matters. Long documents use more memory and slow everything down.
   • Start a new chat for a new task, so old messages don’t confuse it.

   Illustrated exampleTurn a crowded request into one focused message

   Too many jobs in one message

   Summarise this email, draft a reply, rewrite our refund policy and make a social post about it.

   A focused message in a new chat

   1 One job first

   Draft a reply to the customer email below.

   2 A clear limit

   Under 100 words, with a greeting and sign-off.

   3 No guessing

   Use only these details. Write [missing] for anything else.

   4 Relevant text last

   Customer email: “I returned my lamp. When will my refund arrive?”

   Order note: Return received. Refund approved today; allow five working days.

   ↳Need a policy rewrite too? Give it a separate task in a new chat.

6. 6

   Step 6: Save what works

   Once your test task gives a good answer quickly, write down the model name and size, and save the prompt that worked. Many apps let you save it as a preset or system prompt so you don’t have to paste it every time.

   If nothing gets good enough on your computer, that’s useful to know too. Use private AI for the simple, sensitive jobs, and a cloud tool with good privacy settings for the rest.

[One clear job, for example: Draft a reply to the customer email below.]

Keep it to [length, for example: under 100 words].
Format: [for example: a short email with a greeting and sign-off].
Use only the information below. If something is missing, write [missing] instead of guessing.

[Paste only the text it needs, such as the email or the notes.]