Ask questions about your own documents privately

Contracts, price lists, policies and manuals hold answers you need every week, but finding them means opening file after file. This guide shows you how to set up a private AI app on your own computer that reads a folder of your documents and answers questions about them, without uploading anything to the internet.

Before you start

• A computer that can run private AI. If you’re not sure, start with the “Check if your computer can run private AI” guide.
• A private AI app installed with at least one model downloaded. The “Install private AI on your computer” guide walks you through it.
• 10 to 20 documents on one topic, such as your service agreements, staff handbook or product manuals.
• About 30 minutes.

Want to do it yourself?

Difficulty

6 out of 10

Estimated completion time

1–2 hours for the first document folder

Skills you’ll need: Organize files, configure a document-capable AI app, and check answers against the cited source passages.

I’ll do it myselfI’d rather get help

How this helps your brand

Finding answers in approved policies and manuals can help your team respond consistently. Checking the cited passage keeps an AI summary from silently becoming a new business rule.

1. 1

   Step 1: Pick an app that can read documents

   Two beginner-friendly choices, both free to download from their official sites:

   • LM Studio (lmstudio.ai): you attach files to a chat and ask about them. Good for a handful of documents at a time. Its help pages say documents you add stay on your computer.
   • AnythingLLM (anythingllm.com): you set up a “workspace” and add a collection of documents to it, so every chat in that workspace can draw on them. Good for a larger set you’ll ask about again and again. It can use its own built-in model or one from LM Studio or Ollama.

   Tip: Download only from the official website, and read the app’s privacy or data page to confirm what, if anything, it sends online. Some apps can also connect to cloud AI services; make sure you’ve chosen a local model.

2. 2

   Step 2: Make a small, tidy folder

   The AI gives better answers from a clean set of files than from everything on your computer. Make a new folder and copy in only what you need for one topic.

   • Use the latest version of each document and leave out old drafts. Otherwise the AI may quote an out-of-date price or rule.
   • Give files clear names, like “Service agreement - 2026.pdf” rather than “scan0042.pdf”.
   • PDFs, Word documents and plain text files usually work. Scanned images of paper may not, unless the text can be selected.

   Tip: Start with one topic. You can make more folders or workspaces later.

   Illustrated exampleChoose the files that belong in the folder

   Customer agreementsOne topic · current versions only

   • Service agreement - 2026.pdfLatest agreement · text can be selected
     Keep
   • Cancellation policy - 2026.docxCurrent policy for the same topic
     Keep
   • Service agreement - 2024 draft.pdfOld terms could conflict with the current file
     Leave out

   Have a scanned PDF? Try selecting a sentence. A picture of text may need text recognition before the app can read it.

3. 3

   Step 3: Add the documents to the app

   In LM Studio, start a new chat and attach your files using the attach option next to the message box, or drag them in. In AnythingLLM, create a workspace, then add your documents to it so every chat in that workspace can use them.

   Long documents can take a few minutes to process the first time. Menus change between versions, so if you can’t find the option, search the app’s official help pages for “documents” or “attach files”.

4. 4

   Step 4: Ask plain questions and ask where the answer came from

   Ask the way you’d ask a colleague, one question at a time. Ask it to name the document, and quote the line, for every answer. That makes it quick to check.

   Good first questions:

   • “What’s our cancellation notice period? Quote the line and name the document.”
   • “Which of these agreements mention late payment fees?”
   • “Summarise the main safety steps in the equipment manual, in a numbered list.”

   Tip: If the answer is vague or wrong, try naming the document in your question, or ask about a shorter section. Smaller models do better with focused questions.

5. 5

   Step 5: Check anything that matters

   Private AI keeps your files on your computer, but it can still misread them or fill a gap with a guess. For anything you’ll act on, like a price, a deadline or a contract term, open the document and check the quoted line yourself.

   If the app regularly gets things wrong, try a bigger model if your computer can handle it, or split the folder into smaller topics.

   Illustrated exampleTrace an answer back to the original document

   Example questionWhat’s our cancellation notice period? Quote the line and name the document.

   AI answer30 days’ written notice.

   “Either party may cancel with 30 days’ written notice.”

   Source: Service agreement - 2026.pdf, section 4

   Open the file

   Service agreement - 2026.pdf

   Original document · example4. Cancellation

   Either party may cancel with 30 days’ written notice.

   Notice must be sent to the contact named in this agreement.

   Check before acting: the file version, exact wording and any surrounding conditions. A citation is a starting point for checking.

Answer my questions using only the documents I’ve added. I’m a small business owner, so keep answers short and in plain English.

The documents are about: [for example, our customer service agreements]
My question: [your question]

Rules:
1. Use only what’s in the documents. Don’t add outside knowledge or guess.
2. For every answer, name the document it came from and quote the exact line or sentence.
3. If the documents don’t answer the question, say “I couldn’t find this in the documents” and suggest which document I might check.
4. If two documents disagree, show me both and tell me which one looks newer.