AI workflows

How to Chat With Your PDFs Locally, Arabic Included: Ollama or LM Studio in 6 Steps

Want answers from a confidential contract or report without uploading it? Both free apps can do it on your own computer. LM Studio is the easier path for…

8 min read
How to Chat With Your PDFs Locally, Arabic Included: Ollama or LM Studio in 6 Steps

The steps

About 30 minutes · Free tools only

  1. Check your computer: an Apple Silicon Mac or Windows 10 22H2 or newer, ideally with 16GB of RAM.

  2. Install LM Studio or Ollama and download a small model such as qwen3:4b or gemma4 e4b.

  3. Pick a local model, not a cloud one; in Ollama you can turn on local-only mode (OLLAMA_NO_CLOUD=1).

  4. Raise the context length in Ollama's settings (default 4k) and check it with ollama ps.

  5. Attach the PDF or Word file and ask for the exact sentence behind each answer, or "not in the file".

  6. Run the trust test: three questions you know the answers to and one the file cannot answer.

Both LM Studio and Ollama can process private documents on your own hardware without sending your sensitive information to outside servers, but they handle the workflow differently. LM Studio offers the smoother path for document work because you can attach .pdf, .docx, and .txt files directly into the chat session, and the application automatically determines whether to load the full text or switch to retrieval-augmented generation. Ollama features a desktop interface where you can drop a PDF file, but its default context window of 4k tokens on most PCs means it will only see an excerpt of a long file unless you adjust its settings slider. Crucially, your data remains confidential in either application only if you select a downloaded local model rather than an external cloud model. This practical recipe is designed for professionals handling confidential records—such as employment contracts, medical files, university theses, or internal operational guidelines in Arabic or English—who want immediate answers without exposing proprietary data to third-party web services.

Step 1: Check your computer specifications

Running neural networks locally requires adequate system memory and supported processor architectures. Because reading documents expands the memory needed during processing, you should verify that your machine meets the necessary minimums before downloading large model files.

Sponsored

Tired of juggling ten tabs? ToolSuite bundles the AI workflow tools power users rely on — in one place.

Try ToolSuite Now
ApplicationmacOS RequirementsWindows RequirementsMemory Advice
LM StudioApple Silicon chips (M1, M2, M3, or M4) running macOS 14 or newer. Intel-based Macs are not supported.64-bit x86 with AVX2 instruction support or ARM processors (Snapdragon X Elite).At least 16GB of system RAM is recommended; 8GB Macs may still cope with smaller models and modest context sizes. On Windows, at least 4GB of dedicated video memory is recommended.
OllamamacOS Sonoma (version 14) or newer. Apple Silicon chips run across CPU and GPU; Intel x86 chips run on CPU only.Windows 10 version 22H2 or newer. The application installation requires at least 4GB of disk space.Model storage requires tens to hundreds of gigabytes depending on how many weights you install. Operating context and models simultaneously requires substantial RAM.

Linux users can also run LM Studio on Ubuntu 20.04 or newer using its official AppImage release. If your machine meets these specifications, you are ready to set up an application and fetch a foundation model.

Step 2: Install one application and download a lightweight model

Download the free installer for either LM Studio or Ollama from the vendor platform. When opening the software for the first time, you will need to download model weights. Starting with a compact model is the safest approach: smaller models leave ample system memory for your operating system and document context, while loading significantly faster.

In LM Studio, you can discover and download models directly through the built-in search screen. In Ollama, you pull your selected model using the application interface or terminal commands. Three small starting points from the Ollama library:

  • qwen3:4b: A download of 2.5GB with an context window of 256K tokens. This small footprint leaves maximum memory headroom for handling large text passages.
  • qwen3:8b: A download of 5.2GB with a context window of 40K tokens. The model documentation claims support for more than 100 languages and dialects, though it does not explicitly list Arabic by name.
  • gemma4 e4b: A lightweight edge model that comes from a family whose small models have a 128K context window with text and image processing capabilities. The documentation for this model does not publish a language roster.

Beginning with qwen3:4b or gemma4 e4b lets you verify that your computer handles document loading without freezing before you evaluate larger weights.

Step 3: Ensure your setup stays strictly local and private

Both platforms offer access to remote cloud infrastructure alongside local execution. To keep your confidential documents private, you must ensure that your prompts never route through external servers.

In LM Studio, the catalog includes cloud-served frontier open models—such as GLM 5.2, Kimi K3, and DeepSeek V4 Pro—through LM Studio Bionic cloud services. While LM Studio specifies that these cloud endpoints operate with Zero Data Retention, reading private documents requires choosing a downloaded local model from your on-device library. When initiating your chat session, select your downloaded local model rather than any remote option.

Ollama also hosts cloud-based models whose names finish with the suffix :cloud, such as gemma4:31b-cloud. While Ollama’s documentation notes that its local execution never exposes prompts or data, its cloud models process queries on remote infrastructure and Ollama says it does not store or log that content or train on it. If you want absolute assurance that no outbound calls occur, you can enforce local-only execution.

To lock Ollama into local-only mode, set the following line inside your ~/.ollama/server.json configuration file:

"disable_ollama_cloud": true

Alternatively, launch the application with the environment variable OLLAMA_NO_CLOUD=1 before restarting the software. Upon startup, your service logs will confirm this enforcement with the message Ollama cloud disabled: true. You lose cloud models and web search, which is exactly what you want for a confidential file.

Step 4: Configure sufficient context memory for your files

Language models measure document size using tokens rather than raw character counts. As documented by LM Studio, one token equals roughly three-quarters of a word in English. However, Arabic text usually requires more tokens per word than English. Because an Arabic document consumes context allocations more rapidly, managing your context length is essential.

LM Studio approaches document length automatically according to LM Studio document processing documentation. When you attach a document, if the text is short enough to fit inside your active context window, the application includes the entire file in the conversation context. If the document exceeds the active limit, LM Studio switches to retrieval-augmented generation, fishing out relevant text fragments to construct an answer. The documentation notes that while this retrieval strategy can perform well, it sometimes requires tuning and experimentation.

In Ollama, context management relies on allocation parameters outlined in the Ollama context length reference. By default, Ollama assigns context based on available graphics memory:

  • Systems with under 24 GiB of video memory get a default context of 4k tokens.
  • Systems with 24 to 48 GiB of video memory get 32k.
  • Systems with 48 GiB or more get 256k.

A 4k default will only encompass a few pages of text, especially with token-dense Arabic files. You can expand this allocation using the slider inside the application settings. Increasing context requires more system memory. To monitor active consumption, open your terminal and run ollama ps. Check the CONTEXT column to confirm the token capacity assigned to your running session, and inspect the PROCESSOR column to view how computation splits between your graphics card and system processor.

Step 5: Attach your file and ask grounded questions

Once your model is loaded with appropriate context, bring your file into the conversation. LM Studio lets you add files through its attachment dialogue, supporting .docx, .pdf, and .txt formats. Ollama allows direct PDF drag-and-drop into its desktop workspace.

LM Studio documentation advises users to supply detailed queries that explicitly cite the terminology, phrasing, and concepts expected within the original text. To keep the language model anchored strictly to your text and avoid fabricated answers, use structured instructions. Here are four effective prompt formats for interrogating your documents:

  • Targeted clause extraction: “Based strictly on the attached agreement, what are the exact termination penalties under Section 4? Quote the specific sentence you relied on for this answer.”
  • Absence verification: “Does this operational manual describe emergency shutdown steps for cooling systems? If the document does not contain this information, state clearly that it is not in the file.”
  • Terminology alignment: “Using the specific terms found in the financial statement, what was the reported net operating margin for the third fiscal quarter?”
  • Single-item inquiry: “Reviewing only the arbitration provision of the contract, who is designated as the appointing authority? Address this single provision only.”

Submitting focused, individual questions rather than broad multi-part requests prevents the model from dropping essential details when parsing complex files.

Step 6: Run a five-minute trust test, in Arabic too

Neither vendor documents specific benchmarks for Arabic language quality or right-to-left layout handling. We have not conducted independent benchmarks on these tools, so you should not assume flawless output on complex phrasing. Before relying on model conclusions for important decisions, run a quick test on the document itself. If the file is in Arabic, ask the questions in Arabic as well as in English: a model that answers well in one can stumble in the other.

  1. Ask for a specific calendar date written inside the file.
  2. Ask for an exact financial or numerical figure recorded in the text.
  3. Ask for a named individual, department, or company explicitly stated in a clause.
  4. Ask a question regarding a subject completely absent from the file, confirming the model states that the item is missing rather than inventing facts.

If the model hallucinates missing provisions or confuses numerical figures, switch to a larger model parameter size, increase your allocated context limit, or consult our alternative recommendations.

Additionally, perform a copy-paste test on your PDF. Open your file in any standard document viewer and attempt to highlight and copy a sentence. If you cannot highlight the characters because the file contains scanned image pages, these apps may get nothing out of it. Neither developer documents native optical character recognition for scanned records.

For non-sensitive files where privacy restrictions do not apply, you can explore online parsing methods detailed in our cloud PDF summary guide. To explore broader platform differences and local setup options, read our Ollama vs LM Studio comparison alongside our comprehensive tutorial on how to run AI models locally.

Categories: AI workflows Blog
Share:

Comments

0

No comments yet.