Nemotron-Parse: NVIDIA's OCR family for reading documents locally

You have a pile of invoices, contracts and reports in PDF, and every time you pass them to an online service you are deciding to send them to a server you do not control. There is another route: a model that reads documents directly on your hardware, and then hands the text to any local LLM. Since mid-October 2026, NVIDIA has updated its entire Nemotron Parse family — three versions, v1.1, v1.2 and 2.0, re-released in the same week (model card) — and with under 1 billion parameters it does what classic OCR does badly: understand the structure of a page, not just the text.

What Nemotron Parse actually does

Nemotron Parse is not an LLM: it is a vision encoder-decoder model specialized in document understanding. The vision encoder is a ViT-H (NVIDIA's C-RADIO) that compresses the image into a short token sequence, and a 10-block mBART decoder produces a string that contains everything together: formatted text in markdown, formulas in LaTeX, bounding boxes with coordinates, and a semantic class for each element (title, paragraph, table, footnote, header, image). All in the natural reading order of the page.

The difference from classic OCR — which returns a flat string of characters — is that here every element knows what it is and where it is. A table arrives as LaTeX with multirow and multicolumn intact, a formula arrives as LaTeX, an invoice arrives as a structure that an LLM can query without guessing. This is what makes it the natural front-end of a local RAG pipeline: the document is "parked" in structured form, and it is the answers from the language model of your choice, always local, that come next.

Three versions in one week: what changes

NVIDIA shipped v1.1 in November 2025, v1.2 in February 2026 and 2.0 in June 2026 — and re-edited and re-published all three model cards on October 20. All three are under 1 billion parameters.

	v1.1	v1.2	2.0
Release	Nov 2025	Feb 2026	Jul 2026
Parameters	< 1B	< 1B	< 1B
Prompt	3 tokens	4 tokens (mandatory)	4 tokens
Tables	LaTeX	LaTeX, HTML, markdown, JSON, CSV	+ hierarchical JSON, CSV
Charts	no	no	yes, with the <class_Chart> class
Vocabulary	base	base	72,256 entries (+~20k, multilingual)
License	NVIDIA Open Model License	Nemotron Open Model License	OpenMDW-1.1

Two things to know immediately:

1. v1.2 and 2.0 require a fourth token in the prompt. From v1.2 on, the full prompt is </s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic> — the last token tells the model not to extract text inside embedded images, which saves generation. The model does not validate the prompt: with the old 3-token v1.1 prompt it still generates, but the quality "degrades significantly", as NVIDIA puts it. If you are upgrading from v1.1, this is the only mandatory change in your code.

2. 2.0 is a measurable jump, not a touch-up. The numbers NVIDIA reports on its benchmarks, v1.2 against 2.0:

Benchmark	Metric	v1.2	2.0
ParseBench	Overall score	0.5782	0.6391
OmniDocBench Notes	Text edit distance (handwriting, lower = better)	0.9739	0.3395
IndicVisionBench	Character ANLS	0.0612	0.7203
MOSCAR (multilingual)	BoC F1	0.4410	0.9102

The most interesting row for anyone working with mixed documents is handwriting: an edit distance that drops from 0.97 to 0.34 means text that was nearly unreadable for the model before now gets read. And the vocabulary expansion brings multilingual OCR for CJK and Indic scripts; for French and Italian — Latin scripts — the capability was already there, but 2.0 improves dense tables and chart handling.

Minimal setup: from PDF to structured text

The reference path is vLLM: 2.0 is supported on vLLM v0.20 through v0.26 (NVIDIA validated v0.20.0), and it serves an OpenAI-compatible API that any script can query.

vllm serve nvidia/NVIDIA-Nemotron-Parse-2.0 \
  --dtype bfloat16 \
  --max-num-seqs 8 \
  --limit-mm-per-prompt '{"image": 1}' \
  --trust-remote-code \
  --port 8000

Then, from Python, each PDF page becomes a request:

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
with open("page.png", "rb") as f:
    img_b64 = base64.b64encode(f.read()).decode("utf-8")
prompt = "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic>"
resp = client.chat.completions.create(
    model="nvidia/NVIDIA-Nemotron-Parse-2.0",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": prompt},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
    ]}],
    max_tokens=9000, temperature=0.0,
    extra_body={"repetition_penalty": 1.1, "top_k": 1, "skip_special_tokens": False},
)

If it does not work: if the server starts but returns garbled text, check the prompt (the missing token is the number one cause when upgrading from v1.1); if vLLM stalls at load time on recent builds, 2.0 ships a patch for the tied output-head weights — the model card explains how to add it to PYTHONPATH. PDFs must first be rendered as images (one page per request, --limit-mm-per-prompt is set to 1): with pdftoppm -r 200 invoice.pdf page you already have PNG pages at 200 dpi, and the maximum supported resolution is 1664x2048.

And the limits?

Be clear about what this model is not:

• NVIDIA GPU only, Linux. The supported architectures are Turing, Ampere, Hopper and Blackwell. No official path for CPU, Apple Silicon or AMD — the family is not in the Ollama/llama.cpp world, and Mac users stay outside this particular model.
• No official VRAM figure. All three model cards declare no required memory. Under 1 billion parameters in bfloat16 is just over 2 GB of weights, but the inference engine and generation context need space on top: for a concrete estimate on your hardware, look at the VRAM requirements guide.
• Strong multilingual OCR is new. 2.0 shines on CJK and Indic scripts (ANLS from 0.06 to 0.72), but on those script sets it just started; for Latin-script documents the reliability is high, for exotic ones verify on your own samples.
• The benchmarks are NVIDIA's. The numbers above are vendor-declared, not independently measured: before building a workflow on top, test the model on your invoices, not on samples.

The honest border: Nemotron Parse is the smart scanner, not the brain. It reads and structures; it does not answer, reason or decide. The answering is done by your local LLM, on the text the model extracted.

Where this leads

If you want to see how structured text feeds a natural-language query, the next step is local RAG: ask questions to your documents. And if you are sizing your hardware, the guide GPU or CPU: when CPU is enough tells you when this family of models really needs the dedicated card.

Sources

• NVIDIA Nemotron Parse 2.0 — model card
• NVIDIA Nemotron Parse v1.2 — model card
• NVIDIA Nemotron Parse v1.1 — model card
• NVIDIA Nemotron Parse 1.1 — arXiv 2511.20478

Article generated with AI