Merge upstream/main into docs/backport-2194-extraction-docs-fix

Resolve audio-video.md by keeping PR deploy-trim prose with main's
create_ingestor example, ffmpeg step, and workflow link. Keep
custom-metadata.md deleted; vdbs.md redirect covers deep links.

Author: kheiss-uwzoo

README.md

@@ -6,7 +6,7 @@ SPDX-License-Identifier: Apache-2.0
 
 **Important: The default branch is main, which tracks active development and may be ahead of the latest supported release.**
 
-For the latest release line use the [26.05 branch](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05) (RC builds are tagged `26.05-RC1`, `26.05-RC2`, …). The previous stable line is [26.03](https://github.com/NVIDIA/NeMo-Retriever/tree/26.03).
+For the latest supported release, use the [26.05 branch](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05) (GA PyPI and Helm chart version `26.5.0`). The previous stable line is [26.03](https://github.com/NVIDIA/NeMo-Retriever/tree/26.03).
 
 See the corresponding [NeMo Retriever Library documentation](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/).
 

docs/docs/extraction/audio-video.md

@@ -2,7 +2,7 @@
 
 Use this page for speech and audio extraction with Parakeet ASR and for video workflows that combine audio with OCR on frames or derived images.
 
-For air-gapped or disconnected deployments, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment).
+For air-gapped or disconnected deployments, refer to [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment).
 
 **Sections:** [Speech and audio (Parakeet)](#speech-and-audio-extraction) · [Run Parakeet on the cluster (Helm)](#run-parakeet-on-the-cluster-helm) · [Parakeet with hosted inference (build.nvidia.com)](#parakeet-hosted-inference-build-nvidia) · [Video and frame OCR](#video-and-frame-ocr)
 
@@ -18,7 +18,7 @@ Supported file types for speech extraction today:
 - `mp3`, `wav`
 - `mp4`, `mov`, `mkv`, `avi` — common video containers; the audio track is transcribed (same extensions as in [What is NeMo Retriever Library?](overview.md))
 
-[NeMo Retriever Library](overview.md) supports extracting speech from audio for Retrieval Augmented Generation (RAG). Similar to how the multimodal document pipeline uses detection and OCR microservices, NeMo Retriever Library uses the [parakeet-1-1b-ctc-en-us ASR NIM](https://docs.nvidia.com/nim/speech/latest/asr/deploy-asr-models/parakeet-ctc-en-us.html) to transcribe speech to text, then embeddings via the NeMo Retriever embedding path.
+[NeMo Retriever Library](overview.md) supports extracting speech from audio for Retrieval Augmented Generation (RAG). Similar to how the multimodal document pipeline uses detection and OCR microservices, NeMo Retriever Library uses the [parakeet-1-1b-ctc-en-us ASR NIM](https://docs.nvidia.com/nim/speech/latest/asr/deploy-asr-models/parakeet-ctc-en-us.html) to transcribe speech to text, then embeddings through the NeMo Retriever embedding path.
 
 Before running audio extraction from Python with either self-hosted or hosted Parakeet, install the multimedia extra so the Parakeet ASR client can decode and resample audio:
 
@@ -48,63 +48,64 @@ For Kubernetes deployments with network access to package repositories, set
 [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#1-service-image)
 to install ffmpeg/ffprobe at service startup. This runtime path requires
 package-repository network egress, a writable root filesystem, and a security
-policy that allows the image's scoped sudo use. For air-gapped clusters, see
+policy that allows the image's scoped sudo use. For air-gapped clusters, refer to
 [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment).
 
 !!! important
 
     Due to limitations in available VRAM controls in the current release, the parakeet-1-1b-ctc-en-us ASR NIM must run on a [dedicated additional GPU](prerequisites-support-matrix.md#model-hardware-requirements). For the full list of requirements, refer to the [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#model-hardware-requirements).
 
-This pipeline enables retrieval at the speech segment level when you enable segmenting (see examples below).
+This pipeline enables retrieval at the speech segment level when you enable segmenting (refer to the examples below).
 
 ![Overview diagram](images/audio.png)
 
 ## Run Parakeet on the cluster (Helm) { #run-parakeet-on-the-cluster-helm }
 
-Use the following procedure for self-hosted Parakeet on your cluster. For chart enablement, GPU placement, ffmpeg, and endpoint wiring, see [Optional Helm NIMs](prerequisites-support-matrix.md#optional-helm-nims-not-auto-wired-by-default) and [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) in the Helm chart README, plus [Deployment options](deployment-options.md).
+Use the following procedure for self-hosted Parakeet on your cluster. For chart enablement, GPU placement, ffmpeg, and endpoint wiring, refer to [Optional Helm NIMs](prerequisites-support-matrix.md#optional-helm-nims-not-auto-wired-by-default) and [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) in the Helm chart README, plus [Deployment options](deployment-options.md).
 
 1. Deploy or upgrade per that Helm guide and [Deployment options](deployment-options.md).
 
-2. After the services are running, interact with the pipeline from Python.
+2. If the service will process audio or video files, set `service.installFfmpeg=true` in the Helm chart when your cluster allows runtime package installation; for air-gapped clusters, refer to [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment) and the [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#1-service-image) for `service.image` overrides.
 
-    - The `Ingestor` object initializes the ingestion process.
-    - The `files` method specifies the input files to process.
-    - The `extract_audio` method runs audio extraction.
+3. After the services are running, interact with the pipeline from Python (refer to the [Python API guide](nemo-retriever-api-reference.md) for parameter details).
+
+    - In `batch` mode, pass the in-cluster Parakeet gRPC endpoint through `ASRParams.audio_endpoints` (for example `audio:50051` from your Helm release). The retriever service auto-wires this endpoint; graph ingest does not.
 
     ```python
+    from nemo_retriever import create_ingestor
     from nemo_retriever.params.models import ASRParams
 
     ingestor = (
-        Ingestor()
+        create_ingestor(run_mode="batch")
         .files("./data/*.wav")
         .extract_audio(
-            asr_params=ASRParams(segment_audio=True),
+            asr_params=ASRParams(
+                audio_endpoints=("audio:50051", None),  # (grpc_endpoint, http_endpoint)
+                segment_audio=True,
+            ),
         )
     )
+    results = ingestor.ingest()
     ```
 
-To generate one extracted element for each sentence-like ASR segment, pass `asr_params=ASRParams(segment_audio=True)` to `.extract_audio(...)`. This option applies when audio extraction runs with a self-hosted Parakeet NIM or using build.nvidia.com hosted inference, but has no effect when using the local Hugging Face Parakeet model.
+    To generate one extracted element for each sentence-like ASR segment, pass `asr_params=ASRParams(segment_audio=True)` to `.extract_audio(...)`. This option applies when audio extraction runs with a self-hosted Parakeet NIM or using build.nvidia.com hosted inference, but has no effect when using the local Hugging Face Parakeet model.
 
-For more Python examples, refer to [Python Quick Start Guide](https://github.com/NVIDIA/NeMo-Retriever/blob/main/client/client_examples/examples/python_client_usage.ipynb).
+    For more runnable examples, refer to [Workflow: Ingest documents](workflow-document-ingestion.md).
 
 ## Parakeet with hosted inference (build.nvidia.com) { #parakeet-hosted-inference-build-nvidia }
 
 Instead of running the pipeline locally, you can call Parakeet through [build.nvidia.com](https://build.nvidia.com/) hosted inference.
 
 1. On the Parakeet model page on [build.nvidia.com](https://build.nvidia.com/), create or copy an API key and note the function ID for hosted access. You need both before making API calls.
 
-2. Run inference from Python with the hosted gRPC endpoint and credentials from that page (the example below uses the default hosted gRPC hostname; confirm values in the **Get API Key** flow for your deployment).
-
-    - The `Ingestor` object initializes the ingestion process.
-    - The `files` method specifies the input files to process.
-    - The `extract_audio` method runs audio extraction.
-    - The hosted gRPC endpoint, function ID, and API key are routed through `ASRParams`. Pass them via `asr_params=ASRParams(...)`; the ASR actor reads `audio_endpoints`, `function_id`, and `auth_token` from that object.
+2. Run inference from Python with the hosted gRPC endpoint and credentials from that page (the example below uses the default hosted gRPC hostname; confirm values in the **Get API Key** flow for your deployment). Pass hosted endpoint, function ID, and API key through `ASRParams` (`audio_endpoints`, `function_id`, `auth_token`).
 
     ```python
+    from nemo_retriever import create_ingestor
     from nemo_retriever.params.models import ASRParams
 
     ingestor = (
-        Ingestor()
+        create_ingestor(run_mode="batch")
         .files("./data/*.mp3")
         .extract_audio(
             asr_params=ASRParams(
@@ -115,21 +116,22 @@ Instead of running the pipeline locally, you can call Parakeet through [build.nv
             ),
         )
     )
+    results = ingestor.ingest()
     ```
 
     !!! tip
 
-        For more Python examples, refer to [Python Quick Start Guide](https://github.com/NVIDIA/NeMo-Retriever/blob/main/client/client_examples/examples/python_client_usage.ipynb).
+        For more runnable examples, refer to [Workflow: Ingest documents](workflow-document-ingestion.md).
 
 ## Video and frame OCR { #video-and-frame-ocr }
 
-For video assets, NeMo Retriever Library can combine audio or speech processing (see [Speech and audio extraction](#speech-and-audio-extraction) above) with visual text extraction when OCR applies to frames or derived images.
+For video assets, NeMo Retriever Library can combine audio or speech processing (refer to [Speech and audio extraction](#speech-and-audio-extraction) above) with visual text extraction when OCR applies to frames or derived images.
 
-For OCR-oriented extract methods on scanned or image-heavy content, see [OCR and scanned documents](multimodal-extraction.md#ocr-and-scanned-documents), [text and layout extraction](multimodal-extraction.md#text-and-layout-extraction), and [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse) for advanced visual parsing.
+For OCR-oriented extract methods on scanned or image-heavy content, refer to [OCR and scanned documents](multimodal-extraction.md#ocr-and-scanned-documents), [text and layout extraction](multimodal-extraction.md#text-and-layout-extraction), and [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse) for advanced visual parsing.
 
-Container formats and early-access video types are listed under [supported file types and formats](multimodal-extraction.md#supported-file-types-and-formats) (see [What is NeMo Retriever Library?](overview.md) for the full list).
+Container formats and early-access video types are listed under [supported file types and formats](multimodal-extraction.md#supported-file-types-and-formats) (refer to [What is NeMo Retriever Library?](overview.md) for the full list).
 
-For end-to-end RAG stacks that include multimodal ingestion, see the [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) and related solution pages on [NVIDIA Build](https://build.nvidia.com/).
+For end-to-end RAG stacks that include multimodal ingestion, refer to the [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) and related solution pages on [NVIDIA Build](https://build.nvidia.com/).
 
 ## Related topics { #related-topics }
 

docs/docs/extraction/embedding.md

@@ -2,7 +2,7 @@
 
 !!! note "Text-only NeMo Retriever embedding NIM"
 
-    You can still use the NeMo Retriever text embedding NIM (OpenAI-compatible embeddings for passage and query vectors) alongside or instead of the multimodal flows on this page. Product and deployment details are in the [NeMo Retriever Text Embedding NIM documentation](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html). In library and CLI pipelines, route embedding to that NIM with your configured `embed` / invoke URL and model name (see the [graph pipeline examples](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/README.md) for environment-based remote inference).
+    You can still use the NeMo Retriever text embedding NIM (OpenAI-compatible embeddings for passage and query vectors) alongside or instead of the multimodal flows on this page. Product and deployment details are in the [NeMo Retriever Text Embedding NIM documentation](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html). In library and CLI pipelines, route embedding to that NIM with your configured embed endpoint and model name (refer to the [graph pipeline examples](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/README.md) for environment-based remote inference).
 
 This documentation describes how to use [NeMo Retriever Library](overview.md) 
 with the multimodal embedding model [Llama Nemotron Embed VL 1B v2](https://build.nvidia.com/nvidia/llama-nemotron-embed-vl-1b-v2).
@@ -12,6 +12,7 @@ The model can embed documents in the form of an image, text, or a combination of
 Documents can then be retrieved given a user query in text form. 
 The model supports images that contain text, tables, charts, and infographics.
 
+Parameter details for `.extract()` and `.embed()` appear in the [Python API guide](nemo-retriever-api-reference.md).
 
 ## Example with Default Text-Based Embedding
 
@@ -21,27 +22,31 @@ The following example provides a strong baseline for retrieval.
 - The `embed` method is called with no arguments.
 
 ```python
+from nemo_retriever import create_ingestor
+
 ingestor = (
-    Ingestor()
+    create_ingestor(run_mode="batch")
     .files("./data/*.pdf")
     .extract()
-    .embed() # Default behavior embeds all content as text
+    .embed()  # Default behavior embeds all content as text
 )
 results = ingestor.ingest()
 ```
 
 
 ## Example with Embedding Structured Elements as Text + Images
 
-It is common to process PDFs by embedding standard text as text, and embed visual elements like tables and charts as images. 
+It is common to process PDFs by embedding standard text as text and embed visual elements such as tables and charts as images. 
 The following example enables the multimodal model to capture the spatial and structural information of the visual content.
 
 - The `embed` method is configured with `embed_modality="text_image"` to embed the extracted tables and charts as images.
-- This configuration is more accurate than text only with a performance cost
+- This configuration is more accurate than text only, with a performance cost.
 
 ```python
+from nemo_retriever import create_ingestor
+
 ingestor = (
-    Ingestor()
+    create_ingestor(run_mode="batch")
     .files("./data/*.pdf")
     .extract()
     .embed(
@@ -58,16 +63,18 @@ For documents where the entire page layout is important (such as infographics, c
 you can configure NeMo Retriever Library to treat every page as a single image.
 The following example extracts and embeds each page as an image.
 
-- The `embed method` processes the page images.
+- The `embed` method processes the page images.
 
 ```python
+from nemo_retriever import create_ingestor
+
 ingestor = (
-    Ingestor()
+    create_ingestor(run_mode="batch")
     .files("./data/*.pdf")
     .extract()
     .embed(
         embed_modality="image",
-        embed_granularity="page"
+        embed_granularity="page",
     )
 )
 results = ingestor.ingest()
@@ -78,4 +85,3 @@ results = ingestor.ingest()
 - [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
 - [Troubleshoot Nemo Retriever Extraction](troubleshoot.md)
 - [Use the Python API](nemo-retriever-api-reference.md)
-- [Extract Captions from Images](nemo-retriever-api-reference.md)

docs/docs/extraction/nimclient.md

@@ -13,6 +13,8 @@ For advanced usage patterns, refer to the existing model interfaces in `nemo_ret
 
 ## Quick Start
 
+For ingest and pipeline APIs used with NimClient in UDFs, refer to the [Python API guide](nemo-retriever-api-reference.md).
+
 ### Basic NimClient Creation
 
 ```python

docs/docs/extraction/notebooks/index.md

@@ -8,18 +8,17 @@ If you plan to run benchmarking or evaluation tests, you must download the [Benc
 
 ## Getting Started
 
-To get started with the basics, try one of the following notebooks:
+To get started with the basics, try one of the following guides or notebooks:
 
-- [CLI Quick Start Guide](https://github.com/NVIDIA/NeMo-Retriever/blob/main/client/client_examples/examples/cli_client_usage.ipynb)
-- [Python Quick Start Guide](https://github.com/NVIDIA/NeMo-Retriever/blob/main/client/client_examples/examples/python_client_usage.ipynb)
-- [How to add metadata to your documents and filter searches](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/metadata_and_filtered_search.ipynb)
-- [How to reindex a collection](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/reindex_example.ipynb)
+- [Quickstart: retriever CLI](../../reference/retriever-cli-quickstart.md)
+- [Workflow: Ingest documents](../workflow-document-ingestion.md)
+- [How to add metadata to your documents and filter searches](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb)
 
 
 For more advanced scenarios, try one of the following notebooks:
 
 - [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb)
-- [Try Enterprise RAG Blueprint](https://github.com/NVIDIA/NeMo-Retriever/blob/main/deploy/pdf-blueprint.ipynb)
+- [Try Enterprise RAG Blueprint](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)
 - [Evaluate bo767 retrieval recall accuracy with NeMo Retriever Library](https://github.com/NVIDIA/NeMo-Retriever/blob/main/evaluation/bo767_recall.ipynb)
 - [Multimodal RAG with LangChain](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/langchain_multimodal_rag.ipynb)
 - [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/llama_index_multimodal_rag.ipynb)