fix: reject chunk_overlap >= chunk_size in knowledge sources instead of crashing/dropping content

BaseKnowledgeSource._chunk_text() (and 6 identical copies of it duplicated
across string/json/pdf/text_file/csv/excel KnowledgeSource subclasses)
compute the slicing step as chunk_size - chunk_overlap with no validation:

    range(0, len(text), chunk_size - chunk_overlap)

chunk_overlap == chunk_size -> step is 0 -> ValueError("range() arg 3 must
not be zero"), a confusing error unrelated to the actual misconfiguration.

chunk_overlap > chunk_size -> step is negative -> range(0, len(text), neg)
with start(0) < stop(len(text)) silently yields nothing -> _chunk_text
returns [] -> the source's entire content is dropped with no error, no
warning, nothing indexed.

Verified both independently with a real (non-mocked) StringKnowledgeSource
instance before this fix: chunk_overlap=chunk_size raises the confusing
range() ValueError; chunk_overlap>chunk_size raises nothing and silently
produces zero chunks.

Fix: add a single `@model_validator(mode="after")` on BaseKnowledgeSource
that rejects chunk_overlap >= chunk_size at construction time with a clear
message. Since every one of the 7 duplicated _chunk_text implementations
reads self.chunk_size/self.chunk_overlap from the same inherited Pydantic
fields, and Pydantic validators are inherited by subclasses, this one
change protects all 7 call sites without needing to touch each file's
duplicated _chunk_text individually. Confirmed via direct instantiation of
StringKnowledgeSource with the bad configs both now raise ValidationError
at construction time, before any chunking is ever attempted.

Added tests/knowledge/test_base_knowledge_source_chunk_overlap.py: equal
values rejected, overlap > size rejected, a valid overlap < size accepted,
and the library's own defaults (chunk_size=4000, chunk_overlap=200) pass
the new check. Confirmed red->green (reverting the source change reproduces
"DID NOT RAISE ValueError" for both bad-config tests). Full
tests/knowledge/ suite: 58 passed, no regressions. ruff + mypy clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: chuenchen309

lib/crewai/src/crewai/knowledge/source/base_knowledge_source.py

@@ -2,7 +2,8 @@
 from typing import Any
 
 import numpy as np
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+from typing_extensions import Self
 
 from crewai.knowledge.storage.base_knowledge_storage import BaseKnowledgeStorage
 from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage
@@ -28,6 +29,21 @@ class BaseKnowledgeSource(BaseModel, ABC):
     metadata: dict[str, Any] = Field(default_factory=dict)  # Currently unused
     collection_name: str | None = Field(default=None)
 
+    @model_validator(mode="after")
+    def _validate_chunk_overlap(self) -> Self:
+        """Reject a chunk_overlap that would break _chunk_text's slicing step.
+
+        step = chunk_size - chunk_overlap: zero raises ValueError("range() arg 3
+        must not be zero"); negative silently makes range() empty, dropping all
+        content with no error at all.
+        """
+        if self.chunk_overlap >= self.chunk_size:
+            raise ValueError(
+                f"chunk_overlap ({self.chunk_overlap}) must be smaller than "
+                f"chunk_size ({self.chunk_size})."
+            )
+        return self
+
     @abstractmethod
     def validate_content(self) -> Any:
         """Load and preprocess content from the source."""

lib/crewai/tests/knowledge/test_base_knowledge_source_chunk_overlap.py

@@ -0,0 +1,24 @@
+"""Test BaseKnowledgeSource rejects a chunk_overlap that would break _chunk_text's slicing step."""
+
+import pytest
+from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
+
+
+def test_chunk_overlap_equal_to_chunk_size_is_rejected():
+    with pytest.raises(ValueError, match="chunk_overlap"):
+        StringKnowledgeSource(content="hello world", chunk_size=50, chunk_overlap=50)
+
+
+def test_chunk_overlap_greater_than_chunk_size_is_rejected():
+    with pytest.raises(ValueError, match="chunk_overlap"):
+        StringKnowledgeSource(content="hello world", chunk_size=50, chunk_overlap=60)
+
+
+def test_chunk_overlap_smaller_than_chunk_size_is_accepted():
+    source = StringKnowledgeSource(content="hello world", chunk_size=50, chunk_overlap=10)
+    assert source.chunk_overlap == 10
+
+
+def test_default_chunk_size_and_overlap_are_valid():
+    source = StringKnowledgeSource(content="hello world")
+    assert source.chunk_overlap < source.chunk_size