You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
domain: Domain of the task (e.g., "customer service", "code review")
109
-
task_description: Detailed description of what the agent should do
110
-
constraints: Optional list of constraints or requirements
111
-
105
+
"""
106
+
Generate a dataset of synthetic test cases for a given domain and task.
107
+
108
+
Parameters:
109
+
domain (str): Domain name for the generated cases (e.g., "customer service").
110
+
task_description (str): Description of the task the cases should exercise.
111
+
constraints (list[str] | None): Optional list of additional constraints to apply when generating cases.
112
+
112
113
Returns:
113
-
DatasetSource with generated cases
114
+
DatasetSource: A dataset containing generated DatasetCase objects. Each case includes an `id`, `query`, optional `expected_output`, and `metadata` containing at least `difficulty`, `generated` (True), and `domain`.
115
+
116
+
Raises:
117
+
json.JSONDecodeError: If the model response cannot be parsed as JSON.
task_description: Detailed description of what the agent should do
246
-
constraints: Optional list of constraints
247
-
seed_examples: Optional seed examples to guide generation
248
-
245
+
"""
246
+
Generate a synthetic dataset of test cases for a given domain and task.
247
+
248
+
Constructs a chat prompt from the domain, task description, optional constraints, and optional seed examples, sends it to the configured OpenAI chat model, parses the model's JSON array response into DatasetCase objects, and returns a DatasetSource containing those cases.
249
+
250
+
Parameters:
251
+
domain (str): Domain or subject area for which to generate test cases.
252
+
task_description (str): Description of the task or problem the generated cases should exercise.
253
+
constraints (list[str] | None): Optional list of additional constraints to include in the prompt.
254
+
seed_examples (list[dict[str, Any]] | None): Optional list of example cases to guide generation; included verbatim in the prompt.
255
+
249
256
Returns:
250
-
DatasetSource with generated cases
257
+
DatasetSource: A dataset source containing generated DatasetCase entries with metadata (including difficulty, generated=True, and domain).
258
+
259
+
Raises:
260
+
json.JSONDecodeError: If the model's response cannot be parsed as a JSON array of cases.
251
261
"""
252
262
messages= [
253
263
{"role": "system", "content": "You are an expert test case generator."},
0 commit comments