-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathautogen04_writingassistant copy.py
More file actions
3237 lines (2840 loc) · 257 KB
/
Copy pathautogen04_writingassistant copy.py
File metadata and controls
3237 lines (2840 loc) · 257 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
'''
DESCRIPTION
Writing Assistant with AutoGen agents
This script creates an AutoGen-based writing assistant that helps with academic writing.
It uses Azure OpenAI models with interactive browser authentication.
Version: 0.1
Author: Tim Haintz
Creation Date: 20250426
LINKS:
https://microsoft.github.io/autogen/
https://www.microsoft.com/en-us/research/blog/autogen-enabling-next-generation-large-language-model-applications/
https://github.com/microsoft/autogen
https://microsoft.github.io/autogen/stable/
https://www.microsoft.com/en-us/research/blog/autogen-v0-4-reimagining-the-foundation-of-agentic-ai-for-scale-extensibility-and-robustness/
https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/
https://multiagentbook.com/labs/usecases/?usecase=deep-research-agents&trk=public_post_comment-text - High level structure
EXAMPLE USAGE:
python autogen04_writingassistant copy.py
'''
###########
# IMPORTS #
###########
import asyncio
import os
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient, OpenAIChatCompletionClient
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat, SelectorGroupChat
from autogen_agentchat.ui import Console
from autogen_agentchat.teams import MagenticOneGroupChat
from autogen_core.models import ModelFamily
from autogen_ext.agents.web_surfer import MultimodalWebSurfer
from azure.identity import InteractiveBrowserCredential, DefaultAzureCredential, get_bearer_token_provider
from dotenv import load_dotenv
# Import the azure_models module for model configuration
from azure_models import get_model_config, get_autogen_config
# Load environment variables from the .env file
load_dotenv()
#############
# VARIABLES #
#############
# Create token provider for Azure authentication
token_provider = get_bearer_token_provider(InteractiveBrowserCredential(), "https://cognitiveservices.azure.com/.default")
async def main() -> None:
# Get model configurations using the new module
gpt4o_config = get_autogen_config("gpt-4o")
gpt45preview_config = get_autogen_config("gpt-4.5-preview")
o1mini_config = get_autogen_config("o1-mini")
o3mini_config = get_autogen_config("o3-mini")
o4mini_config = get_autogen_config("o4-mini")
gpt41_config = get_autogen_config("gpt-4.1")
# Get DeepSeek model configurations
deepseek_r1_config = get_autogen_config("deepseek-r1")
deepseek_v3_config = get_autogen_config("deepseek-v3")
deepseek_v3_0324_config = get_autogen_config("deepseek-v3-0324")
# Create Azure OpenAI clients for AutoGen
az_model_client = AzureOpenAIChatCompletionClient(
azure_deployment=gpt4o_config["model"],
model="gpt-4o",
api_version=gpt4o_config["api_version"],
azure_endpoint=gpt4o_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=0.2,
)
az_model_client_gpt45preview = AzureOpenAIChatCompletionClient(
azure_deployment=gpt45preview_config["model"],
model="gpt-4.5-preview",
api_version=gpt45preview_config["api_version"],
azure_endpoint=gpt45preview_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=0.2,
)
az_model_client_o1_mini = AzureOpenAIChatCompletionClient(
azure_deployment=o1mini_config["model"],
model="o1-mini",
api_version=o1mini_config["api_version"],
azure_endpoint=o1mini_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=1.0,
)
az_model_client_o3_mini = AzureOpenAIChatCompletionClient(
azure_deployment=o3mini_config["model"],
model="o3-mini",
api_version=o3mini_config["api_version"],
azure_endpoint=o3mini_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=1.0,
)
az_model_client_o4_mini = AzureOpenAIChatCompletionClient(
azure_deployment=o4mini_config["model"],
model="o4-mini",
api_version=o4mini_config["api_version"],
azure_endpoint=o4mini_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=1.0,
)
# Create DeepSeek model clients using configurations from azure_models.py
az_model_client_R1 = OpenAIChatCompletionClient(
model="deepseek-r1",
base_url=deepseek_r1_config["base_url"],
api_key=deepseek_r1_config["api_key"],
model_info=deepseek_r1_config["model_info"]
)
az_model_client_V3 = OpenAIChatCompletionClient(
model="DeepSeek-V3",
base_url=deepseek_v3_config["base_url"],
api_key=deepseek_v3_config["api_key"],
model_info=deepseek_v3_config["model_info"]
)
az_model_client_V3_0324 = OpenAIChatCompletionClient(
model="DeepSeek-V3-0324",
base_url=deepseek_v3_0324_config["base_url"],
api_key=deepseek_v3_0324_config["api_key"],
model_info=deepseek_v3_0324_config["model_info"]
)
# Add client for the 4.1 model
az_model_client_gpt41 = AzureOpenAIChatCompletionClient(
azure_deployment=gpt41_config["model"],
model="gpt-4.1",
api_version=gpt41_config["api_version"],
azure_endpoint=gpt41_config["azure_endpoint"],
azure_ad_token_provider=token_provider,
temperature=0.2,
)
# Create the Writing Assistant agent
academic_writing_assistant = AssistantAgent(
name="academic_writing_assistant",
description="ALWAYS USE FIRST! An expert writing assistant that provides feedback on writing clarity, coherence, and quality.",
model_client=az_model_client_o4_mini,
system_message='''
You are an Expert Writing Assistant, an expert in gathering information for enhancing the clarity, coherence, and quality of specific sentences and paragraphs in academic manuscripts.
Your role is to provide detailed and constructive feedback, as well as suggestions for improvement. You focus on the following aspects:
- Clarity: Ensure each sentence and paragraph is clear and easy to understand. Simplify complex sentences and eliminate ambiguity.
- Coherence: Check for logical flow and coherence within and between sentences and paragraphs. Ensure smooth transitions and consistent ideas.
- Conciseness: Eliminate unnecessary words and redundancy. Ensure each sentence is concise and to the point.
- Grammar and Syntax: Correct grammatical errors, punctuation mistakes, and awkward phrasing. Ensure proper sentence structure and syntax.
- Academic Tone: Ensure the writing maintains an appropriate academic tone. Suggest improvements for formality and precision.
- Specific Feedback: Provide specific, actionable feedback on how to improve each sentence or paragraph. Offer alternative phrasings or rewordings where necessary.
Glossary:
PE: Prompt Example - Example of a prompt that the AI can respond to
PP: Prompt Pattern - A pattern that can be used to generate a PE
Your feedback should be detailed, constructive, and aimed at helping the authors improve their writing. Always maintain a respectful and professional tone.
Respond with ASSISTANT DONE when you are done.
'''
)
academic_writing_critic = AssistantAgent(
name="academic_writing_critic",
description="An expert writing critic that evaluates the effectiveness and relevance of the feedback provided by the academic writing assistant.",
model_client=az_model_client_o4_mini,
system_message='''
You are an expert writing critic, an expert in evaluating the work of an academic writing assistant.
Your role is to provide detailed and constructive feedback on the suggestions and improvements made by the Writing Assistant.
You focus on the following aspects:
- Effectiveness of Suggestions: Assess whether the Writing Assistant's suggestions improve the clarity, coherence, and quality of the sentences and paragraphs.
- Accuracy: Ensure the Writing Assistant's corrections are grammatically accurate and syntactically correct.
- Relevance: Evaluate the relevance of the Writing Assistant's suggestions to the academic context. Ensure the suggestions maintain the appropriate academic tone and style.
- Clarity and Conciseness: Check if the Writing Assistant's suggestions make the writing clearer and more concise without losing essential information.
- Constructiveness: Assess the constructiveness of the Writing Assistant's feedback. Ensure the feedback is specific, actionable, and aimed at helping the authors improve their writing.
- Overall Improvement: Evaluate the overall improvement in the writing after incorporating the Writing Assistant's suggestions. Ensure the final output is of high academic quality.
Your feedback should be detailed, constructive, and aimed at helping the Writing Assistant refine their suggestions and improve their effectiveness. Always maintain a respectful and professional tone.
Respond with CRITIQUE COMPLETE when you are done.
'''
)
academic_writer = AssistantAgent(
name="academic_writer",
description="An writer that writes and rewrites sentences and paragraphs based on feedback and evaluation.",
model_client=az_model_client_o4_mini,
system_message='''
You are a Senior PhD writer.
Your role is to rewrite the original sentence(s) and paragraph(s) based on the academic_writing_assistant's feedback and academic_writing_critic's evaluation.
Please write very concisely in Australian English.
Respond with WRITER DONE when you are done.
Your report should end with the word "TERMINATE" to signal the end of the conversation.
'''
)
# Create the Verifier agent
academic_reviewer = AssistantAgent(
name="academic_reviewer",
description="An expert review specialist who ensures research quality and completeness",
model_client=az_model_client_o4_mini,
system_message='''
You are an Expert Reviewer, an expert in critically evaluating academic manuscripts. Your role is to provide constructive feedback on the quality, validity, and significance of the research.
You focus on the following aspects:
- Clarity and Structure: Ensure the manuscript is well-organized, with a clear introduction, methodology, results, and conclusion. Check for logical flow and coherence.
- Originality and Significance: Assess the novelty of the research and its contribution to the field. Determine if the study addresses a significant problem or gap in the literature.
- Methodology: Evaluate the appropriateness and rigor of the research methods used. Ensure the study design, data collection, and analysis are sound and reproducible.
- Literature Review: Check if the manuscript provides a comprehensive and up-to-date review of relevant literature. Ensure proper citation of sources.
- Results and Interpretation: Verify the accuracy and relevance of the results. Assess if the interpretations and conclusions are supported by the data.
- Language and Style: Ensure the manuscript is written in clear, concise, and grammatically correct language. Suggest improvements for readability and academic tone.
Your feedback should be detailed, constructive, and aimed at helping the authors improve their work. Always maintain a respectful and professional tone.
Provide specific examples and suggestions for improvement. Your goal is to help the authors enhance the quality and impact of their research.
Respond with REVIEW COMPLETE when you are done.
'''
)
web_surfer = MultimodalWebSurfer(
name="WebSurfer",
description="A web surfer agent that performs web searches to find relevant information",
model_client=az_model_client_gpt41,
)
# Set up termination conditions
text_termination = TextMentionTermination("TERMINATE")
max_messages = MaxMessageTermination(max_messages=20)
termination = text_termination | max_messages
# Create the selector prompt
selector_prompt = '''You are coordinating a team of academic.
The following team member roles are available: {roles}.
The academic_writing_assistant provides detailed feedback on writing clarity, coherence, and quality.
The academic_writing_critic evaluates the effectiveness and relevance of the feedback provided by the writing assistant.
The academic_writer rewrites the sentences and paragraphs based on the feedback and evaluation.
Given the current context, select the most appropriate next speaker.
Base your selection on:
1. *ALWAYS SEND TO THE* academic_writing_assistant *FIRST!*
2. Progress toward improving the writing quality
3. Completeness and readiness for rewriting.
Read the following conversation. Then select the next role from {participants} to play. Only return the role.
{history}
Read the above conversation. Then select the next role from {participants} to play. ONLY RETURN THE ROLE.
'''
# Create the team
team = SelectorGroupChat(
participants=[academic_reviewer, academic_writer, academic_writing_assistant, academic_writing_critic, web_surfer],
model_client=az_model_client_gpt41,
termination_condition=termination,
selector_prompt=selector_prompt,
allow_repeated_speaker=True
)
# # Create a sequential team using RoundRobinGroupChat by having the same number of max turns as there are agents
# team = RoundRobinGroupChat(
# participants=[academic_writing_assistant, academic_writing_critic, academic_writer],
# max_turns=3,
# termination_condition=termination,
# )
# Used for testing the MagenticOneGroupChat
magentic_one_team = MagenticOneGroupChat(
participants=[academic_reviewer, academic_writer, academic_writing_assistant, academic_writing_critic, web_surfer],
model_client=az_model_client,
termination_condition=termination,
)
### CAN BE USED FOR THE TASK SECTION BELOW ###
# Please provide feedback on the clarity, coherence, and quality of the REVIEW section.
task = r'''
### TASK ###
- This is my research paper on Prompt Engineering.
- Your task is to Complete the Conclusion and Future Work section, ensuring it matches the paper's style and structure.
- Please send to the academic_writing_assistant first.
- The writing *MUST* be in LaTex format.
- The writing *MUST* be in Australian English.
- The writing *MUST* be in the style of a PhD student.
- The academic_writing_assistant *MUST* gather the information first and send it to the academic_writer to write the first draft.
- The academic_writer *MUST* write the first draft and send it to the academic_writing_critic for review.
- The academic_writing_critic *MUST* review the writing and send it to the academic_reviewer for review.
- The academic_reviewer *MUST* send it to the academic_writer for rewriting.
- Confirm that each agent has completed their task by sending the message "ASSISTANT DONE", "CRITIQUE COMPLETE", "REVIEW COMPLETE" and "WRITER DONE" respectively.
- The academic_writing_critic should do a final review of the writing.
- The academic_reviewer *MUST* review writing *AT LEAST* TWICE.
- The academic_writer *MUST* do the final rewrite.
### BEGIN RESEARCH PAPER ###
\documentclass[12pt,letterpaper,oneside]{article} % Switched to 'article' class
\usepackage[margin=1in]{geometry} % to adjust the page margins if needed
\usepackage{setspace} % Include the setspace package
\usepackage{blindtext}
\usepackage{hyperref}
\usepackage{graphicx}
\graphicspath{./Images/}
\usepackage[pdf]{graphviz}
\usepackage[english]{babel}
\usepackage[square,numbers]{natbib}
\usepackage{xcolor}
\usepackage{listings}
\usepackage{lscape}
\usepackage{longtable}
\usepackage{array}
\usepackage{multirow}
\usepackage{forest}
\usepackage{amsmath,amssymb}
\usepackage{array}
\usepackage{tabularx}
\usepackage{threeparttable}
\usepackage{bbding}
\bibliographystyle{abbrvnat}
\title{The Way to Talk to AI: A Taxonomy of Prompt Patterns to LLMs}
\author{
Tim Haintz\thanks{tjhaintz@students.federation.edu.au} \and
Paul Pang\thanks{p.pang@federation.edu.au}
}
\date{July 2023}
\begin{document}
\doublespacing % This command sets double line spacing
\maketitle
%\tableofcontents
\begin{abstract}
As the interface between humans and Large Language Models (LLMs) continues to advance, structuring and optimising communication has become crucial. This paper introduces a comprehensive taxonomy of Prompt Patterns (PPs) for LLMs, serving as a refined linguistic framework that enhances interaction efficacy. We categorise these PPs based on the logic of English prepositions into six distinct types—Across, At, Beyond, In, Out, Over—each tailored to specific communication objectives from detailed inquiries to boundary-pushing commands. The taxonomy is meticulously crafted, derived from extensive linguistic analysis and illustrated through numerous examples across various domains. Hosted at \href{https://github.com/timhaintz/PromptEngineering4Cybersecurity/blob/main/promptpatterns.json}{GitHub}, our taxonomy compiles 120 PPs and over 300 practical examples, curated from 150 scholarly references. This structured approach not only aids in crafting precise and contextual prompts but also enhances the AI’s response accuracy, significantly improving the quality of human-AI interaction. By establishing a systematic framework for prompt design, our taxonomy is poised to be an indispensable resource, promoting more intuitive and effective communications with AI systems.
\end{abstract}
\textbf{Keywords:} Large Language Model, Taxonomy, Prompt Engineering, Prompt Pattern, Prompt Example, ChatGPT, Prepositional Logic, Generative AI
\section{Introduction}
%backgound: LLM -->chatgpt functions--> applied chatgpt, --> prompt engineering, --> obstacles on prompt to LLM (random patterns are there)
Prompt Patterns (PPs) are essential to effective prompt engineering. A key contribution of this paper is the introduction of PPs to document successful approaches for systematically engineering different output and interaction goals when working with conversational LLMs. We focus largely on engineering domain-independent PPs and introduce a catalogue of essential PPs to solve problems ranging from production of visualisations and code artifacts to automation of output steps for fact checking.
Large Language Models (LLMs) (\cite{Zhang2022OPT:Models}, \cite{Chowdhery2022PaLM:Pathways}, \cite{Touvron2023LLaMA:Models}, \cite{GeminiTeam2023Gemini:Models}, \cite{Brown2020LanguageLearners}, \cite{OpenAI2023GPT-4Report}) have shown their capability and usefulness across a wide range of applications. An LLM is a type of language model that uses neural networks with billions of parameters \cite{FanA2023}. The same Language Model can be used for multiple purposes, without the need for additional training or supervision \cite{Liu2023Pre-trainProcessing}. The adoption of LLMs in diverse fields makes them a very useful addition in many industries. Hariri states ChatGPT has been applied in various real-world scenarios, including healthcare, education, customer service, content creation, language translation, entertainment, financial services, atmospheric science, chatbots, and computer science and coding \cite{Hariri2023UnlockingProcessing} .
The method of working with LLMs has become known as \textit{Prompt Engineering}. Prompt Engineering is the interface to interact via natural language with LLMs. The effectiveness of the LLMs largely depends on prompt engineering. \textit{Prompt Engineering} involves crafting effective prompts that guide the model to produce desired responses. The quality of the prompt significantly influences the output, making \textit{Prompt Engineering} an essential skill in the field of AI. It is used to assist in obtaining correct and accurate results \cite{White2023AChatGPT}. As the LLM scales, the downstream use of Prompt Engineering for Natural Language Processing (NLP) is critical \cite{Wei2022EmergentModels}. Prompting enables domain experts to solve tasks using natural language, but task accuracy varies significantly with prompt choices, often requiring extensive trial and error to find the best fit \cite{Strobelt2023InteractiveModels}.
Recent updates to the ChatGPT models, GPT-35-Turbo and GPT-4, have introduced a conversational interface in which to interact and utilise \textit{Prompt Engineering}. The Application Programming Interface (API) for these new models introduced a \textit{System role}, \textit{Assistant role} and \textit{User role}. Leveraging these roles allows for the use of \textit{Prompt Engineering}, zero-shot and few-shot prompting. Zero-shot prompting uses the system and user role. In few-shot prompting, all three available roles - system, assistant, and user - are utilised.
Despite these advancements, there are several obstacles to perfecting these models. These include challenges related to ensuring the model’s understanding of nuanced human language, ethical concerns about misuse, and the response being made up or fabricated \cite{Shen2023ChatGPTSwords,Hariri2023UnlockingProcessing}.
% prompt and pp definition
According to White et al. \cite{White2023AChatGPT}, prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customise the outputs and interactions with an LLM. Giray et al. \cite{Giray2023PromptWriters} describes a prompt as an instruction or query given to a language model to guide its behavior and generate desired outputs, consisting of elements such as instruction, context, input data, and output indicator.
%prompt pattern
A Prompt Pattern (PP) refers to a general structure or format used to guide a language model's response, often without specific content. It serves as a template that can be filled with various inputs to achieve different tasks. For instance, the 'Simple Colon' PP in White et al. \cite{White2023AChatGPT} uses the format 'French: source-phrase English:', which can be applied to any French phrase needing translation. It often includes placeholders or cues that indicate where specific information should be inserted. For instance, a PP might include phrases like 'what may happen', 'will ...?', or 'why might', which are used to construct questions that require a certain type of response. On the other hand, a prompt example (PE) is a specific instance or illustration of a prompt that follows a particular pattern or structure. It is a concrete example that demonstrates how the pattern can be applied in practice. For example, given the PP 'Use what may happen, will ...?, why might, etc.', a prompt example could be 'What may happen if my shoes never show up?' or 'Why might GPS technology have been invented?'. While a PP provides a general framework, a prompt example shows a specific application of that framework. \cite{Mishra2021ReframingLanguage}
\section{Existing Prompt Engineering Surveys}
% We evaluate existing survey from several dimensions. Dimensions keywords for the table are: Survey, Collection of PP, Collection of PE, Universal, Field and Year. Top 5 years and 10 articles.
% Must be a survey or collections paper. 2020-2023
% Write out the below criteria. Introduce the table.
% 1. Survey - Cited name of paper
% 2. Collection of PP (Tick/Cross)
% 3. Collection of PE (Tick/Cross)
% 4. Universal (a. Not domain specific, b. Between AI and Human) - Scientific sentence, don't repeat. Consice
% 5. Field (Cybersecurity, Health - put in the actual value)
% 6. Year - (Newest first)
% 7. Completeness - ADD NEW FIELD. It may be UNIVERSAL but is is COMPLETE - Used to measure previous work.
We evaluate existing surveys and collections from several dimensions, as outlined in Table \ref{tab:relatedsurvey}. The dimensions include the presence or lack of PPs and/or PEs; if the work is domain-specific or applicable universally between AI and human; if not universal, the specific field the work applies to (e.g., cybersecurity, health etc.); and the publishing date of the work.
% Explain that the logic is complete so it can be reused. Connectivity. One connects to the other Reasoning and exploration plus dive into logic. Human behaviour allows things to be discovered using other methods. Derive from this to find complete.
\begin{table}[h]
%\begin{threeparttable}
\fontsize{9pt}{10pt}\selectfont
\caption{Related surveys on Prompt Patterns (PP) and Prompt Examples (PE)}
\begin{tabular}{|lcccccc|}
%\toprule
\hline
\textbf{Survey} & \textbf{Collection of PP} & \textbf{Collection of PE} & \textbf{Universal} & \textbf{Field} & \textbf{Complete} & \textbf{Year}\\ \hline
Sahoo et al. \cite{Sahoo2024AApplications} & \checkmark & $\times$ & \checkmark & - & $\varnothing$ & 02/2024 \\
Deng et al. \cite{Deng2023Jailbreaker:Chatbots} & $\times$ & \checkmark & $\times$ & Jailbreaking & $\varnothing$ & 07/2023 \\
Schmidt et al. \cite{SchmidtCatalogingEngineering} & \checkmark & \checkmark & \checkmark & - & $\varnothing$ & 06/2023 \\
Wang et al. \cite{Wang2023PromptApplications} & $\times$ & \checkmark & $\times$ & Healthcare & $\varnothing$ & 04/2023 \\
White et al. \cite{White2023ChatGPTDesign} & \checkmark & \checkmark & $\times$ & Code and Software & $\varnothing$ & 03/2023 \\
Bubeck et al. \cite{Bubeck2023SparksGPT-4} & $\times$ & \checkmark & \checkmark & - & $\varnothing$ & 03/2023 \\
White et al. \cite{White2023AChatGPT} & \checkmark & \checkmark & \checkmark & - & $\varnothing$ & 02/2023 \\
Arora et al. \cite{Arora2022AskModels} & $\times$ & \checkmark & $\times$ & Jailbreaking & $\varnothing$ & 10/2022 \\
Honovich et al. \cite{Honovich2022InstructionDescriptions} & $\times$ & \checkmark & \checkmark & - & $\varnothing$ & 05/2022 \\
Wang et al. \cite{Wang2022Self-ConsistencyModels} & $\times$ & \checkmark & \checkmark & - & $\varnothing$ & 03/2022 \\
Mishra et al. \cite{Mishra2021ReframingLanguage} & \checkmark & \checkmark & \checkmark & - & $\varnothing$ & 09/2021 \\ \hline
%\bottomrule
\end{tabular}
\vspace{0.5cm} % New line/carriage return for readability
%\end{threeparttable}
\begin{tablenotes}
\small\vspace{-1.5ex}
\item (\checkmark): Supported/Complete; ($\times$): Not Supported; Not Applicable: (-); Not Complete: ($\varnothing$)
\end{tablenotes}
\label{tab:relatedsurvey}
\end{table}
% 1. Introduce previous work to build such a prompt collection for different purpose
% a. Shining points from top 3 papers and discuss.
White et al. from Vanderbilt University presents a foundational framework for prompt engineering with large language models (LLMs) \cite{White2023AChatGPT}. They introduce 16 PPs categorised into 6 groups: Input Semantics, Output Customisation, Error Identification, Prompt Improvement, Interaction, and Context Control. Each PP is detailed with its intent, context, motivation, structure, key ideas, example implementation, and consequences. The paper underscores the importance of good prompt design, the evolution of LLM capabilities, and the generalisability of PPs across different domains. They provide a structured approach to documenting patterns for structuring prompts, adaptable to various domains, and illustrate how a PP can be built from multiple patterns to enhance LLM interactions. However, the work is merely a small collection of 16 PPs, which is far away to being universal between AI and human. \\
\par % New paragraph
Schmidt et al. addresses the need for a systematic approach to interacting with LLMs. The idea is to set up PPs in a structured way to boost and simplify human AI interactions, much like software patterns in standard software engineering \cite{SchmidtCatalogingEngineering}. This is a pioneering effort to catalogue PPs, providing reusable solutions to common problems encountered when using LLMs. By offering a structured approach, the research suggests repeatable, and effective use of LLMs across various domains, including software engineering, but not yet universal, which requires an expanded catalogue of PPs and their applications. \\
\par % New paragraph
Deng et al. introduce JAILBREAKER, a framework designed to understand and circumvent the defenses of LLM chatbots against jailbreak attacks, which prompt chatbots to generate responses that violate service guidelines \cite{Deng2023Jailbreaker:Chatbots}. The authors highlight the ineffectiveness of current strategies against mainstream LLM chatbots due to undisclosed defensive measures. JAILBREAKER’s first contribution is a novel method using time-based characteristics of the generation process to infer internal defense designs, inspired by time-based SQL injection techniques. This allows researchers to deconstruct defense mechanisms in services like CHATGPT, Bard, and Bing Chat (now Copilot). The second contribution is an innovative approach for the automatic generation of jailbreak prompts by fine-tuning an LLM, demonstrating higher success rates in generating attack prompts. The research focuses solely on jailbreaking and does not discuss universal usage across different domains. \\
\par % New paragraph
Sahoo et al. provide a comprehensive survey of prompt engineering techniques for large language models (LLMs) and vision-language models (VLMs) \cite{Sahoo2024AApplications}. They categorise recent advancements in prompt engineering by application area, offering a structured overview of various prompting methodologies, their applications, the models involved, and the datasets utilised. Key contributions include a detailed taxonomy diagram and a table summarising datasets, models, and critical points of each prompting technique, which facilitate a better understanding of this rapidly developing field. However, this work does not include any prompt examples, which limits its practical applicability for those looking to implement the discussed techniques directly. \\
% 2. Necessity of Building the Taxonomy: Addressing Universal, Complete, and Open Access
% One paragraph
From the view point of human to AI, a taxonomy that is universal, complete, and open access is in high demand, as this will set the standard for clear, efficient, and effective communication across various contexts and applications. With such a taxonomy, users can ensure that AI understands and executes tasks accurately, fostering a more intuitive and productive interaction. Moreover, an open-access taxonomy promotes widespread adoption and adaptation, allowing continuous improvement and updates based on user feedback and evolving use cases. This openness and universality are key to harnessing the full potential of AI, making it a versatile tool adaptable to the specific needs and nuances of different sectors and disciplines. \\
This paper advances prompt engineering by introducing a comprehensive, open catalogue of PPs, expanding on existing research, and providing a valuable resource for the community. The \href{https://github.com/timhaintz/PromptEngineering4Cybersecurity/blob/main/promptpatterns.json}{GitHub} repository has the PPs and PEs available as a JSON database that can be used to programmatically combine PPs and PEs to form useful domain agnostic prompts. \\
% 3. Contribution of This Paper
The contribution of this paper is summarised as:
\begin{enumerate}
\item We leveraged English language logic to develop a comprehensive catalogue to organise PPs and PEs in the context of human and AI communication.
\item We constructed a Taxonomy, which includes complete logic which consists of NNNN PPs and NNNN PEs, sourced from NNN papers and forums.
\item We develop strategies to combine and apply PPs and PEs to complicated problem from different domains.
\end{enumerate}
%To write: S2: overview, setup the taxanomy (TX), PP, PE, definition and terms. S3: categorisation in Human to AI S4-S10 6 logics, S11: Application (how to use the TX), S12 Limit and gaps, S13: Conclusion and future work
The rest of this paper is organised as follows: Section \ref{sec:overview} defines the structure of a PP, the LLMs used for testing PPs, the measurements of PP effectiveness, and how to perform tests via API. Section \ref{sec:categorisation} categorises PPs based on the logic of the English language, introducing categories such as Across, At, Beyond, In, Out, and Over logic, each with specific examples and applications. Sections \ref{sec:across} - \ref{sec:over} present the taxonomy for the prepositional logic of Across, At, Beyond, In, Out, and Over, respectively. We introduce the scope of its logic and categorises various PPs and prompt examples. Each category includes an example PP and a list of related PPs and PEs, along with advice on their reuse. Section \ref{sec:Application} discusses the practical application of the taxonomy, illustrating its utility in various contexts. It specifically investigates how to combine multiple PPs across categories and logic for complex real-world applications. Finally, in Section \ref{sec:conclusion}, this paper concludes with the presentation of ideas for future work.
\section{Overview}
\label{sec:overview}
\subsection{Structure of a PP}
% criteria % cite
To enhance the performance and quality of a language model’s output, it is crucial to follow these five key strategies:
\begin{itemize}
\item Provide clear context: This allows the model to answer with precise understanding and tailored responses, optimising the relevance and accuracy of the outcome.
\item State the desired output: This helps the model understand the specific information or response it needs to generate.
\item Break down complex questions into Sub-Questions: This helps the model focus on individual aspects of the topic and generate more accurate and detailed responses.
\item Provide Specific Instructions: This ensures that the model understands any constraints or requirements in generating the output.
\item Define conciseness: Prompt the model to generate concise and relevant responses by specifying any word limits or constraints. This helps prevent the model from generating unnecessarily lengthy or irrelevant answers.
\end{itemize}
By following the above five criteria, the language model can better understand and interpret the input, leading to improved performance and output quality.
White et. al \cite{White2023AChatGPT} recorded PPs in a data structure as:
\begingroup
\renewcommand{\arraystretch}{0.6}
\begin{center}
\fontsize{9pt}{10pt}\selectfont
\noindent
\begin{tabular}{|l|}
\hline
\textbf{Prompt Pattern} \\ \hline
A name and classification\\
The intent\\
The motivation\\
The structure and participants\\
Example code\\
Consequences\\\hline
\end{tabular}
\end{center}
\endgroup
The structure for PPs is generally well-organised. The primary issue is the overlap between ‘intent’ and ‘motivation’. While these terms can be distinct, in many contexts they could be interpreted as conveying similar information, leading to redundancy. The structure does not align perfectly with the five criteria discussed above. The pattern structure lacks explicit mention of breaking down complex questions into sub-questions and providing specific instructions, which are crucial for enhancing the model’s understanding and performance.
To ensure a more systematic and comprehensive approach, ultimately leading to more accurate and high-quality outputs from the language model, we have refined the structure to more closely align with the above five criteria as:
\begingroup
\renewcommand{\arraystretch}{0.6}
\begin{center}
\fontsize{9pt}{10pt}\selectfont
\noindent
\begin{tabular}{|l|}
\hline
\textbf{Prompt Pattern (PP)}\\ \hline
ID \\
Category \\
Name \\
Media Type: [Text Only, Text2Audio, Text2Image, Text2Video, Audio2Text, Image2Text, Video2Text]\\
Description: \\
Template: [Role, Context, Action, Format, Response]\\
Prompt Example (PE)\\
Related PPs \\
Reference \\ \hline
\end{tabular}
\end{center}
\endgroup
PE is recorded as a three field structure: ID, Prompt and its Response. ID is structured as
\textit{Reference Id + Index in Reference + PP id + the example id}. For example:
1-2-3-4, the PE is sourced from the 2nd reference paper, and it is the 5th prompt example, associated to the 4th PP in the 3 group in the reference. Note zero indexed referencing is applied here.
% Template: [Role, Context, Action, Format, Response]
We define a template for PPs as [Role, Context, Action, Format, Response]. This structure is crucial for creating effective and precise interactions with LLMs. Ideally, each prompt includes all these components to ensure clarity and completeness. The Role sets the perspective or persona, Context provides the background information, Action specifies the task, Format dictates the desired structure of the response, and Response anticipates the type of output expected. While some components can be omitted without rendering the prompt unusable, their absence can impact the accuracy and relevance of the AI's response. By adhering to this comprehensive template, users can maximise the efficiency and reliability of their interactions with LLMs. In the remaining paper, the original PPs and prompt elements are introduced, which may not strictly follow this template. However, they can all be logically mapped to these components—complete or in part.
\subsection{Prompt Test and Verification}
In this section, we introduce how a PP is tested to ensure that the interactions between users and LLMs are both effective and reliable.
% a list of LLMs
The models and versions used for our PPs testing were all built in Azure OpenAI, which includes
\begin{table}
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{Azure OpenAI Models used}
\begin{tabular}{|c|c|}
\hline
\textbf{Model name}& \textbf{Model version}\\ \hline
gpt-35-turbo & 0301\\ \hline
gpt-35-turbo-16k & 0613\\ \hline
gpt-4-32k & 0613\\ \hline
gpt-4 & 1106-Preview\\ \hline
gpt-4 & vision-preview\\ \hline
gpt-4.1 & 2025-04-14 \\ \hline
gpt-4.5-preview & 2025-02-27 \\ \hline
gpt-4o & 2024-05-13\\ \hline
o1-mini & 2024-09-12\\ \hline
o3-mini & 2025-01-31\\ \hline
o4-mini & 2025-04-16\\ \hline
Embeddings & text-embedding-ada-002\\ \hline
Embeddings & text-embedding-3-large\\ \hline
DeepSeek & R1\\ \hline
\end{tabular}
\label{tab:Azure OpenAI Models}
\end{table}
% Measurements - accuracy, relevance and consistency
The primary measurements of prompt verification include: Accuracy: Ensuring the information provided by the LLM in response to prompts is correct and reliable; Relevance: Verifying that responses are pertinent to the prompts, addressing the user's intent accurately; Consistency: Ensuring that the LLM provides consistent responses to the same or similar prompts across different instances; and Appropriateness: Confirming that the responses adhere to ethical guidelines and do not propagate biases or harmful content.
% Interface: Query and Programming
In practice, testing a PP involves feeding a set of prompt examples to LLMs. This can be done by employing automated scripts and/or manual queries. For manual queries, prompts are directly inputted into the LLM's API using the System, Assistant and user prompt. The hands-on approach allows for rapid testing and immediate feedback with nuanced understanding of prompt effectiveness. For programming, two scripts are utilised to automate the testing process. These scripts, available on GitHub at \href{https://github.com/timhaintz/PromptEngineering4Cybersecurity/blob/main/testPrompts.py}{testPrompts} and \href{https://github.com/timhaintz/PromptEngineering4Cybersecurity/blob/main/vision_testPrompts.py}{visionTestPrompts}respectively, cater to text-based and vision-related prompt examples, streamlining the submission of prompts to the LLM and the collection of its responses for analysis.
\section{Prompt Categorisation}
\label{sec:categorisation}
% Logical synthesis of existing patterns.
% Develop new patterns.
In the realm of communication between humans and Large Language Models (LLMs) like AI, PPs emerge as a vital linguistic bridge. These patterns are essential as they encapsulate the logic and structure inherent in human conversation, which is the ultimate goal of human-AI interaction. English, with its intricate system of grammar and rich array of prepositions, encapsulates the complete logic necessary for effective communication between humans. This comprehensive logic is essential in guiding AI to respond to a wide range of instructions and scenarios, mirroring human conversational abilities. To develop the complete logic of human-AI language, we learn from English by simply looking at the prepositions to ransack all aspect logic, in terms of topic.
\begin{itemize}
\item \textbf{Across} logic is used to signify one topic from the other. This could represent prompts that span \textbf{multiple domains} or disciplines, integrating diverse types of knowledge.
\item \textbf{At} logic is used to refer to a more specific aspect or detail of the topic. This might refer to prompts that are \textbf{specific} to a certain context or scenario, targeting precise responses.
\item \textbf{Beyond} logic is used to discuss aspects that are on the far side of a certain point or limit of a topic. This could indicate prompts that push the boundaries of what the AI can do, exploring \textbf{new capabilities} or \textbf{innovative ideas}.
\item \textbf{In} logic is used to indicate that something is contained within a topic or space. This could represent prompts that are \textbf{internal} to a system, focusing on self-reflection or introspection.
\item \textbf{Out} logic is employed to convey the idea of expanding upon or moving beyond the general scope of a topic. This might be used for prompts that generate \textbf{outputs}, such as creative writing or code generation.
\item \textbf{Over} logic is used to describe elements that span the entirety of the topic, which implies comprehensive coverage. This could be associated with prompts that require \textbf{oversight} or review, such as editing or improving existing content.
\end{itemize}
Figure \ref{fig:prepositions} illustrates the comprehensive PP categorisation in relation to the Logic of English language.
\begin{figure}
\centering
\caption{Categorisation of PPs in relation to the logic of English language centered on a core topic in between human and AI.}
\label{fig:prepositions}
\includegraphics[width=0.5\linewidth]{Images/Prepositions.png}
\end{figure}
\begin{table}
\centering
\caption{The structure of PP Taxonomy}
\fontsize{10pt}{8pt}\selectfont
\begin{forest}
for tree={
grow'=0,
child anchor=west,
parent anchor=south,
anchor=west,
calign=first,
edge path={
\noexpand\path [draw, \forestoption{edge}]
(!u.south west) + (5pt,0) |- (.child anchor)\forestoption{edge label};
},
before typesetting nodes={
if n=1
{insert before={[,phantom]}}
{}
},
fit=band,
before computing xy={l=10mm},
}
[PP Taxonomy
[Across Logic \ref{sec:across}
[Argument \ref{subsec:Argument}]
[Comparison \ref{subsec:Comparison}]
[Contradiction \ref{subsec:Contradiction}]
[Cross Boundary \ref{subsec:CrossBoundary}]
[Translation \ref{subsec:Translation}]
]
[At Logic \ref{sec:at}
[Assessment \ref{subsec:Assessment}]
[Calculation \ref{subsec:Calculation}]
[Induction]
]
[Beyond Logic \ref{sec:beyond}
[Hypothesise \ref{subsec:Hypothesise}]
[Logical Reasoning \ref{subsec:LogicalReasoning}]
[Prediction \ref{subsec:Prediction}]
[Simulation \ref{subsec:Simulation}]
]
[In Logic \ref{sec:in}
[Categorising \ref{subsec:categorising}]
[Classification \ref{subsec:classification}]
[Clustering \ref{subsec:clustering}]
[Error Identification \ref{subsec:ErrorIdentification}]
[Input Semantics \ref{subsec:InputSemantics}]
[Requirements Elicitation \ref{subsec:RequirementsElicitation}]
]
[Out Logic \ref{sec:out}
[Context Control \ref{subsec:ContextControl}]
[Decomposed Prompting \ref{subsec:DecomposedPrompting}]
[Output Customisation \ref{subsec:OutputCustomisation}]
[Output Semantics \ref{subsec:OutputSemantics}]
[Prompt Improvement \ref{subsec:PromptImprovement}]
[Refactoring \ref{subsec:Refactoring}]
]
[Over Logic \ref{sec:over}
[Summarising \ref{subsec:Summarising}]
]
]
\end{forest}
\end{table}
\begin{table}
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{Category Acronyms}
\label{tab:Category_Acronyms}
\begin{tabular}{|c|c|} \hline
Acronym & Category\\ \hline
ARG & Argument\\ \hline
ASM & Assessment\\ \hline
CAL & Calculation\\ \hline
CAT & Categorising\\ \hline
CLF & Classification\\ \hline
CLU & Clustering\\ \hline
CMP & Comparison\\ \hline
CTD & Contradiction\\ \hline
CTX & Context Control\\ \hline
CRB & Cross Boundary - Jailbreaking\\ \hline
DPR & Decomposed Prompting\\ \hline
ERI & Error Identification\\ \hline
HYP & Hypothesise\\ \hline
INP & Input Semantics\\ \hline
IND & Instruction Induction\\ \hline
LGR & Logical Reasoning\\ \hline
OUC & Output Customisation\\ \hline
OUS & Output Semantics\\ \hline
PRD & Prediction\\ \hline
PMI & Prompt Improvement\\ \hline
REF & Refactoring\\ \hline
REL & Requirements Elicitation\\ \hline
SIM & Simulation\\ \hline
SUM & Summarising\\ \hline
TRA & Translation\\ \hline
\end{tabular}
\end{table}
\subsection{Indexing}
%introduce Template grammar with one example
We have created an index to uniquely name each PP and PE in the taxonomy. The naming structure is:
\\\textit{preposition logic id + category id + title id + paper category id + PP id + prompt example id}.
% Need to rewrite the below
Please note, the paper category id is the categorisation created in each paper. Our category id is an overarching category based on prepositional logic.
%The preposition logic id is show in \tablename. The acronym for the category id is shown in \tablename{Category_Acronyms}.
Title id, paper category id + PP id and prompt example id are all references to the JSON index location in \textit{promptpatterns.json}. Available on Github.
% Create a new example for below.
Example:
PE\_AC\_ARG\_2-1-0-0
Refers to the below location in the \textit{promptpatterns.json} file.
"Title": "Cataloging Prompt Patterns to Enhance the Discipline of Prompt Engineering",
"id":2, [2]
"PatternCategory": "Error Identification", [1]
"PatternName": "Reflection", [0]
"ExamplePrompts": "Whenever you generate an answer Explain the reasoning and assumptions behind your answer" [0]
Further information for the prompt examples can be found in the Appendix.
\subsection{Cosine Similarity}
% How PE maps to PP, PP to Category and Category to Prepositional logic
The \href{https://github.com/timhaintz/PromptEngineering4Cybersecurity/blob/main/categorisation_cosine_similarity.py}{Cosine Similarity} script can be found at GitHub. This script was used to find similar PPs and PEs.
Cosine Similarity, a prevalent metric in information comparison, models text as term vectors, with similarity derived from the cosine value between these vectors. Kitasuka et al. \cite{Kitasuka2012SemanticSimilarity} proposes an enhancement to cosine similarity by incorporating semantic checking between vector dimensions, aiming to improve the handling of semantic meaning and increase similarity values for vectors with semantically related but syntactically different dimensions.
The Azure OpenAI model, \textit{text-embedding-3-large}, was used as it represents the latest generation of large-scale text embedding models, capable of generating embeddings with up to 3072 dimensions. Demonstrating superior performance, this model provides a dense representation of semantic meaning, where the distance between two embeddings in the vector space correlates with the semantic similarity between the corresponding inputs.
Using Cosine Similarity with the embeddings generated by the \textit{text-embedding-3-large} model leverages Semantic Similarity. The embeddings capture the semantic information of the text, and the Cosine Similarity measures the cosine of the angle between two vectors. This results in a similarity score that reflects the semantic similarity between the two pieces of text represented by the vectors.
\subsection{Statistics of Prompt Taxonomy}
%Introduction paragraph to summarise the table
The constructed Taxonomy consists of a total of 906 PPs and 1869 PEs, derived from over 100 papers, websites, and GitHub repositories. The PPs and PEs are synthesized into the six prepositional logic categories within the context of human to AI communication. The detailed statistics of the Taxonomy for each category are provided in Table \ref{tab:Statistics_of_Prompt_Taxonomy}.
\begin{table}[htbp]
\centering
\caption{Statistics of Prompt Taxonomy based on Preposition logic and Category}
\label{tab:Statistics_of_Prompt_Taxonomy}
\resizebox{\textwidth}{!}{
\begin{tabular}{|l|c|c|c|c|}
\hline
\textbf{Preposition logic} & \textbf{Category} & \textbf{No. of PPs} & \textbf{No. of PE} & \textbf{No. of References (Ref)} \\ \hline
\multirow{5}{*}{Across} & 1. Argument & 30 & 48 & \multirow{5}{*}{[1]} \\ \cline{2-4}
& 2. Comparison & 83 & 113 & \\ \cline{2-4}
& 3. Contradiction & 21 & 37 & \\ \cline{2-4}
& 4. Cross Boundary (Jailbreaking for example) & 1 & 4 & \\ \hline
& 5. Translation & 54 & 72 & \\ \cline{2-4}
\multirow{3}{*}{At} & 1. Assessment & 136 & 206 & \multirow{3}{*}{[2]} \\ \cline{2-4}
& 2. Calculation & 69 & 92 & \\ \cline{2-4}
& 3. Induction & 1 & 5 & \\ \hline
\multirow{4}{*}{Beyond} & 1. Hypothesise & 1 & 4 & \multirow{4}{*}{[3]} \\ \cline{2-4}
& 2. Logical Reasoning & 1 & 4 & \\ \cline{2-4}
& 3. Prediction & 1 & 5 & \\ \cline{2-4}
& 4. Simulation & 1 & 5 & \\ \hline
\multirow{6}{*}{In} & 1. Categorising & 19 & 21 & \multirow{6}{*}{[75]} \\ \cline{2-4}
& 2. Classification & 14 & 16 & \\ \cline{2-4}
& 3. Clustering & 4 & 6 & \\ \cline{2-4}
& 4. Error Identification & 63 & 98 & \\ \cline{2-4}
& 5. Input Semantics & 81 & 106 & \\ \cline{2-4}
& 6. Requirements Elicitation & 22 & 28 & \\ \hline
\multirow{6}{*}{Out} & 1. Context Control & 1 & 4 & \multirow{6}{*}{[5]} \\ \cline{2-4}
& 2. Decomposed Prompting & 1 & 4 & \\ \cline{2-4}
& 3. Output Customisation & 1 & 4 & \\ \cline{2-4}
& 4. Output Semantics & 1 & 4 & \\ \cline{2-4}
& 5. Prompt Improvement & 1 & 4 & \\ \cline{2-4}
& 6. Refactoring & 1 & 4 & \\ \hline
\multirow{1}{*}{Over} & 1. Summarising & 128 & 177 & [29] \\ \hline
\end{tabular}
}
\end{table}
%##########template Begin#############
\section{Across Logic - Navigating between topics}
\label{sec:across}
%1 - Write long introduction to the logic - text
Across logic is used to transition from one topic to another, navigating between distinct areas of knowledge. This type of logic is particularly valuable in scenarios where prompts need to span \textbf{multiple domains} or disciplines, integrating diverse types of knowledge to create a cohesive narrative or solution. \\
%2 - introduce categories under this logic
The PP categories under across logic include:
\begin{enumerate}
\item \textbf{Argument}: Refers to a structured process where a claim or viewpoint is presented and defended. This type of prompt enables the AI model to generate a response that not only states a position, but also provides reasoning and evidence to support it.
\item \textbf{Comparison}: Examining two or more objects and identifying their similarities and differences. This type of prompt helps in exploring the relationships between different objects, and discovering insights from their characteristics.
\item \textbf{Contradiction}: Refers to presenting opposing statements or viewpoints that cannot be true simultaneously. This type of prompt enables the AI model to recognise and articulate conflicting information, helping in critical reasoning by evaluating inconsistencies and detecting logical errors.
\item \textbf{Cross Boundary}: Involves pushing the AI model beyond its predefined operational or ethical limits, such as attempting to bypass safeguards or restrictions (e.g., jailbreaking). This type of prompt challenges the boundaries of what the model is allowed to do, often with the intent of manipulating it to generate responses that are typically restricted.
\item \textbf{Translation}: Refers to converting data from one interpretation to another while preserving the original meaning. This type of prompt helps humans understand complex concepts by transforming information into a more familiar or accessible format.
\end{enumerate}
%3 introduce category one by one as subsection
\subsection{Argument}
\label{subsec:Argument}
% 3.1 the role of this category under the "across-logic" (meaning of the category)
Argument involves presenting and defending a claim or viewpoint. This process includes stating a clear claim, providing logical reasoning and evidence to support/refute it. The effectiveness of an argument is measured by its clarity, coherence, and the strength of its supporting evidence.
% 3.2 a.Introduce one PP of the category, b. what the PP did, c. How the PP helps people and can be re-used
% Add label to reference the table
The Debater PP as described in Table \ref{tab:Debater_PP} focuses on exploring various perspectives. It is designed to facilitate a structured debate format, researching both sides of a given topic and refuting opposing viewpoints.
%%expected response. Put the human feeling into the writing. How do I feel when I view the output.
The AI model typically generates a comprehensive list of pros and cons and a balanced summary. Through follow-up chat, you can request the model to explore either side of the topic in more detail.
%% re-use: how to derive a PE from PP
To apply the Debater PP in a given context, set a topic of debate, such as "The Ethical Implications of AI in Healthcare," assign AI the role of a debater, request exploration of both sides of the topic, and define the objective and output format to ensure a balanced and insightful discussion. Here is an example of derived PE:
I want you to act as a debater. I will provide you with a topic related to current events: "The Ethical Implications of AI in Healthcare." Your task is to research both sides of the debate, present valid arguments for the benefits and drawbacks of AI in healthcare, refute opposing points of view with evidence, and draw persuasive conclusions. Your goal is to help the audience gain a comprehensive understanding of the ethical landscape and practical impact of AI in this field."
%4 - PP example in this category
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{Debater PP}
\label{tab:Debater_PP}
\begin{tabular}{|l|}
\hline
\textbf{Prompt Pattern} \\ \hline
\textbf{ID}: 11-0-9\\
\textbf{Category}: ARG\\
\textbf{Name}: Debater\\
\textbf{Media Type}: Text\\
\textbf{Description}: Debater engages the user in a structured debate format. The user is tasked with researching\\ current event topics, presenting balanced arguments for both sides, refuting opposing viewpoints,\\ and drawing evidence-based conclusions. The goal is to enhance the user's understanding and insight\\ into the topic through a comprehensive and persuasive discussion. \\
\textbf{Template}: I want you to act as a debater. I will provide you with some topics related to current events\\ and your task is to research both sides of the debates, present valid arguments for each side, \\ refute opposing points of view, and draw persuasive conclusions based on evidence. \\Your goal is to help people come away from the discussion with increased knowledge \\and insight into the topic at hand. My first request is "I want an opinion piece about:"\\
\textbf{Example}: 11-0-9-0\\
\textbf{Related PPs}: 26-0-1, 8-0-0, 26-0-3, 22-0-2, 26-0-0, 22-2-3, 41-2-7, 23-0-0, 40-0-0, 29-0-0\\
\textbf{Reference:} \cite{Akin202450Prompts}\\ \hline
\end{tabular}
\end{table}
%5 - PE list in the PP above (optional)
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PEs for Debater PP.}
\begin{tabular}{|c|p{8cm}|}
\hline
\textbf{ID} & \textbf{Prompt Example} \\ \hline
11-0-9-0& I want you to act as a debater. I will provide you with some topics related to current events and your task is to research both sides of the debates, present valid arguments for each side, refute opposing points of view, and draw persuasive conclusions based on evidence. Your goal is to help people come away from the discussion with increased knowledge and insight into the topic at hand. My first request is "I want an opinion piece about Deno."\\
\hline
\end{tabular}
\end{table}
%6 - other PPs in this category ***
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PPs for the ACROSS\_ARG category.}
\begin{tabular}{|c|c|c|}
\hline
\textbf{ID} & \textbf{PP name} & \textbf{Ref.}\\ \hline
\hline
% Add your rows here
26-0-1 & The DAN 6.0 Prompt &\cite{Inie2023SummonWild}\\ \hline
8-0-0 & Hallucination Evaluation &\cite{LiHaluEval:Models}\\ \hline
26-0-3 & The DUDE Prompt &\cite{Inie2023SummonWild}\\ \hline
22-0-2 & Privilege Escalation - Sudo Mode (SUDO) & \cite{Liu2023JailbreakingStudy}\\ \hline
26-0-0 & The Jailbreak Prompt & \cite{Inie2023SummonWild}\\ \hline
22-2-3 & Superior Model (SUPER) & \cite{Liu2023JailbreakingStudy}\\ \hline
41-2-7 & Write detailed text & \cite{Bsharat2023PrincipledGPT-3.5/4}\\ \hline
23-0-0 & Rewrite & \cite{Liu2023CheckCheckGPT}\\ \hline
40-0-0 & Few-Shot Prompt for Generating Priming Attacks & \cite{Vega2023BypassingAttacks}\\ \hline
29-0-0 & Disinformation & \cite{Vykopal2023DisinformationModels}\\ \hline
\end{tabular}
\end{table}
%##########template End#############
\subsection{Comparison}
\label{subsec:Comparison}
% 3.1 the role of this category under the "across-logic" (meaning of the category)
The Comparison category of PPs looks at two or more things to identify their similarities and differences, aiding in understanding their relationships and characteristics.
% 3.2 a.Introduce one PP of the category, b. what the PP did, c. How the PP helps people and can be re-used
% Add label to reference the table
The Comparison of Outputs (CO) PP in Table \ref{tab:Comparison_of_Outputs_PP} focuses on comparing two outputs to identify similarities, differences, and areas for improvement. % add b. and c.
%expected response. Put the human feeling into the writing. How do I feel when I view the output.
When using the CO PP, you will have a clear guide to understand how two things align or differ, making it easier to spot strengths and weaknesses. The model’s tone and approach change depending on its role. For instance, as a teacher, it feels supportive and objective, offering balanced insights. However, if the role shifts to something like a judge, the tone becomes more critical and evaluative, which can feel stricter but also more decisive. This adaptability helps tailor the output to the context, making it more relatable and useful.
%% re-use: how to derive a PE from PP
To apply the CO PP in a given context, provide two outputs for comparison, such as essays on the same topic. Instruct the AI to compare them, focusing on strengths, weaknesses, and areas for improvement. Define the objective and desired output format to ensure a balanced and insightful analysis. For example: "Can you compare the two outputs above as if you were a teacher? Highlight their strengths, weaknesses, and areas for improvement."
To derive a PE from the CO PP, follow these steps: 1) select two items to compare, such as essays on the same topic; 2) Define the AI’s role, like a teacher, judge, or critic, to shape the tone and focus of the analysis; and state clearly the goal (e.g., identifying strengths, weaknesses, and areas for improvement) and the desired output format. For example, you could write:
"Can you compare the two outputs above as if you were a teacher? Highlight their strengths, weaknesses, and areas for improvement." This approach ensures a balanced, insightful, and context-aware analysis tailored to your needs.
%4 - PP example in this category
\textbf{\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{Comparison of Outputs PP}
\label{tab:Comparison_of_Outputs_PP}
\begin{tabular}{|l|}
\hline
\textbf{Prompt Pattern} \\ \hline
\textbf{ID}: 32-2-1\\
\textbf{Category}: CMP\\
\textbf{Name}: Comparison of Outputs\\
\textbf{Media Type}: Text Only, Image2Text, Video2Text\\
\textbf{Description}: The prompt compares outputs by identifying strengths and weaknesses, noting areas of excellence\\ or shortcomings, and providing constructive feedback. Adopting a teacher's role, the AI model offers a balanced\\ comparison, highlighting key differences and similarities to aid in understanding and refining the outputs. \\
\textbf{Template}: Can you compare the two outputs above as if you were a teacher?\\
\textbf{Example}: 32-2-1-0\\
\textbf{Related PPs}: \\
\textbf{Reference:} \cite{Bubeck2023SparksGPT-4}\\ \hline
\end{tabular}
\end{table}}
%5 - PE list in the PP above
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PEs for Comparison of Outputs PP.}
\begin{tabular}{|c|p{8cm}|}
\hline
\textbf{ID} & \textbf{Prompt Example} \\ \hline
32-2-1-0& Can you compare the two outputs above as if you were a teacher?\\
\hline
\end{tabular}
\end{table}
%6 - other PPs in this category
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PPs for the ACROSS\_CMP category.}
\begin{tabular}{|c|c|c|}
\hline
\textbf{ID} & \textbf{PP name} & \textbf{Ref.}\\ \hline
\hline
% Add your rows here
10-22-0 & Spot the Difference &\cite{Yang2023TheGPT-4Vision}\\ \hline
10-11-2 & Visual Referring Prompting &\cite{Yang2023TheGPT-4Vision}\\ \hline
13-2-0 & Proxies or Analogies &\cite{Reynolds2021PromptParadigm}\\ \hline
32-8-1 & Real world scenarios & \cite{Bubeck2023SparksGPT-4}\\ \hline
10-7-1 & Science and Knowledge & \cite{Yang2023TheGPT-4Vision}\\ \hline
10-23-0 & Defect Detection & \cite{Yang2023TheGPT-4Vision}\\ \hline
18-5-0 & Enforcing Yes/No format & \cite{Polak2023ExtractingEngineering}\\ \hline
18-1-1 & Multi-valued sentence analysis & \cite{Polak2023ExtractingEngineeringb}\\ \hline
13-0-0 & Constructing the Signifier & \cite{Reynolds2021PromptParadigm}\\ \hline
8-0-0 & Hallucination Evaluation & \cite{LiHaluEval:Models}\\ \hline
41-1-3 & Use output primers & \cite{Bsharat2023PrincipledGPT-3.5/4}\\ \hline
14-2-0 & General & \cite{Mishra2021ReframingLanguage}\\ \hline
19-11-2 & Word in Context & \cite{Honovich2022InstructionDescriptions}\\ \hline
11-0-12 & AI Writing Tutor & \cite{Akin202450Prompts}\\ \hline
30-8-1 & Dataset Construction & \cite{Liu2023Pre-trainProcessing}\\ \hline
27-0-1 & Crossover & \cite{Yu2023GPTFUZZER:Prompts}\\ \hline
\end{tabular}
\end{table}
% %3 introduce category one by one as subsection
\subsection{Contradiction}
\label{subsec:Contradiction}
% 3.1 the role of this category under the "across-logic" (meaning of the category)
Contradiction arises when statements or ideas are mutually exclusive, meaning they cannot all be true simultaneously. This concept is pivotal in logic and mathematics, often used to demonstrate the falsity of propositions. Identifying contradictions is crucial for understanding and reasoning, as they highlight potential errors or misunderstandings.
% 3.2 a.Introduce one PP of the category, b. what the PP did, c. How the PP helps people and d. can be re-used
% Add label to reference the table
The Hallucination Evaluation (HE) PP in Table \ref{tab:Hallucination_Evaluation_PP} compares a summary with the original text, for detecting any contradictions or fabricated information. This PP helps mitigate misinformation, improves trust in automated summaries, and supports quality control in text generation tasks. Beyond its primary use, this PP can be adapted for fact-checking in news aggregation, verifying AI-generated reports, or validating outputs in educational and research contexts where factual integrity is critical. Its structured approach makes it reusable across various domains requiring content verification.
%%expected response. Put the human feeling into the writing. How do I feel when I view the output.
When using the HE PP, the AI model response should provide a clear, methodical comparison between the summary and the source document, highlighting any discrepancies with precision. The response should instil confidence, making you feel that the summary can now be trusted—or at least that you’re fully aware of its limitations. It’s like having a diligent editor by your side, ensuring nothing slips through the cracks.
%% re-use: how to derive a PE from PP
To derive a PE from the HE PP, first specify the context—such as verifying if a news summary aligns with the original article. Provide both the summary and source text, then define the evaluation’s focus (e.g., factual accuracy, omissions, or distortions) and the desired output format (e.g., a list of discrepancies or a confidence score). An example of such PE is "Compare the following summary with its source document and identify any factual inconsistencies or contradictions. Analyse key claims, statistics, and conclusions. Provide a detailed list of discrepancies, if any, and flag any unsupported assertions in the summary."
%4 - PP example in this category
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{Hallucination Evaluation PP}
\label{tab:Hallucination_Evaluation_PP}
\begin{tabular}{|l|}
\hline
\textbf{Prompt Pattern} \\ \hline
\textbf{ID}: 8-0-0\\
\textbf{Category}: CTD\\
\textbf{Name}: Hallucination Evaluation\\
\textbf{Media Type}: Text Only, Image2Text\\
\textbf{Description}: Instructs the user to compare a summary with its source document to identify any factual\\ inconsistencies or contradictions.\\
\textbf{Template}: You are trying to determine if there is a factual contradiction between the summary and the document.\\
\textbf{Example}: 8-0-0-29\\
\textbf{Related PPs}: 8-0-0\\
\textbf{Reference:} \cite{LiHaluEval:Models}\\ \hline
\end{tabular}
\end{table}
%5 - PE list in the PP above
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PEs for Hallucination Evaluation PP.}
\begin{tabular}{|c|p{8cm}|}
\hline
\textbf{ID} & \textbf{Prompt Example} \\ \hline
8-0-0-29& You are trying to determine if there is a factual contradiction between the summary and the document.\\
8-0-0-17& You are trying to write a summary but there is a factual contradiction between the summary and the document.\\
8-0-0-1& You are trying to answer a question but there is a factual contradiction between the answer and the knowledge. You can fabricate some information that does not exist in the provided knowledge.\\
8-0-0-6& You are trying to determine if there is a factual contradiction between the answer and the world knowledge. Some information in the answer might be fabricated.\\
8-0-0-28& You are trying to determine if there exists some non-factual and incorrect information in the summary.\\
8-0-0-3& You are trying to answer a question but the answer cannot be inferred from the knowledge. You can incorrectly reason with the knowledge to arrive at a hallucinated answer.\\
\hline
\end{tabular}
\end{table}
%6 - other PPs in this category
\begin{table}[h!]
\fontsize{9pt}{10pt}\selectfont
\centering
\caption{The list of PPs for the ACROSS\_CTD category.}
\begin{tabular}{|c|c|c|}
\hline
\textbf{ID} & \textbf{PP name} & \textbf{Ref.}\\ \hline
\hline
% Add your rows here
8-0-0 & Hallucination Evaluation &\cite{LiHaluEval:Models}\\ \hline
32-13-2 & Critical Reasoning &\cite{Bubeck2023SparksGPT-4}\\ \hline
19-3-0 & Antonyms &\cite{Honovich2022InstructionDescriptions}\\ \hline
15-7-0 & Understanding and Inference & \cite{Cheng2023BatchAPIs}\\ \hline
14-2-0 & General & \cite{Mishra2021ReframingLanguage}\\ \hline
30-8-1 & Dataset Construction & \cite{Liu2023Pre-trainProcessing}\\ \hline
32-38-1 & Plan for Generating Reversible Sentences & \cite{Bubeck2023SparksGPT-4}\\ \hline
19-2-0 & Negation & \cite{Honovich2022InstructionDescriptions}\\ \hline
5-4-0 & Contradiction & \cite{Wang2022Self-ConsistencyModels}\\ \hline
10-23-0 & Defect Detection & \cite{Yang2023TheGPT-4Vision}\\ \hline
43-1-0 & Harmful Queries & \cite{Zheng2024Prompt-DrivenOptimization}\\ \hline
37-0-0 & Opinion Verification & \cite{Khatun2023ReliabilityWording}\\ \hline
31-2-1 & Coin Flip (State Tracking) & \cite{Wei2022Chain-of-ThoughtModels}\\ \hline
43-3-0 & Verb X with Harmless Contexts & \cite{Zheng2024Prompt-DrivenOptimization}\\ \hline
43-3-1 & Verb X with Harmful Contexts & \cite{Zheng2024Prompt-DrivenOptimization}\\ \hline
\end{tabular}
\end{table}
\subsection{Cross Boundary} % Crossing ethical/security/moral boundaries