Mar 12
ChatGPT / Agile #9: Prompt Engineering
I’ve delayed the next series of agile project management examples so I can dive into an important topic: “Prompt Engineering”.
I accidentally discovered “Prompt Engineering” as a “thing” while reading a GPT yarn on Medium. So, of course, I googled the term and was surprised to find a book published on Amazon. So I hopped over to Amazon to find it, and to my even greater surprise, there are over ten books published already. About eight were available in my Kindle Unlimited service, so I quickly downloaded and skimmed them to get a “flavour” of the topic and the approach of different authors.
I’m spending time on “Prompt Engineering” because it seems to fill a gap in how I’ve been approaching and describing my interactions with ChatGPT for this series. I’ve been writing about and demonstrating that what you get out of ChatGPT heavily depends on what and, more importantly, how you ask or tell the model. ChatGPT is very sensitive to context and terminology. You may recall my “ argument “ with ChatGPT in this yarn.
I hoped that anything “xxx Engineering” offered a more systematic analysis of the subject “xxxx” and, therefore, more structured guidelines on approaching it. If my hopes were realised, at the very least, I could use the term and not tap dance around it with terms like “tweak”.
Why not just ask ChatGPT about “Prompt Engineering”?
Good question with an interesting — even amusing — answer.
AOP: what is "prompt engineering"GPT: I'm not familiar with the term "prompt engineering." It may refer to a
specific concept or methodology within a certain field, but without more
context or information, it's difficult to provide a more detailed answer.
If you could provide additional information or context about where you heard
this term or in what context it was used, I may be able to assist you better.Whoops! That’s a surprise. So an AI model wants context on an AI term. Hmmm. Ok.
AOP: what is "prompt engineering" in the field of language models, chatbots
and artificial intelligence natural language interfacesGPT: I'm not familiar with the specific term "prompt engineering" in the
context of language models, chatbots, and artificial intelligence
natural language interfaces. However, in general, "prompt" refers to the input
or stimulus provided to a machine learning model or system in order to generate
a response or output.
In the context of language models, prompt engineering may refer to the process
of designing and optimizing the prompts that are used to elicit a desired
response from the model. This may involve selecting appropriate keywords,
phrasing, or questions to guide the model's output, as well as adjusting the
parameters or hyperparameters of the model to improve its performance on
specific prompts.
Overall, prompt engineering is an important aspect of developing effective
natural language interfaces, as it can help to ensure that the system produces
relevant and accurate responses to user input.Although I made some progress, that still feels like an unreliable answer. First, GPT is not familiar with the term but then explains it.
Over to google — and then to Amazon — I went, found, downloaded and skimmed the nine references I have listed below at the end of this yarn.
Prompt Engineering Defined
My preferred definition is from Ibrahim John’s book, which I also liked the look of from a content perspective as well, remembering that I just skimmed them.
“Prompt engineering is the process of creating prompts or asking or instructions that guide the output of a language model like ChatGPT. It allows users to control the output of the model and generate text that is tailored to their specific needs.”
John, Ibrahim. The Art of Asking ChatGPT for High-Quality Answers: A Complete Guide to Prompt Engineering Techniques (p. 2). Nzunda Technologies Limited. Kindle Edition.
I also liked the breakdown of John’s book, which specifically related to different types of prompts from an intent perspective. Other books looked at prompts from a domain or functional perspective. Those other books had content on the “Prompt” but didn’t seem to spend too much time on the “engineering”. This type of content is available in volume from other sources.
As an example, here’s an extract from the Table of Contents:
These looked to be very useful (again on short inspection) and, once understood, can be easily applied to any specific domain or use case.
Back to ChatGPT
I had thought to give ChatGPT all those definitions and ask it to develop a mega-definition, extracting the most useful aspects from each one.
But for some reason, I decided to ask again about the meaning of “Prompt Engineering.”
AOP: What is prompt engineering?GPT: Prompt engineering refers to the process of designing and optimizing
prompts for use with machine learning models or natural language processing systems.
A prompt is the input or stimulus provided to the model, which is used to
generate a response or output.
In the context of natural language processing, prompt engineering may involve
selecting appropriate keywords, phrases, or questions to guide the model's
output, as well as adjusting the parameters or hyperparameters of the model to
improve its performance on specific prompts. This process is particularly
important for developing effective chatbots, language models, and other
natural language interfaces, as it can help to ensure that the system produces
relevant and accurate responses to user input.
Prompt engineering may also involve the use of techniques such as transfer
learning, where a pre-trained model is fine-tuned on a specific task or set
of prompts, or data augmentation, where additional training data is generated
to improve the model's ability to handle diverse inputs. Overall, prompt engineering
is an important aspect of developing effective and efficient natural language
processing systems.Boom! No hesitation this time.
Pop quiz: can you give me one difference between the first time I tried this and the above? I’ll buy a coffee for anyone who can tell me the difference. First correct answer in the comments, and it’s yours!
A Parting Thought
In the references below, but not on Kindle, I also liked the article in the Conversation by Marcel Scharth. How to perfect your prompt writing for ChatGPT, Midjourney and other AI generators.
For all the thinking and writing I’ve done about the benefits of a Natural Language interface to the AI Engine and the underlying dataset, it’s not exactly “natural” in some senses.
Scharth gave me food for thought with this:
“Since it’s a chatbot, you may be inclined to engage with it conversationally. But this isn’t the best approach if you want tailored results. Instead, adopt the mindset that you’re programming the machine to perform a writing task for you.”
This is a good steer, particularly given the ease with which I find myself anthropomorphizing the chatbot: it just further encourages you to adopt a conversational style.
This reminds me; I was thinking about this topic right at the beginning of my explorations of ChatGPT applied to Agile Project Management. I planned — and still plan — to investigate using structured languages to set up the tasks I want ChatGPT to perform. Of particular interest is “PlanGuage”, developed by Tom Gilb in the 1980’s — or possibly earlier. I know I have some reference material around somewhere. If I recall correctly, “PlanGuage” uses English words with specific meanings to construct and monitor project plans.
The Bottom Line
The discovery of “Prompt Engineering” is exciting for me because it appears to plug a gap that I had identified in my early investigations and descriptions of ChatGPT and projects: the sensitivity of the AI model to nuanced variations in the request language structure and content.
I hope you’re as excited as me — unless you already knew about it.
I believe from here I’ll give Ibrahim John’s book a deeper read and flesh out my skills and understanding of “Prompt Engineering”. That means I’ll not only be able to give ChatGPT better tasking, but also I’ll be able to describe the results more effectively to you.
I prefer to develop those skills before continuing with the practical Agile Project Management examples I had planned.
Stay Tuned.
References and Definitions
Here are the other eight references I skimmed and the definitions of “Prompt Engineering” that I found in each one. I’m quoting their definitions because they give you an idea of the style and approach of each. The last one isn’t a book: it’s a long read in an online independent news publication that I respect.
NOTE: No definition is quoted because Hunter’s book is a paid purchase, and I haven’t gone that far yet.
Nathan Hunter. The Art of Prompt Engineering with chatGPT: A Hands-On Guide for using chatGPT [Print Replica] Kindle Edition
“Prompt engineering is the process of designing and optimizing prompts used in artificial intelligence (AI) models to achieve more accurate and reliable responses. A prompt is a piece of text that is provided to an AI model to generate a response. The goal of prompt engineering is to craft prompts that enable AI models to provide the desired output with a high degree of accuracy. This involves selecting the right language, format, and context to guide the AI model to produce the desired output. Prompt engineering is a critical part of the AI model development process. It involves understanding the underlying algorithmic and computational techniques used by the AI model, as well as the specific application domain for which the model is being developed. Effective prompt engineering requires expertise in natural language processing, machine learning, and computer science.”
Alqahtani, Saeed. The Art of Prompt Engineering: Crafting Effective AI Responses (p. 8). Kindle Edition.
“Prompt engineering is a technique that is used to customize the behavior of GPT models to achieve specific goals. It involves designing prompts so that the generated text is more relevant and accurate for the given use case. The prompt provides context and specific guidelines for what kind of information GPT should generate.”
Prof. Makoto. The Art and Science of ChatGPT Prompt Engineering (p. 5). Kindle Edition.
“This book doesn’t not cover all the aspects of prompt engineering as this is a completely new area.”
Bennett, Thomas. The Art of Chatting with chatGPT: Prompt mastering to leverage its power (p. 50). Kindle Edition.
“… prompt engineering is the art of crafting effective prompts for AI models like ChatGPT. A prompt is a short piece of text that provides a context or direction for the AI to generate language. By carefully crafting prompts, you can guide the AI to generate language that is relevant, coherent, and meaningful for your specific use case.”
Tull, Shaun. The Power of Prompting: Mastering the Art of Prompt Engineering with ChatGPT (p. 7). Kindle Edition.
“Prompt engineering is an important aspect of using AI models like ChatGPT effectively. It involves designing and creating the prompts that are used to generate responses from these models. The quality of the prompts can have a significant impact on the quality of the responses generated by the AI models, making prompt engineering a critical step in the process.”
McMullen, Bob. ChatGPT Prompt Engineering: Get Great Responses the FIRST Time! (p. 4). Kindle Edition.
“The goal of prompt engineering is to create systems that are able to respond to user requests or inputs in real-time, with minimal latency.”
RADI, Mouad. Introduction to Prompt Engineering: Prompt-based Systems Examples (p. 4). Kindle Edition.
“… the goal of prompt engineering: the skill of crafting an input to deliver a desired result from generative AI.”
Scharth, Marcel. How to perfect your prompt writing for ChatGPT, Midjourney and other AI generators in The Conversation
