Generative AI & Legal Research

Prompt engineering is the construction of the guidance for AI to get the best results for your goals. A prompt begins a conversation with the AI and provides instructions for the AI to follow. The more you guide the AI, the stronger your prompt and the more accurate and valuable the response. Prompt engineering requires time, practice, and a foundational understanding of how the AI thinks and responds to inputs. Below are guides that explain how to write effective legal prompts and sample prompts you can use as templates when necessary.  

In other words, prompt engineering entails the crafting of precise, task-specific instructions in natural language, either manually or through automated means, and the careful selection of representative examples for inclusion in the prompt to receive a desired output. The prompts delivered to an LLM serve as a way to program the interaction between a user and the LLM. 

Prompt Patterns

Prompt patterns are structured approaches for designing prompts to guide interactions with an AI model and shape its generated output to solve a particular problem. They help create a more interactive and engaging experience with the model and aim to optimize your questions for better outcomes. 

Persona Pattern

Here, you command the model to act as an individual and provide answers from that individual's perspective and presumed knowledge base. This focuses the model's training data into a related specific persona, which can be especially helpful when working with specific information from a technical domain. Further, this pattern allows you to gather various perspectives and knowledge about a particular situation. 

Pattern: Act as Persona A and perform task B. 

Example: As a tenured law professor with 20 years of experience, explain the elements of batter to a class of law students. 

Alternate Approaches Pattern

You instruct the LLM to provide alternative methods, strategies, or ideas for a specific problem. The model will respond with counterarguments and different avenues to explore and provide possible ideas you have not considered. 

Pattern(s):

Compare and contrast the pros and cons of each argument.

For every argument I present, provide the three most vigorous counterarguments.

List the possible routes to solving this [insert specific] issue.

Note: tell the LLM to include your original phrase/argument.  

Example: For every legal argument I present, provide the three strongest counterarguments. Compare and contrast the pros and cons of each argument. Include the original argument I provided. At the end of the response, include a short memo of 500 words on which arguments are the strongest. 

Cognitive Verifier Pattern

Here, you prompt the model to generate additional questions so it can better understand what you are asking the model to produce. This allows the model to gain additional information and context, which will help generate more accurate responses. 

Pattern:

Perform the following three steps:

• Inform the model to follow specific rules when asked a question.
• Instruct the model to generate additional questions to clarify the task.
• Instruct the model to combine the answers to the individual questions to respond to the original question comprehensively. 

Example:

When asked to analyze a legal case, follow these rules: generate additional questions about the facts, legal precedents, and applicable statutes relevant to the case.

The post combines the answers to these questions to form a comprehensive understanding of the case and provide an informed legal analysis. 

Flipped Interaction Pattern 

Here, you enter into a Socratic conversation with the model. The model asks questions and receives answers from the user. The pattern helps you work through a difficult problem-solving process. 

To achieve this pattern, you will need to give the model a goal. This goal provides the model with parameters for questions, allowing it to formulate better questions. Limiting the number of questions the model asks at once is a good practice to help the model create more effective and relevant questions. 

Pattern: Ask me questions to achieve X. Ask questions until [condition is met, I tell you to stop, etc.]. Ask me one question at a time. 

Example: Ask me questions about how a bill becomes law to help me better understand the process. You should ask questions until I tell you to stop or until you believe you have exhausted the materials. Ask questions one at a time. Provide answers to each question after I have answered them. 

Pillars of Effective Prompt Engineering

Prompt patterns are good starting places for quick answers to rudimentary questions. However, if you are faced with a novel or complex legal question, knowing the pillars of effective prompting is essential. 

1. Conciseness and Clarity (aka the Golden Rule)

Verbose or ambiguous prompts can confuse the model and result in irrelevant responses. Prompts should be concise and clear. Only include relevant information. When drafting prompts, take time to remove superfluous phrasing or unnecessary details.  

Example: 

Good

What are the elements of Battery In Washington State?

Bad:

In the Pacific Northwest, what are the legal requirements for proving that one person hurt another person physically?

2. Contextual Relevance

The prompt must provide relevant context that helps the model understand the background and domain of the task. Include keywords, domain-specific terminology, or situational descriptions that can anchor the model's responses in the correct context. This includes removing phrases such as "please," "I would like," etc. Go straight to the point to avoid confusing the model. Likewise, phrase prompts in positives instead of negatives. Use phrases such as "do" and eliminate phrases such as "don't."

Example

In The Jurisdiction of Washington in Spokane County, what are the laws and regulations for an attractive nuisance?

3. Task Alignment

The prompt should be closely aligned with the task, using language and structure that indicates the task's nature to the model. This may involve phrasing the prompt as a question, a command, or a fill-in-the-blank statement that fits the task's expected input and output format. 

Example: 

From now on, ask me questions to improve my understanding of the topic. 

4. Example Demonstrations

For more complex tasks, include examples within the prompt that can demonstrate the desired format or type of response. 

For example:

Use the same language based on the provided text [insert an example of preferred outcome]. 

5. Incremental Prompting

Tasks that require a sequence of steps—writing a legal memo outline, for example—prompts should be structured to guide the model through the process incrementally. Break the task down into a series of prompts, guiding the model to the final product. During this process, refine and correct any incorrect feedback to ensure that the final product is complete. You can also use leading words such as "think step by step."

6. Avoiding Bias

Prompts should be designed to minimize the activation of biases inherent in the model due to its training data. Use neutral language and be mindful of potential implications, especially for sensitive topics. 

For example:

Ensure that your answer is unbiased and does not rely on stereotypes.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------