Instruction Tuned LLM的训练通常是从已经训练好的基本 LLMs 开始,该模型已经在大量文本数据上进行了训练。然后,使用人工标注的prompt-answer数据集来对其进行有监督微调,要求它遵循这些指令。最后通常使用一种称为 RLHF(reinforcement learning from human feedback,人类反馈强化学习)的技术进行进一步改进,使系统更能够有帮助地遵循指令(具体原理可参考《李沐论文精度系列之九:InstructGPT》)。
本课程由吴恩达和OpenAI 的技术团队成员 Isa Fulford 教授一起授课。在本课程中,我们将与您分享一些LLM的可能性以及实现它们的最佳实践,希望这能激发你创建新应用的想象力(LLM 或大型语言模型作为开发人员的更强大功能是使用 API 调用到 LLM,以快速构建软件应用程序。)。本课程包括:
如果任务太复杂,模型在匆忙中可能会推导出错误的结论(编造一些错误的猜测),就像在没有时间计算出答案的情况下完成复杂的数学问题,人类也可能会犯错误。所以我们可以重新构思prompt,请求模型先进行一系列相关的推理,再给出最终答案。这等于指示模型花更多时间一步步思考问题(step by step),也意味着它在任务上花费了更多的计算资源。
Completion for prompt 1:
Two siblings, Jack and Jill, go on a quest to fetch water from a well on a hilltop, but misfortune strikes and they both tumble down the hill, returning home slightly battered but with their adventurous spirits undimmed.
Deux frères et sœurs, Jack et Jill, partent en quête d'eau d'un puits sur une colline, mais un malheur frappe et ils tombent tous les deux de la colline, rentrant chez eux légèrement meurtris mais avec leurs esprits aventureux intacts.
Noms(法语中的姓名): Jack, Jill.
{
"french_summary": "Deux frères et sœurs, Jack et Jill, partent en quête d'eau d'un puits sur une colline, mais un malheur frappe et ils tombent tous les deux de la colline, rentrant chez eux légèrement meurtris mais avec leurs esprits aventureux intacts.",
"num_names": 2
}
Completion for prompt 2:
Summary: Two siblings, Jack and Jill, go on a quest to fetch water from a well on a hilltop, but misfortune strikes and they both tumble down the hill, returning home slightly battered but with their adventurous spirits undimmed.
Translation: Jack and Jill, deux frères et sœurs, ont eu un accident en allant chercher de l'eau dans un puits de montagne, mais ils ont continué à explorer avec un esprit d'aventure.
Names: Jack, Jill
Output JSON: {"french_summary": "Jack and Jill, deux frères et sœurs, ont eu un accident en allant chercher de l'eau dans un puits de montagne, mais ils ont continué à explorer avec un esprit d'aventure.", "num_names": 2}
Introducing our beautiful medieval-style office furniture collection, including filing cabinets, desks, bookcases, and conference tables. Choose from a variety of shell colors and base coatings, with optional plastic or fabric/leather decoration. The chair features a plastic-coated aluminum base with five wheels and pneumatic height adjustment. Perfect for home or commercial use. Made in Italy.
# 更进一步,要求在描述末尾包含 7个字符的产品IDprompt =f"""
Your task is to help a marketing team create a
description for a retail website of a product based
on a technical fact sheet.
Write a product description based on the information
provided in the technical specifications delimited by
triple backticks.
The description is intended for furniture retailers,
so should be technical in nature and focus on the
materials the product is constructed from.
At the end of the description, include every 7-character
Product ID in the technical specification.
Use at most 50 words.
Technical specifications: ```{fact_sheet_chair}```
"""response = get_completion(prompt)
print(response)
Introducing our beautiful medieval-style office furniture collection, featuring file cabinets, desks, bookshelves, and conference tables. Available in multiple shell colors and base coatings, with optional plastic or fabric/leather decorations. The chair comes with or without armrests and has a plastic-coated aluminum base with five wheels and pneumatic height adjustment. Suitable for home or commercial use. Made in Italy.
Product IDs: SWC-100, SWC-110
prompt =f"""
Your task is to generate a short summary of a product review from an ecommerce site.
Summarize the review below, delimited by triple
backticks, in at most 30 words.
Review: ```{prod_review}```
"""response = get_completion(prompt)
print(response)
Soft and cute panda plush toy loved by daughter, but a bit small for the price. Arrived early.
review_1 = prod_review
# review for a standing lampreview_2 ="""
Needed a nice lamp for my bedroom, and this one \
had additional storage and not too high of a price \
point. Got it fast - arrived in 2 days. The string \
to the lamp broke during the transit and the company \
happily sent over a new one. Came within a few days \
as well. It was easy to put together. Then I had a \
missing part, so I contacted their support and they \
very quickly got me the missing piece! Seems to me \
to be a great company that cares about their customers \
and products.
"""# review for an electric toothbrushreview_3 ="""
My dental hygienist recommended an electric toothbrush, \
which is why I got this. The battery life seems to be \
pretty impressive so far. After initial charging and \
leaving the charger plugged in for the first week to \
condition the battery, I've unplugged the charger and \
been using it for twice daily brushing for the last \
3 weeks all on the same charge. But the toothbrush head \
is too small. I’ve seen baby toothbrushes bigger than \
this one. I wish the head was bigger with different \
length bristles to get between teeth better because \
this one doesn’t. Overall if you can get this one \
around the $50 mark, it's a good deal. The manufactuer's \
replacements heads are pretty expensive, but you can \
get generic ones that're more reasonably priced. This \
toothbrush makes me feel like I've been to the dentist \
every day. My teeth feel sparkly clean!
"""# review for a blenderreview_4 ="""
So, they still had the 17 piece system on seasonal \
sale for around $49 in the month of November, about \
half off, but for some reason (call it price gouging) \
around the second week of December the prices all went \
up to about anywhere from between $70-$89 for the same \
system. And the 11 piece system went up around $10 or \
so in price also from the earlier sale price of $29. \
So it looks okay, but if you look at the base, the part \
where the blade locks into place doesn’t look as good \
as in previous editions from a few years ago, but I \
plan to be very gentle with it (example, I crush \
very hard items like beans, ice, rice, etc. in the \
blender first then pulverize them in the serving size \
I want in the blender then switch to the whipping \
blade for a finer flour, and use the cross cutting blade \
first when making smoothies, then use the flat blade \
if I need them finer/less pulpy). Special tip when making \
smoothies, finely cut and freeze the fruits and \
vegetables (if using spinach-lightly stew soften the \
spinach then freeze until ready for use-and if making \
sorbet, use a small to medium sized food processor) \
that you plan to use that way you can avoid adding so \
much ice if at all-when making your smoothie. \
After about a year, the motor was making a funny noise. \
I called customer service but the warranty expired \
already, so I had to buy another one. FYI: The overall \
quality has gone done in these types of products, so \
they are kind of counting on brand recognition and \
consumer loyalty to maintain sales. Got it in about \
two days.
"""reviews = [review_1, review_2, review_3, review_4]
1
2
3
4
5
6
7
8
9
10
11
12
for i inrange(len(reviews)):
prompt =f"""
Your task is to generate a short summary of a product \
review from an ecommerce site.
Summarize the review below, delimited by triple \
backticks in at most 20 words.
Review: ```{reviews[i]}```
""" response = get_completion(prompt)
print(i, response, "\n")
0 Soft and cute panda plush toy loved by daughter, but a bit small for the price. Arrived early.
1 Affordable lamp with storage, fast shipping, and excellent customer service. Easy to assemble and missing parts were quickly replaced.
2 Good battery life, small toothbrush head, but effective cleaning. Good deal if bought around $50.
3 The product was on sale for $49 in November, but the price increased to $70-$89 in December. The base doesn't look as good as previous editions, but the reviewer plans to be gentle with it. A special tip for making smoothies is to freeze the fruits and vegetables beforehand. The motor made a funny noise after a year, and the warranty had expired. Overall quality has decreased.
defget_completion(prompt, model="gpt-3.5-turbo"):
messages = [{"role": "user", "content": prompt}]
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=0, # this is the degree of randomness of the model's output )
return response.choices[0].message["content"]
# 中文story_zh ="""
在政府最近进行的一项调查中,要求公共部门的员工对他们所在部门的满意度进行评分。调查结果显示,NASA 是最受欢迎的部门,满意度为 95%。
一位 NASA 员工 John Smith 对这一发现发表了评论,他表示:
“我对 NASA 排名第一并不感到惊讶,这是一个与了不起的人一起工作的好地方。我为成为这样一个创新组织的一员感到自豪。”
NASA 的管理团队也对这一结果表示欢迎,主管 Tom Johnson 表示:
“我们很高兴听到我们的员工对 NASA 的工作感到满意。
我们拥有一支才华横溢、忠诚敬业的团队,他们为实现我们的目标不懈努力,看到他们的辛勤工作得到回报真是太棒了。”
调查还显示,社会保障管理局的满意度最低,只有 45%的员工表示满意。政府承诺解决调查中员工提出的问题,并努力提高所有部门的工作满意度。
"""
在机器学习中这称为 Zero-Shot :没有给模型任何有标注的训练数据,仅凭prompt,它就能确定新闻文章涵盖了哪些主题。 3. 新闻提醒 如果我们想生成一个新闻提醒,也可以使用这个处理新闻的过程。假设我非常喜欢 NASA 所做的工作,就可以构建一个这样的系统,每当 NASA 新闻出现时,输出提醒。
1
2
3
topic_dict = {i.split(':')[0]: int(i.split(':')[1]) for i in response.split(sep='\n')}
if topic_dict['美国航空航天局'] ==1:
print("提醒: 关于美国航空航天局的新消息")
user_messages = [
"La performance du système est plus lente que d'habitude.", # System performance is slower than normal "Mi monitor tiene píxeles que no se iluminan.", # My monitor has pixels that are not lighting"Il mio mouse non funziona", # My mouse is not working"Mój klawisz Ctrl jest zepsuty", # My keyboard has a broken control key"我的屏幕在闪烁"# My screen is flashing]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
for issue in user_messages:
prompt =f"告诉我以下文本是什么语种,直接输出语种,如法语,无需输出标点符号: ```{issue}```" lang = get_completion(prompt)
print(f"原始消息 ({lang}): {issue}\n")
prompt =f"""
将以下消息分别翻译成英文和中文,并写成
中文翻译:xxx
英文翻译:yyy
的格式:
```{issue}```
""" response = get_completion(prompt)
print(response, "\n=========================================")
原始消息 (法语): La performance du système est plus lente que d'habitude.
中文翻译:系统性能比平时慢。
英文翻译:The system performance is slower than usual.
=========================================
原始消息 (西班牙语): Mi monitor tiene píxeles que no se iluminan.
中文翻译:我的显示器有一些像素点不亮。
英文翻译:My monitor has pixels that don't light up.
=========================================
原始消息 (意大利语): Il mio mouse non funziona
中文翻译:我的鼠标不工作了。
英文翻译:My mouse is not working.
=========================================
原始消息 (波兰语): Mój klawisz Ctrl jest zepsuty
中文翻译:我的Ctrl键坏了
英文翻译:My Ctrl key is broken.
=========================================
原始消息 (中文): 我的屏幕在闪烁
中文翻译:我的屏幕在闪烁。
英文翻译:My screen is flickering.
=========================================
text = [
"The girl with the black and white puppies have a ball.", # The girl has a ball."Yolanda has her notebook.", # ok"Its going to be a long day. Does the car need it’s oil changed?", # Homonyms"Their goes my freedom. There going to bring they’re suitcases.", # Homonyms"Your going to need you’re notebook.", # Homonyms"That medicine effects my ability to sleep. Have you heard of the butterfly affect?", # Homonyms"This phrase is to cherck chatGPT for spelling abilitty"# spelling]
1
2
3
4
5
6
7
8
9
10
for i inrange(len(text)):
prompt =f"""请校对并更正以下文本,无需输出原始文本,只输出纠正后的文本,且保持语种不变。
如没有发现任何错误,请说“未发现错误”。
例如:
输入:I are happy.
输出:I am happy.
```{text[i]}```""" response = get_completion(prompt)
print(i, response)
0 The girl with the black and white puppies has a ball.
1 未发现错误。
2 It's going to be a long day. Does the car need its oil changed?
3 Their goes my freedom. They're going to bring their suitcases.
4 输出:You're going to need your notebook.
5 That medicine affects my ability to sleep. Have you heard of the butterfly effect?
6 This phrase is to check chatGPT for spelling ability.
text =f"""
Got this for my daughter for her birthday cuz she keeps taking mine from my room. Yes, adults also like pandas too. She takes \
it everywhere with her, and it's super soft and cute. One of the ears is a bit lower than the other, and I don't think that was \
designed to be asymmetrical. It's a bit small for what I paid for it though. I think there might be other options that are bigger for \
the same price. It arrived a day earlier than expected, so I got to play with it myself before I gave it to my daughter.
"""
I got this for my daughter's birthday because she keeps taking mine from my room. Yes, adults also like pandas too. She takes it everywhere with her, and it's super soft and cute. However, one of the ears is a bit lower than the other, and I don't think that was designed to be asymmetrical. It's also a bit smaller than I expected for the price. I think there might be other options that are bigger for the same price. On the bright side, it arrived a day earlier than expected, so I got to play with it myself before giving it to my daughter.
1
2
3
response ="""
I got this for my daughter's birthday because she keeps taking mine from my room. Yes, adults also like pandas too. She takes it everywhere with her, and it's super soft and cute. However, one of the ears is a bit lower than the other, and I don't think that was designed to be asymmetrical. It's also a bit smaller than I expected for the price. I think there might be other options that are bigger for the same price. On the bright side, it arrived a day earlier than expected, so I got to play with it myself before giving it to my daughter.
"""
text =f"""
Got this for my daughter for her birthday cuz she keeps taking mine from my room. Yes, adults also like pandas too. She takes \
it everywhere with her, and it's super soft and cute. One of the ears is a bit lower than the other, and I don't think that was \
designed to be asymmetrical. It's a bit small for what I paid for it though. I think there might be other options that are bigger for \
the same price. It arrived a day earlier than expected, so I got to play with it myself before I gave it to my daughter.
"""