202406 - Cape Town Snowflake User Group - LLM & RAG.pdf

© 2023 Snowflake Inc. All Rights Reserved
CAPE TOWN
SNOWFLAKE
USER GROUP
Chapter Leader

© Snowflake Inc. All Rights Reserved
Agenda
Welcome to Cape Town Snowflake User Group
What is Snowflake?
Introduction to LLM & RAG
Even more AI Features
Closing Remarks
2

© Snowflake Inc. All Rights Reserved 3
About the User Group
Get frequent updates on the latest Snowflake features and product roadmap from
subject matter experts.
Stay Current with Snowflake
Review real Snowflake use cases, methodologies, and 3rd party technologies with
your peers.
Discuss Best Practices
Meet other community members across many stages in their Snowflake journeys.
Grow Your Professional Network

Chapter Leaders
4
Douglas Day Reza du Plooy

Please download and install the
Slido app on all computers you
use
Why are you
attending the
Snowflake User
Group?
ⓘ Start presenting to display the poll results on this slide.

Call for Speakers
Discuss effective strategies, tips, and lessons learned from your experience with
Snowflake to help others avoid common pitfalls and maximize their success.
Best Practices & Lessons Learned
Share real-world examples of how you've leveraged Snowflake to solve business
challenges, improve performance, or achieve specific outcomes
Cases Studies & Success Stories
Present unique and creative ways you've used Snowflake's features and capabilities to
drive innovation within your organization.
Innovative Use Cases

Upcoming Events
➔ 17th July: Johannesburg User Group, W17 Rosebank
➔ 21st August: Cape Town User Group, Innovation City

Join the Chapter
Go to:
usergroups.snowflake.com
This is the homepage for all
Snowflake user groups!
Locate the chapter:
South Africa
Join other Snowflake users
within the space.
Become a member:
Click ‘join’ and you’re in!
Get invited to upcoming
meetings and access exclusive
content!
Get Updates About Upcoming Meetings
1 2 3

Find Your Local Chapter

Today’s Speakers
10
Director, Data Engineer
datalab
Douglas Day
Engineer
Snowflake
Cliff Wachiuri

Introduction to Snowflake

One Platform

LLM & RAG in Snowflake

Breakdown
Introduction to using LLM
Retrieval Augmented Generation
18

• Large Language Model (LLM)
• Hallucination
• Retrieval Augmented Generation (RAG)
• Tokens
• Chunking
• Embedding
• Prompt
Key Terms

What Is It
An intelligent, fully managed service that
hosts and serves industry-leading AI
models, LLMs and vector functions
Why Use It
Quickly and securely analyze your data and
build AI applications contextualized with
your enterprise data
How To Use It
Access the power of Snowflake Cortex via
serverless SQL / Python functions or as part
of LLM-powered experiences such as
Document AI and Snowflake Copilot
Snowflake Cortex
Llama 2
forecast
General-Purpose
Specialized
complete
Snowflake Copilot
Document AI
complete()
embed()
...
translate()
forecast()
...
Serverless Functions

Snowflake for GEN AI & LLMS

Use Case: Call Centre Analytics
Customer Feedback Data
LLM Functions
Snowsight
(Snowflake User Interface)

LARGE LANGUAGE MODELS
USE CASES
FUNCTION SUMMARIZING INFERRING TRANSFORM EXPAND
COMPLETE    
SUMMARIZE 
EXTRACT_ANSWER 
SENTIMENT 
TRANSLATE 

SUMMARIZE
The SUMMARIZE function returns a summary of the given English text.

EXTRACT_ANSWER
The EXTRACT_ANSWER function extracts an answer to a given question from a text document.
The document may be a plain-English document or a string representation of a semi-structured (JSON) data
object.

SENTIMENT
The SENTIMENT function returns sentiment as a score between -1 to 1 (with -1 being the most negative and 1
the most positive, with values around 0 neutral) for the given English-language input text.

TRANSLATE
The TRANSLATE function translates text from the indicated or detected source language to a target language.

TRANSLATE
LANGUAGE Code
English 'en'
French: 'fr'
German 'de'
Italian 'it'
Japanese 'ja'
Korean 'ko'
LANGUAGE Code
Polish 'pl'
Portuguese 'pt'
Russian 'ru'
Spanish 'es'
Swedish 'sv'

COMPLETE
The COMPLETE function, given a prompt will generate a response (completion) using your choice of
supported language model.

COMPLETE

Developer Model Parameters Use Cases Size Tokens
Snowflake snowflake-arctic 17B SQL generation, coding and instruction Medium 4,096
Mistral AI mistral-large 40B NLP tasks, large-scale data processing Large 32,000
Meta llama3-70b 70B Complex tasks, research Large 4096
Reka AI reka-flash 20B
multilingual 32 languages, writing product descriptions or
blog posts, coding, and extracting answers
Medium 100,000
Mistral AI mixtral-8x7b 56B text generation, classification, and question answering Medium 32,000
Meta llama3-8b 8B text classification, summarization, and sentiment analysis Small 8,000
Meta llama2-70b-chat 70B text classification, summarization, and sentiment analysis Medium 8,000
Mistral AI mistral-7b 7B
simplest summarization, structuration, and question
answering tasks that need to be done quickly
Small 32,000
Google Deepmind gemma-7b 7B suitable for simple code and text completion tasks Small 8,000

PROMPT ENGINEERING

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Constraint or Guideline
• Examples (Few-Shot)
• Tone & Style
• Format Specification
• Subject Matter
• Target Audience
• Data

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
Definition: The main directive or question posed to the
model.
Purpose: Sets the context and intention for the
response.
Example: "Write a poem about the sea.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
Definition: The specific persona or perspective
assigned to the model.
Purpose: Provides the model with a context or point of
view to enhance the quality and relevance of the
response.
Example: "As an experienced sailor, write a poem
about the sea."

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
Definition: Additional information or background that
helps the model understand the setting or topic.
Purpose: Provides clarity and relevance, ensuring the
model generates a coherent and accurate response.
Example: “The poem is based on a sailors experience
of looking at a moonlit night and the vastness of the
ocean”

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
Definition: Specific rules or limitations that the model
should follow while generating the response.
Purpose: Ensures the output meets certain criteria,
such as length, format, or tone.
Example: "Write a summary of this article in less than
150 words.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: Sample inputs and outputs provided to the
model to illustrate the desired format or style.
•Purpose: Helps the model learn from patterns and
generate similar responses.
•Example: Specific format of output

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: Instructions on the desired tone, style, or
register of the response.
•Purpose: Ensures the output aligns with the expected
mood or voice.
•Example: "Explain quantum physics in a
conversational and humorous tone.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: Details about how the response should be
structured or formatted.
•Purpose: Ensures the response adheres to specific
structural requirements, such as bullet points, lists, or
sections.
•Example: "List the pros and cons of remote work in
bullet points.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: The main topic or theme that the prompt
focuses on.
•Purpose: Guides the content and scope of the
response.
•Example: "Discuss the impact of climate change on
polar bears.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: Information about who the response is
intended for.
•Purpose: Tailors the response to the knowledge level,
interests, or needs of the audience.
•Example: "Explain the benefits of renewable energy to
a group of high school students.“

PROMPT ENGINEERING
• Instruction
• Role Persona
• Context
• Tone & Style
• Subject Matter
• Target Audience
• Data
•Definition: Specific data or text that the model should
use as a basis for the response.
•Purpose: Provides concrete information that the model
needs to process or refer to.
•Example: "Using the following data, write a report on
sales performance: [data]."

PROMPT ENGINEERING
INFERRING
As a call centre analyst
Return these emotions as a comma-delimited list
only
Role
Context
Instruction
analysing call centre transcripts
Constraint
Identify 5 emotions expressed by the caller

PROMPT ENGINEERING
TRANSFORMING
Return your answer as a single word: 'True' or
'False'
Role
Instruction
Context
Determine if the customer expressed frustration
Constraint
in the call centre transcript.

PROMPT ENGINEERING
SUMMARIZING
As a call centre analyst, provide unbiased advice
Role
Instruction
Context
Summarize the relevant product defects mentioned
in the call centre transcript
Focus exclusively on product defects only
Format
Return the message in this format: 'Here is the
summary of product defects: <product>: <defect>
Example
Example output: Buckles: “The buckles are stuck”,
“Zips: zips are broken”

PROMPT ENGINEERING
EXPANDING
As a professional customer service assistant.
Role
Instruction
Constraint
Format
Send a follow-up email regarding the recent call centre
interaction.
Summarize the issue, resolution, and any next steps from
the call centre transcript.
Keep the email concise.
Sign the email as AI customer agent.
Maintain a formal and professional tone throughout the
email.
Ensure the email is properly formatted as a professional
follow-up correspondence.
Tone

PROMPT ENGINEERING
TRANSFORM
Assume the role of a specialist language translator
format the response as JSON ONLY, with the following
structure: "language" and "conversation" (consisting of
"Person", "Original", “English“, “Afrikaans”).
Role
Context
Instruction
1: Identify the language of the transcript
2: Identify the Agent and the Customer within the transcript
3: Translate the conversation into English and Afrikaans
Format
Translate the call centre transcript step by step

PROMPT ENGINEERING
REASONING
Return the message in this format only: "Here is the
summary of product defects: <product>: <defect> <original
quantity > (<defect quantity>) <defect percentage>%"
examples" Buckles: The buckles are stuck 10(5) 50%
Role
Context
Instruction
Analyze the call centre transcript step by step
1: Identify the original number of products purchased that
had a defect
2: Determine how many of the products have defects and
the percentage
3: Summarise the list of product and defects.
Format

DEMO

RETRIVAL
AUGMENTED
GENERATION

Use Case: Document Analytics
PDF Documents
LLM Functions
Snowsight
(Snowflake User Interface)

Use Case: Document Analytics
PDF Documents
Python LLM Functions
Streamlit App

WHAT IS RAG?
RETRIEVAL AUGMENTED GENERATION

CHUNKING
• Chunk Size
• Semantic Coherence
• Overlap
• Metadatag
• Relevance
• Efficiency
Key Considerations

EMBEDDING
• Vector embeddings are a method of
representing data as vectors (lists of
numbers) in a continuous vector space.
• Embeddings transform high-dimensional
data into a lower-dimensional space while
preserving the essential properties and
relationships of the original data.

EMBEDDING
FUNCTIONS
FUNCTION EMBEDDING MODEL
EMBED_TEXT_768 snowflake-arctic-embed-m
EMBED_TEXT_768 e5-base-v2
EMBED_TEXT_1024 nv-embed-qa-4

EMBEDDING
WHICH EMBEDDING?
ASPECT LOW HIGH
Computational Complexity Lower Higher
Storage Requirements Lower Higher
Risk of Overfitting Lower Higher
Expressiveness Lower Higher
Information Capture May lose detail Capture more nuance
Task Complexity Simple Complex

Safe Harbor and Disclaimers
REV 01.05.24
© 2024 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature and service names mentioned in the Materials are registered
trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used in the Materials are for identification
purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be sponsored or endorsed by, any such holder(s).
Other than statements of historical fact, all information contained in these materials
and any accompanying oral commentary (collectively, the “Materials”), including
statements regarding (i) Snowflake’s business strategy, plans or priorities, (ii)
Snowflake’s new or enhanced products, services, and technology offerings,
including those that are under development or not generally available, (iii) market
growth, trends, and competitive considerations, (iv) our vision for Snowpark, the
Data Cloud, and industry-specific Data Clouds, including the expected benefits and
network effects of the Data Cloud; and (v) the integration, interoperability, and
availability of Snowflake’s products, services, or technology offerings with or on
third-party platforms or products, are forward-looking statements. These forward-
looking statements are subject to a number of risks, uncertainties and assumptions,
including those described under the heading “Risk Factors” and elsewhere in the
Annual Reports on Form 10-K and the Quarterly Reports on Form 10-Q that
Snowflake files with the Securities and Exchange Commission. In light of these
risks, uncertainties, and assumptions, the future events and trends discussed in the
Materials may not occur, and actual results could differ materially and adversely
from those anticipated or implied in the forward-looking statements. As a result, you
should not rely on any forwarding-looking statements as predictions of future
events. Forward-looking statements speak only as of the date the statements are
first made and are based on information available to us at the time those
statements are made and/or management's good faith belief as of that time. Except
as required by law, we undertake no obligation, and do not intend, to update the
forward-looking statements in these Materials.
Any future product or roadmap information (collectively, the “Roadmap”) is
intended to outline general product direction. The Roadmap is not a
commitment, promise, or legal obligation for Snowflake to deliver any future
products, features, or functionality; and is not intended to be, and shall not
be deemed to be, incorporated into any contract. The actual timing of any
product, feature, or functionality that is ultimately made available may be
different from what is presented in the Roadmap. The Roadmap information
should not be used when making a purchasing decision. In case of conflict
between the information contained in the Materials and official Snowflake
documentation, official Snowflake documentation should take precedence
over these Materials. Further, note that Snowflake has made no
determination as to whether separate fees will be charged for any future
products, features, and/or functionality which may ultimately be made
available. Snowflake may, in its own discretion, choose to charge separate
fees for the delivery of any future products, features, and/or functionality
which are ultimately made available.
The Materials may contain information provided by third-parties. Snowflake
has not independently verified this information, and usage of this information
does not mean or imply that Snowflake has adopted this information as its
own or independently verified its accuracy.

In Dev Private Public GA
Soon
Built with Meta Llama 3 and Mistral Large models

Cortex Analyst - Talk to Your Data From Any App
In Dev Private Public
Soon
GA
Cortex Analyst
Snowflake Databases
Semantic
Model
REST
API

Document AI
WHAT IS IT
Fully managed workflow that uses Arctic-TILT for
efficient extraction of text, table values, and hand
written content from PDFs and other unstructured
documents
WHY USE IT
Use industry-leading LLM for higher efficiency,
lower cost of manual labor and lower human error
in document processing
HOW TO USE IT
● Business user: Test and fine-tune (if needed)
in no-code UI
● Data team: Efficiently extract fields to
Snowflake tables from multiple documents in
batch using SQL

Cortex AI: Serverless Fine-Tuning
WHAT IS IT
Serverless fine-tuning for subset of Mistral and
Llama 3 LLMs available in Cortex AI.
WHY USE IT
● Customize LLMs securely and effortlessly to
increase model accuracy and performance for
use-case specific tasks.
● Manage access and governance of custom
LLMs with Snowflake Model Registry.
HOW TO USE IT
Fine-tune models via a function or directly in the
AI/ML Studio. Easily access the fine-tuned models
through the COMPLETE function.
SNOWFLAKE.CORTEX.FINETUNE(
'CREATE',
<model_name>,
<base_model>,
<training_data>,
<validation_data>
);

Cortex Search – fully managed indexing and retrieval
Query:
“What are the new ECB
capital requirements?”
Filter:
“Year > 2022”
User Query Hybrid Retrieval
Result Fusion
and Reordering
Keyword /
Lexical Lookup
Semantic
Lookup
RESULT_1:
weighted average of Pillar
2 requirements set at 1.1%
in 2023, unchanged from
last year…”
RESULT_2:
“Overall CET1 capital
requirements and guidance
increased from 10.7% to
11.1%...”
RESULT_K:”...”
Result Set
Documents

CLOSING REMARKS

RESOURCES
TO GET STARTED
Quick Start Guide to
Snowflake Artic
Large Language Model
Functions
Snowflake Developers
LLM Walk-Through

Competition Time
https://app.sli.do/event/vEtGVF45kLyjfPGwvVwCNP
6105041

use
When was Snowflake
originally founded?

use
What was Snowflakes
IPO evaluation?

use
Which of these is not
a Snowflake Cortex
function?

use
Which of these LLM's
in not directly
supported in
Snowflake?

use
What does RAG
stand for?

use
Which of these is
NOT part of the RAG
process?

use
When is the next
Snowflake Cape
Town User Group?

Closing Remarks
Look out for the survey to provide us with feedback and topic requests.
Go to usergroups.snowflake.com to join the chapter, and discover other chapters.
Email usergroups@snowflake.com if you want to become a chapter leader!
Please hang around for drinks.

THANK YOU!

202406 - Cape Town Snowflake User Group - LLM & RAG.pdf

Recommended

Recommended

More Related Content

Similar to 202406 - Cape Town Snowflake User Group - LLM & RAG.pdf

Similar to 202406 - Cape Town Snowflake User Group - LLM & RAG.pdf (20)

Recently uploaded

Recently uploaded (20)

202406 - Cape Town Snowflake User Group - LLM & RAG.pdf