DI&A Slides: Data Lake vs. Data Warehouse

The First Step in Information Management
www.firstsanfranciscopartners.com
Produced by:
MONTHLY SERIES
Brought to you in partnership with:
February 2, 2017
Data Lake vs. Data Warehouse

Topics for Today’s Webinar
pg 2© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
§ Defining the Data Lake and Data Warehouse
§ Key differences between the Data Lake and Data Warehouse
§ How to optimize the Data Lake
§ How to optimize the Data Warehouse
§ Sample Data Lake and Data Warehouse architectures (+ use cases)
§ How a Data Lake can solve the problems of a Data Warehouse
§ Key findings and takeaways
§ Wrap-up
Combine?

Poll
Which type of data repository does your organization currently have and actively use?
§ Data Lake
§ Data Warehouse
§ Both a Data Lake and Data Warehouse
§ Neither 
If your organization has a data repository, are there plans to enhance (i.e., improve,
streamline and/or upgrade or replace) it in 2017? 
§ Yes, we will likely make some changes this year.
§ No, we are not likely to make any changes this year.
§ Not sure what we'll do this year.

pg 4
Defining the Data Lake and Data Warehouse (Gartner)
§ A Data Warehouse is a storage
architecture designed to hold
data extracted from transaction
systems, operational data stores
and external sources. The
warehouse then combines that
data in an aggregate, summary
form suitable for enterprise-
wide data analysis and reporting
for predefined business needs.
§ A Data Lake is a collection of
storage instances of various
data assets. These assets are
stored in a near-exact, or
even exact, copy of the
source format and are in
addition to the originating
data stores.
© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
BIG
DATA

Defining the Data Lake and Data Warehouse
Think of a Data Mart as a store of
bottled water—it’s cleansed, packaged,
and structured for easy consumption.
The Data Lake, meanwhile, is a large
body of water in a more natural state.
The contents of the Data Lake stream
in from a source to fill the lake, and
various users of the lake can come to
examine, dive in or take samples.
James
Dixon
Pentaho CTO and
creator of the
term Data Lake

Key Differences Between the Data Lake and Data Warehouse
Analysis Source: “A Big Data Cheat Sheet: What Marketers Want to Know” by Tamara Dull

Data Lake Challenges
TBD text
Clutter
Sandbox-like
Limited # of
SMEs
Governance
not built in
Security
Privacy
Inflexible
Resource-
intensive
Over-confidence
in capabilities

Traditional Enterprise Data Warehouse Challenges
Data
Warehouse
Traditional
Data
Sources
BI / Analytical Tools
ODS
Mart 1
Mart 2
Mart 3
Mart 4
Mart n
Source
Research
Analyst
Existing
EDWIdeally based on an enterprise model,
which has proven difficult
Stage
Source
Source
Source
Source
Source
Source
Source
Source Adding new data and subjects
takes a long time
When the desired number of users is
achieved, performance Is not acceptable
Without a lot of governance, usability
rarely meets expectations
Without a the new data I need, I am
better off gathering and storing it myself
Analyst
Analyst
Analyst
Research
Scalability is limited w/o significant
investment in migration and
infrastructure
HADOOP requires training
and tends to silo

Use Cases: Business Strategy Drives Everything in the Data Lake*
pg 9© 2017 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
§ Cost Center (i.e. not
designed for revenue
generation)
− Analytical tools for internal use
− Elastic computing for internal
infrastructure optimization
§ Operational Differentiator
(i.e. technology is used to
differentiate offering)
− Cloud and analytics are supporting
the product offering
− Analytics are used to provide
differentiating features
Example: Johnson and Johnson
§ Revenue Multiplier (i.e. cloud and
Big Data analytics integrated into
the business offerings)
− Elastic infrastructure is offered to others
companies as an infrastructure service
(marketing analysis, back-up, analytics
testing, etc.)
− Big Data analytics are sold as a data service
Example: eBay, Netflix Example: NY Stock Exchange
Starting point for the roles and responsibility is to decide the purpose of the Data Lake *Orbis Technologies
Large Medical Device
Manufacturer

Use Case: Data Warehouse Architecture to Address
Massive Market Changes
Data “Warehouse”
Analytical
Data
Operational
Data
Apps
Internal
DataSourcesExternal
DataSources
(e.g.,Axciom)
(e.g.,MDM)
Others
Sources
Collection
Cleansing
Integration
Enterprise
Data
Repositories
Presentation
And
Delivery
ETL
(Extract,Transform,Load)
ETL
(Extract,Transform,Load)
Mart
Mart
Delivery/PresentationLayer
End-Users
Reporting
Query/
Analytics
Standard
Extracts
Operational
Data
Agent and Policyholder Retention
Independent Agents -
Easy to switch
allegiances
Sales and
Marketing
Customer - Widely
changing demographics
and markets
Underwriting
Claims
Management

But This is Not About Choosing
Usage basis
What
Happened?
Why did it
happen?
What will
happen?
Make it
happen by
itself
What do I
want to
happen?
What
should we
do next?
Perceived
Maturity
Reporting Analyzing Predictive
Operation-
alize
Adaptive Foresight
Capability Survival Defined
Characteristics Batch ETL ETL / EAI
Web
Services
Streaming
Managed Optimized / Automous
Data Insight Maturity

But This is Not About Choosing
Usage basis
What
Happened?
Why did it
happen?
What will
happen?
Make it
happen by
itself
What do I
want to
happen?
What
should we
do next?
Perceived
Maturity
Reporting Analyzing Predictive
Operation-
alize
Adaptive Foresight
Capability Survival Defined
Characteristics Batch ETL ETL / EAI
Web
Services
Streaming
Managed Optimized / Automous
Data Insight Maturity
“DW-ish” “DL-ish”

How to Optimize the Data Lake
§ Design for rapid and scalable ingestion.
§ Know, govern and protect your data.
§ Remove data silos and mitigate chaos with pervasive data quality/management.
§ Ensure your Data Lake is not isolated or crudely bolted onto existing infrastructure.
§ Monitor performance (ensure no degrading).
§ Assess your users’ capabilities and identify gaps and critical skills required to
expertly swim in the lake. Then create a skills development program to cultivate
required skills and bridge any gaps.
Tips

How to Optimize the Data Warehouse
§ Understand and reinforce the strengths of the four essential components
of the Data Warehouse, including:
− Focus on supporting self-service and ease of use of Metadata
− Strengthen the Structure by boosting consistency and understanding
of performance
− Establish trust by ensuring Quality
− Improve accuracy and reduce risk with Governance
Tips

Data Warehouse blended with Data Lake
How a Data Lake Can Solve the Problems of a Data Warehouse
§ Lack of agility
§ Performance
§ Hard to extend
§ Structured data only
§ Enables experimentation
§ Satisfies timing and
turnaround issues
§ Allows unstructured data

Data Warehouse blended with Data Lake
How a Data Lake Can Solve the Problems of a Data Warehouse
§ Lack of agility
§ Performance
§ Hard to extend
§ Structured data only
§ Enables experimentation
§ Satisfies timing and
turnaround issues
§ Allows unstructured data
Data Lake technology to leap frog Data Warehouse
Organizations without
data warehouse simply
start with a Data Lake
Or organizations that need to
evolve their warehouse replace
it with a Data Lake

Key Findings and Takeaways
§ Over time, this is not an “either/or” debate.
§ Understand your overall requirements (based on
business needs) and BLEND Data Warehouse and Data
Lake capabilities, as well as the other structures related
to these, for a best fit or logical data warehouse.
Tips

Sample Architecture pg 18
Wrap-up – When in Doubt …
Business
Need
Operational Managerial Analysis Analytics
Business
Area
Business
Area
Business
Area
Reports
Reports
Ad hoc
BI
Alert s
Trends
Predictive
Analytics
Models
Metadata
Ingestion
Data
Lake
Data
Ware-
house
Sandbox
Landing
BI
Analytics

Sample Architecture
Ingestion
Data
Lake
Data
Ware-
house
Sandbox
Landing
BI
Analytics
pg 19
Wrap-up – When in Doubt …
Business
Need
Operational Managerial Analysis Analytics
Business
Area
Business
Area
Business
Area
Reports
Reports
Ad hoc
BI
Alert s
Trends
Predictive
Analytics
Models
Document the business needs,
drivers, metadata and actions
around data.
Analyze the characteristics,
patterns and classify if the lake
or warehouse or ??? supports
the need.
Metadata
Gather together the various
groupings of how you can
satisfy your data insights needs.
Craft your version of
the best fit “logical
Data Warehouse.”

Q & A

pg 21
Thank you!
See you Thursday, March 2 for the next webinar,
Descriptive, Prescriptive and Predictive Analytics
John Ladley @jladley
john@firstsanfranciscopartners.com
Kelle O’Neal @kellezoneal
kelle@firstsanfranciscopartners.com
© 2016 First San Francisco Partners www.firstsanfranciscopartners.com

DI&A Slides: Data Lake vs. Data Warehouse

More Related Content

What's hot

What's hot (20)

Similar to DI&A Slides: Data Lake vs. Data Warehouse

Similar to DI&A Slides: Data Lake vs. Data Warehouse (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

DI&A Slides: Data Lake vs. Data Warehouse