MySQL 5.7 - What's new and How to upgrade

Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7 News
Abel Flórez
Technical Account Manager
abel.florez@oracle.com

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied
upon in making purchasing decisions. The development, release, and timing of any
features or functionality described for Oracle’s products remains at the sole discretion of
Oracle.
2

Agenda
News in MySQL 5.7
Upgrading to MySQL 5.7
1
2
3

MySQL 5.7 is GA!
4
Enhanced InnoDB: faster online & bulk
load operations
Replication Improvements (incl. multi-
source, multi-threaded slaves...)
New Optimizer Cost Model: greater
user control & better query
performance
Performance Schema Improvements
MySQL SYS Schema
Performance & Scalability Manageability
3 X Faster than MySQL 5.6
Improved Security: safer initialization,
setup & management
Native JSON Support
And many more new features and enhancements. Learn more at: dev.mysql.com

MySQL 5.7 Sysbench Benchmark: SQL Point Selects
3x Faster than MySQL 5.6
4x Faster than MySQL 5.5 1,600,000 QPS
8 16 32 64 128 256 512 1,024
0
500,000
1,000,000
1,500,000
2,000,000
MySQL 5.7:Sysbench OLTP Read Only (SQL Point Selects)
MySQL 5.7
MySQL 5.6
MySQL 5.5
Connections
QueriesperSecond
Intel(R) Xeon(R) CPU E7-8890 v3
4 sockets x 18 cores-HT (144 CPU threads)
2.5 Ghz, 512GB RAM
Linux kernel 3.16
5

82% Faster than MySQL 5.6
8 16 32 64 128 256 512 1,024
0
20,000
40,000
60,000
80,000
100,000
120,000
MySQL 5.7:Sysbench OLTP Read Only
MySQL 5.7
MySQL 5.6
MySQL 5.5
Connections
Connect/sec
2.5 Ghz, 512GB RAM
Linux kernel 3.16
100K Connect / Sec
MySQL 5.7 Sysbench Benchmark: Connection Requests
6

MySQL 5.7 Sysbench Benchmark: OLTP Read Only
8 16 32 64 128 256 512 1,024
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
MySQL 5.7:Sysbench OLTP Read Only
MySQL 5.7
MySQL 5.6
MySQL 5.5
Connections
QueriesperSecond
2.5 Ghz, 512GB RAM
Linux kernel 3.16
~ 1,000,000 QPS
7

MySQL 5.7 Sysbench Benchmark: OLTP Read Write
1.5x Faster than MySQL 5.6
8 16 32 64 128 256 512 1,024
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
MySQL 5.7:Sysbench OLTP Read Write
MySQL 5.7
MySQL 5.6
MySQL 5.5
Connections
QueriesperSecond
2.5 Ghz, 512GB RAM
Linux kernel 3.16
8

• Optimizer and Parser refactoring
– Readability, maintainability and stability
– Separate parsing, optimizing, execution
stages
– Easier feature additions, with lessened risk
• New hint framework
– Easier to manage
– With support for additional new hints
• Improved JSON EXPLAIN
• EXPLAIN for running thread
• New Cost based Optimizer
– Easier to extend
– Configurable and tunable
• mysql.server_cost and mysql.engine_cost tables
• API for where data resides: on disk or in cache
• InnoDB for internal temp tables
• Better ONLY_FULL_GROUP_BY mode
• Many specific new optimizations
• Generated Columns
MySQL 5.7: Optimizer Improvements
9
Queries execute faster, while using less CPU and disk space!

Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10, CPU bound)
Q3 Q7 Q8 Q9 Q12
0
20
40
60
80
100
MySQL 5.6
MySQL 5.7
5 out of 22 queries get a much improved query plan (others remain the same)
Executiontimerelativeto5.6(%)
10

Q2 Q18
0
20
40
60
80
100
CPU bound
5.6
5.7
Executiontimerelativeto5.6(%)
Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10)
2 out of 22 queries get a significantly improved query plan
(others remain the same)
Q2 Q18
0
20
40
60
80
100
Disk bound
5.6
5.7
Executiontimerelativeto5.6(%) 11

MySQL 5.7: Query Rewrite Plugin
• New pre and post parse query rewrite APIs
– Users can write their own plug-ins
• Provides a post-parse query plugin
– Rewrite problematic queries without the need to make application changes
– Add hints
– Modify join order
– Many more …
• Improve problematic queries from ORMs, third party apps, etc
• Eliminates many legacy use cases for proxies
12

MySQL 5.7: Optimizer - Cost Info in JSON EXPLAIN
• Expanded JSON EXPLAIN
– Now includes all available cost info
– Used for Visual Explain In MySQL
Workbench
•
{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "200.40"
},
"table": {
"table_name": "nicer_but_slower_film_list",
"access_type": "ALL",
"rows_examined_per_scan": 992,
"rows_produced_per_join": 992,
"filtered": 100,
"cost_info": {
"read_cost": "2.00",
"eval_cost": "198.40",
"prefix_cost": "200.40",
"data_read_per_join": "852K"
},
"used_columns": [
"FID",
"title",
"description",
"category",
"price",
"length",
"rating",
"actors"
],
...
{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "200.40"
},
"table": {
"table_name": "nicer_but_slower_film_list",
"access_type": "ALL",
"rows_examined_per_scan": 992,
"rows_produced_per_join": 992,
"filtered": 100,
"cost_info": {
"read_cost": "2.00",
"eval_cost": "198.40",
"prefix_cost": "200.40",
"data_read_per_join": "852K"
},
"used_columns": [
"FID",
"title",
"description",
"category",
"price",
"length",
"rating",
"actors"
],
...
13

MySQL 5.7: JSON
• Native JSON data type
– Native internal binary format for efficient processing & storage
• Built-in JSON functions
– Allowing you to efficiently store, search, update, and manipulate Documents
• JSON Comparator
– Allows for easy integration of Document data within your SQL queries
• Indexing of Documents using Generated Columns
– InnoDB supports indexes on both stored and virtual Generated Columns
– New expression analyzer automatically uses the best “functional” index available
• New inline syntax for easy SQL integration
•
14

• 5.7 supports functions to CREATE, SEARCH, MODIFY and RETURN JSON
values:
MySQL 5.7: JSON Functions
15
JSON_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
https://dev.mysql.com/doc/refman/5.7/en/json-functions.html
• 5.7 supports functions to CREATE, SEARCH, MODIFY and RETURN JSON
values:
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()

MySQL 5.7: JSON and Text Datatype Comparison
# With feature column as JSON type
SELECT DISTINCT
feature->"$.type" as json_extract
FROM features;
+--------------+
| json_extract |
+--------------+
| "Feature" |
+--------------+
1 row in set (1.25 sec)
Unindexed traversal of 206K documents
# With feature column as TEXT type
SELECT DISTINCT
feature->"$.type" as json_extract
FROM features;
+--------------+
| json_extract |
+--------------+
| "Feature" |
+--------------+
Explanation: Binary format of JSON type is very efficient at searching. Storing as TEXT
performs over 10x worse at traversal.
16

MySQL 5.7: Functional Indexes with JSON
17
ALTER TABLE features ADD feature_type VARCHAR(30) AS (JSON_UNQUOTE(feature->'$.type'));
Query OK, 0 rows affected (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 0
ALTER TABLE features ADD INDEX (feature_type);
Query OK, 0 rows affected (0.73 sec)
Records: 0 Duplicates: 0 Warnings: 0
SELECT DISTINCT feature_type FROM features;
+--------------+
| feature_type |
+--------------+
| "Feature" |
+--------------+
From table scan on 206K documents to index scan on 206K materialized values
Meta data change only (FAST).
Does not need to touch table..
Creates index only, does not
touch row data.
Down from 1.25 sec to 0.06
sec
Create table features (
id int primary key,
feature json
)

Memory Instrumentation
• Aggregates statistics by
– Type of memory used
(caches, internal buffers,
…)
– Thread/account/user/host
indirectly performing
the memory operation
• Attributes include
– Memory used (bytes)
– Operation counts
– High/Low Water Marks
Statement
Instrumentation
• Stored Procedures
• Stored Functions
• Prepared Statements
• Transactions
Additional Information
• Replication slave status
• MDL lock instrumentation
• Status and variables per
thread
• Server stage tracking
• Track long running SQL
• Improved configuration and
ease-of-use
• All while reducing total
footprint and overhead
MySQL 5.7: Performance Schema
18

MySQL 5.7: SYS Schema
Helper objects for DBAs, Developers and Operations staff
• Helps simplify DBA / Ops tasks
- Monitor server health, user, host statistics
- Spot, diagnose, and tune performance issues
• Easy to understand views with insights into
- IO hot spots, Locking, Costly SQL statements
- Schema, table and index statistics
• SYS is similar to
- Oracle V$ catalog views
- Microsoft SQL DMVs (Dynamic Mgmnt Views)
19

• Replaced custom code with Boost.Geometry
– For spatial calculations
– For spatial analysis
– Enabling full OGC compliance
– We’re also Boost.Geometry contributors!
• InnoDB R-tree based spatial indexes
– Full ACID, MVCC, & transactional support
– Index records contain minimum bounding box
• GeoHash
• GeoJSON
• Helper functions such as ST_Distance_Sphere() and ST_MakeEnvelope()
MySQL 5.7: GIS Improvements
20

• Native Partitioning
– Eliminates previous limitations
– Eliminates resource usage problems
– Transportable tablespace support
• Native Full-Text Search
– Including full CJK support!
• Native Spatial Indexes
• Transparent page compression
• Support for 32K and 64K pages
– Use with transparent page compression for
very high compression ratios
• General TABLESPACE support
– Store multiple tables in user defined shared
tablespaces
• Support for MySQL Group Replication
– High priority transactions
• Improved support for cache preloading
– Load your hottest data loaded at startup
• Configurable fill-factor
– Allows for improvements in storage
footprint
• Improved bulk-data load performance
• Resize the InnoDB Buffer Pool online
MySQL 5.7: InnoDB Improvements
21

MySQL 5.7: InnoDB Compression
Thank you, SanDisk Fusion-io
• Transparent Page Level Compression
– Happens transparently in background threads
– Managed entirely within the IO layer
– Uses sparse file and "hole punching" support in OS kernels and File Systems
• Reduces IO
– Improves MySQL performance
– Improves storage efficiency
– Reduces write cycles, thus increasing SSD lifespan
• Applies to all InnoDB data, including the system tablespace and UNDO logs
22

• AES 256 Encryption now the default
• Password rotation policies
– Can be set globally, and at the user level
• Deployment: enable secure unattended
install by default
– Random password set on install
– Remove anonymous accounts
– Deployment without test account, schema,
demo files
• Easier instance initialization and setup:
mysqld –initialize
• New detection and support for systemd
• SSL
– Enabled by default
– Auto-detection of existing keys and certs
– Auto generation of keys and certs when
needed
– New helper utility: mysql_ssl_rsa_setup
– New --require_secure_transport option to
prevent insecure communications
– Added SSL support to binary log clients
• Extended Proxy User Support
– Added Built-in Authentication Plugins support
for Proxy Users
– Allows multiple users to share a single set of
managed privileges
MySQL 5.7: Security Improvements
23

MySQL 5.7: Server-Side Statement Timeouts
Thank you Davi Arnaut!
• Server side statement timeouts
– Global for server, per session, or for individual SELECT statements
–
• Expanded to Windows and Solaris, restricted by removing USER option
SELECT /*+ MAX_EXECUTION_TIME(1000) */ *
FROM my_table;
24

• GTID enhancements
– On-line, phased deployment of GTIDs
– Binary logging on slave now optional
• Enhanced Semi-synchronous replication
– Write guaranteed to be received by slave
before being observed by clients of the
master
– Option to wait on Acks from multiple slaves
• Multi-Source Replication
– Consolidate updates from multiple Masters
into one Slave
• Dynamic slave filters
•
• 8-10x Faster slave throughput
– Often removes slave as a bottleneck; keep
pace with master with 8+ slave threads
– Option to preserve Commit order
– Automatic slave transaction retries
–
–
MySQL 5.7: Replication Improvements
25
0%
50%
100%
150%
200%
250%
1 8 24 48
Slave Threads
Slave throughput vs. 96 Thread Master

• Multi-Source Replication
– Consolidate updates from multiple
Masters into one Slave
• Consolidated view of all shards
• More flexible topologies
• Centralized point for backups
– Compatible with Semi-Sync
Replication & enhanced MTS
• Performance Schema tables for
monitoring slave
• Online Operations: Dynamic
Replication Filters, switch master
MySQL 5.7: Replication Improvements
Slave
26
BinlogBinlog
Master 1Master 1
BinlogBinlog
Master 2Master 2
……
……
BinlogBinlog
Master NMaster N
IO 1IO 1
Relay 1Relay 1
CoordinatorCoordinator
W1W1 W2W2 …… WXWX
IO 2IO 2
Relay 2Relay 2
……
……
IO NIO N
Relay NRelay N

• Connection and Transaction routing
• Transparently improve your MySQL apps
– Transparent support for MySQL Group Replication clusters
– Transparent support for custom clusters and HA setups
• Easily extendable using plugin APIs
• Many new plugins to come – Aggregation, Binary Log, Load Balancing, …
MySQL Router
27

Extensible Architecture
MySQL Router
28Copyright © 2016, Oracle and/or its affiliates. All rights
reserved
App Servers with
MySQL Router
Binary Log?
More plugins to
come…
???
Aggregation?
MySQL HA
cluster
Harness – Logging,
thread handling ;
config, service, and
plugin management
Core – Connection
routing, transaction
routing
Plugins – Fabric cache,
Group Replication
Load Balance?

MySQL Workbench 6.3
• Performance Dashboard
– Performance Schema Reports & Graphs
• Visual Explain
• GIS Viewer
• Migration
– New: Microsoft Access
– Microsoft SQL Server, Sybase,
PostgreSQL, SQLite
• MySQL EE features
•
GA
29
• New Easy to Use Wizards for
– Fast Data Migration
– Table<->File Data Import/Export (like Excel)
– SSL Certificate Creation

• Active/Active Update Anywhere
– Conflict detection and resolution (transaction rollback)
– Optimistic State Machine Replication
• Automatic group membership management and
failure detection
– No need for server fail-over
– Elastic scale out/in
– No single point of failure
– Automatic reconfiguration
• Well integrated
– InnoDB
– GTID-based replication
– PERFORMANCE_SCHEMA
MySQL Group Replication
Application
MySQL
Nodes
Replication
Plugin
Replication
Plugin
APIAPI
MySQL
Server
MySQL
Server
Group CommsGroup Comms
labs.mysql.com
30

The Future of MySQL Scaling (HA + Sharding)
Global Data Shard 1 Shard 2
MySQL Fabric
controller cluster
SQL Queries
Server/Shard State &
Mapping
Global Group HA Group
Coordination
and Control
HA Group
Group Replication
cluster
31
Group Replication
cluster
Group Replication
cluster
MySQL
Router

MySQL Repos
• Distributions
– Oracle, Red Hat, CentOS
– Fedora
– Ubuntu, Debian
– SUSE
• Official MySQL Docker Image from Oracle
• Coming Soon
– Preconfigured Containers
– Improved support for popular DevOps
deployment tools
–
https://dev.mysql.com/downloads/repo
MySQL on GitHub
• Git for MySQL Engineering
– Fast, flexible and great for a distributed team
– Great tooling
– Large and vibrant community
• GitHub for MySQL Community
– Easy and fast code availability to the
community and to downstream projects
– Pull Requests
https://github.com/mysql
32

Agenda
News in MySQL 5.7
Upgrading to MySQL 5.7
1
2
Tuesday,
33

MySQL Upgrade – Supported Paths
• All minor upgrades like 5.7.9 to 5.7.10 or doing a jump like 5.7.9 to 5.7.11 is
supported.
• Upgrading one major (5.6 - 5.7) release level is supported. It’s
recommended upgrading to the latest 5.6 release before moving to 5.7.
• Doing long-jumps (5.1 – 5.7) is supported if you upgrade one release level
at a time.
• Direct upgrades that skip a release level (for example, upgrading directly
from MySQL 5.5 to 5.7) is not recommended.
34

MySQL Upgrade – Before you start!
• Review the “Release Notes” for target MySQL version
• Review the manual page on upgrade for this MySQL version, take special
care of sections covering:
– “Server and Status Variables and Options Added, Deprecated, or Removed …”
– “Checking Whether Tables or Indexes Must Be Rebuilt”
– “Changes Affecting Upgrades to MySQL 5.N”
• Create a plan for your upgrade/downgrade path
• Create a backup of your database
•
35
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html

MySQL Upgrade – In-place Upgrade
1. Do all necessary preparations/plans as explained earlier
2. Do a graceful shutdown of MySQL (innodb_fast_shutdown=0)
3. Do upgrade, replace the old binaries with the new ones
4. Start new MySQL Server using new binaries
5. Run mysql_upgrade binary to upgrade internal mysql repository
6. Done!
36
http://mysqlserverteam.com/upgrading-directly-from-mysql-5-0-to-5-7-using-an-in-place-upgrade/
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html#upgrade-procedure-inplace

MySQL Upgrade – Logical Upgrade
2. Do a dump of only the user databases/schemas (skipping the mysql
system schema, remember stored procedures, views and functions)
3. Do a clean installation of new MySQL version (remember to migrate
important my.cnf options)
4. Load the dump file into the new MySQL server instance
5. Done!
37
http://dev.mysql.com/doc/refman/5.7/en/upgrading.html#upgrade-procedure-logical
http://mysqlserverteam.com/upgrading-directly-from-mysql-5-0-to-5-7-with-mysqldump/

MySQL Upgrade – Using Replication “simplified”
2. Restore backup from production (5.6) database to new slave server (5.7)
3. Start replication from master to slave
4. Once slave server has caught up with master
5. Change application to connect to slave (5.7)
6. Done!
38
MySQL 5.6 MySQL 5.7
Application
Replication
http://dev.mysql.com/doc/refman/5.7/en/replication.html

MySQL Upgrade – Regression Problems
• Explain is you friend!
– Optimizer_switch can help get back old query plans for slow queries
• Monitor resources/MySQL before/after to spot changes
– Without proper monitoring you have little chance to find root cause of problem
• Did you change more than just MySQL version?
– Normally we see people refresh HW/OS at the same time with creates many
possible regression scenarios
• Use PERFORMANCE_SCHEMA and SYS schema to spot problems.
39

Upgrading from MySQL 5.1 or 5.5 to MySQL 5.7?
News in MySQL 5.6
• Need version 5.1.20 of Connector/J (due to retire of “SET OPTION”)
• InnoDB “Files per table” now default
• The Query Cache is disabled by default
• 8 new reserved key words have been defined (“get” and “partition”)
• New data format of TIMESTAMP and DATETIME (microseconds)
• Password on command line causes warnings (problematic for scripts)
• New optimizer features may cause different query plans
• Old password are not allowed by default
40
https://dev.mysql.com/doc/relnotes/mysql/5.6/en/

New Servers Default’s in MySQL 5.7
• innodb_strict_mode is “ON”
• InnoDB file format is now Barracuda
• sql_mode option “NO_AUTO_CREATE_USER, STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, NO_ZERO_IN_DATE” is
now default
• More changes to optimizer_switch, may cause changed query plans!
– Use explain to identify problem and set optimizer_switch to solve problem
• MySQL accounts expire after 360 days. The default-password-lifetime is 360
• … and many more, see link below for more details
41
http://mysqlserverteam.com/improved-server-defaults-in-5-7/

Removed Features in MySQL 5.7
• Old password are not allowed in MySQL 5.7
• Remove IGNORE for ALTER TABLE in 5.7, WL#7395
• Remove of INSERT DELAYED, WL#6073
• Remove YEAR(2) support, WL#6263
• Deprecate Unique Option Prefixes, WL#6978
• Deprecate skip-innodb in 5.6 and remove it in 5.7, WL#7976
• For complete list: http://mysqlserverteam.com/removal-and-deprecation-
in-mysql-5-7/
42

MySQL 5.7 – Installation changes
• mysql_install_db is removed
– Replaced by mysqld --initialize
• MySQL is more Secure by default:
– TLS/SSL enabled by default and for some packages keys are created by default
– No creation of test database
– No creation of anonymous users
– Only one root user and password is set at installation time
– All user accounts will automatically have their passwords rotated after 360 days
– New mysql_ssl_rsa_setup tool
• Use mysqld --initialize-insecure to get old behavior
43

MySQL 5.7: Additional Info
• http://mysqlserverteam.com/whats-new-in-mysql-5-7-first-release-candidate/
• http://mysqlserverteam.com/category/performance/optimizer/
• http://mysqlserverteam.com/category/innodb/
• http://mysqlserverteam.com/category/mysql/performance-schema/
• http://dev.mysql.com/doc/refman/5.7/en/
• http://dev.mysql.com/doc/refman/5.7/en/upgrading.html
• https://www.mysql.com/support/supportedplatforms/database.html
•
44

MySQL 5.7 - What's new and How to upgrade

MySQL 5.7 - What's new and How to upgrade

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to MySQL 5.7 - What's new and How to upgrade

Similar to MySQL 5.7 - What's new and How to upgrade (20)

Recently uploaded

Recently uploaded (20)

MySQL 5.7 - What's new and How to upgrade

Editor's Notes