SQL Server 2016 Automated Installs and a SQL 2016 Webinar

I hate installing things through a GUI–I’ve talked about this before at PASS Summit, and when I worked at big cable, we build a private cloud and fully automated the SQL Server installation process. When SQL Server 2014 launched, my process had virtually no changes, however a number of things broke it in SQL Server 2016 (mostly for the better)

factory_automation_robotics_palettizing_bread

  • TempDB installs with the right number of files–they are still too small though
  • The behavior of trace flags 1117 and 1118 are built in to TempDB so you don’t need to turn them on
  • SSMS is a now a separate installation and download
  • I used batch and I wanted to rewrite into PowerShell

So since this is 2016, I decided to share it on Github. You can find it here: https://github.com/DC-AC/SQL2016_Scripted_Install. Feel free to fork and make edits in the code, or tell me how crappy my PowerShell logic is. Read the readme–there’s a few assumptions about your configuration. I’d like to parameterize those over time.

One other thing I wanted to mention is that Denny Cherry and I are doing a webinar this Friday July 29th at 11 PDT/2 EDT talking about managing SQL Server 2016 in production. We’ve have a customer live for a year now, and will share what we’ve learned and answer your questions. You can sign up here.

 

 

What Happens to Pages in the Buffer Pool when your Availablity Group Fails Over?

Recently at SQL Saturday Philadelphia, we started discussing failover  as it relates to mirroring and Always On Availability Groups. Specifically, we were wondering what would happen if you had a relatively busy readable secondary replica (which would have a lot of pages in the buffer pool on the secondary instance) and if those pages would be flushed from cache or anything like that. So I reached out to the product group and Kevin Farlee from Microsoft was extremely helpful:

Pages in the buffer pool are still valid, as they are updated by the redo thread.  By the time the secondary has transitioned to primary, all pending updates will have been applied, so pages in the bufferpool will all have correct contents. Note that the set of pages in bufferpool may be different between primary and secondary so you may have some cache warm up to do.”

So in a nutshell, if you have a busy readable secondary your cache won’t take much time to warmup, since the hot pages are already there.

PASS Summit 2016 I’m Speaking

pass_2016_websiteI have been honored to be selected to speak at this October’s PASS Global Summit in Seattle this October. I will be speaking on a topic near and dear to my heart, Security in Azure SQL Database. I worked last year to write a white paper with Stacia Varga and Microsoft discussing the best practices and security for Azure SQL Database.

In this session we’ll talk about all the encryption features, the security certifications that Azure has, and how audit is better in Azure SQL Database than it is in SQL Server. Additionally, you’ll learn about some of the other enhancements Microsoft has made to protect your data in Azure.

 

 

SQL Server 2016—The Licensing Info

SQL Server 2016 launched last week to great reviews and with a ton of great new features. I have been working with this version for well over a year now and extremely happy to see it hit RTM and be broadly adopted. So as DBAs it always sucks when you get excited about new features, only to find out the price changed, or vendor “O” made that feature a cost option. So what’s new with SQL Server 2016 licensing? (you won’t this as a session title at any upcoming SQL Server events).  Well first the good news—SQL Server 2016 is the same price and 2012 and 2014 (roughly $6800 core for Enterprise Edition). That’s definitely good news—Microsoft gave us a bunch of new functionality and didn’t raise the price. Additionally, if you see my below post on what is in Standard Edition, they added a lot of functionality there, too.

But we know finance and marketing employees have jobs to do as well, and there is no way they were letting a major version release happen without some changes. So let’s take a look at the one’s Denny Cherry (b|t) and I could glean out of the licensing guide. Please download and read for yourself.

Licensing Changes

Additional licenses are required when:
 A single hardware thread is supporting multiple virtual cores. (A core license is required for each v-core.)
 Multiple hardware threads are supporting a single virtual core simultaneously. (A core license allows a
single v-core to be supported by a single hardware thread.)

What does this mean? It means you can’t over provision CPUs on your VMs (which you shouldn’t be doing for DB servers anyway). There may be something that applies to hyperthreading here, but if you are licensing individual VMs, you probably shouldn’t be using hyperthreads.

“Beginning with SQL Server 2016, deploying and running SQL Server PDW is done through SQL Enterprise
Edition Per Core licensing with SA coverage. The number of SQL Server Enterprise Edition core licenses for an
APS appliance will depend on the total number of physical cores in the SQL Server PDW compute servers
configured within the appliance.”

What does this mean? It means you can actually know what an APS costs, at least from a licensing perspective. Hardware costs will need to be gathered from a reseller. This is a good change as it makes SQL Server’s pricing consistent across platforms (pro-tip: use SQLDW)

“For each server licensed with SQL Server 2016 and covered by active SA, customers can run up to the same
number of passive failover instances in a separate, OSE to support failover events. A passive SQL Server
instance is one that is not serving SQL Server data to clients or running active SQL Server workloads. The
passive failover instances can run on a separate server. These may only be used to synchronize with the primary
server and otherwise maintain the passive database instance in a warm standby state in order to minimize
downtime due to hardware or software failure.

 The secondary server used for failover support does not need to be separately licensed for SQL Server as
long as it is truly passive, and the primary SQL Server is covered with active SA. If it is serving data, such as
reports to clients running active SQL Server workloads, or performing any “work”, such as additional
backups being made from secondary servers, then it must be licensed for SQL Server.”

What does this mean? I had to consult Microsoft on this one, as it was a change in my understanding of the “free” secondary licensing benefit. Basically, if you are going to dedicated hardware (that you own or lease) your secondary license (if you have SA) is still included in your primary license. However, if you go to Azure from on-prem for your HA model, you will need to license the secondary. This does get murky because if both of those workloads are in Azure, you only license the primary.

“All SQL Server licenses with active SA can be reassigned to another server within the server farm as often as
needed; however, they can only be reassigned to another server in another server farm, or to a non-private
cloud, once every 90 days.
A server farm may consist of up to two data centers located in time zones that are within four hours of
one another and/or with the European Union (EU) and/or European Free Trade Association (EFTA).
A given data center may only be part of one server farm.”

What Does this Mean? Basically you can’t cross an ocean for HA or DR and not pay for it.

SQL Server Developer Edition
SQL Server 2016 Developer Edition is a fully featured version of SQL Server software—including all of the
features and capabilities of Enterprise Edition—licensed for development, test and demonstration purposes
only. SQL Server Developer Edition may not be used in a production environment or with product data. Any
test data that was used for design, development or test purposes must be removed prior to deploying the
software for production use.
Customers may install and run the SQL Server Developer Edition software on any number of devices. This is
significant, because it allows customers to run the software on multiple devices (for testing purposes, for
example) without having to license each non-production server system.”

What does this mean? It basically means only your production environments need to be licensed as long as you are following Microsoft’s rules for not using production data. This is a huge benefit, note that you can’t just restore prod to dev, you need to create some testing data, which is best practice anyway.

“Version Upgrade Rights are offered as a Software Assurance (SA) benefit for qualified licenses and allow
customers access to upgrade their deployments at no additional cost. Existing SQL Server 2012 software
licenses covered by SA are automatically upgraded to licenses for the corresponding SQL Server 2016
edition.”

What does this mean? You need to have 2012 licenses (at least) to upgrade to SQL 2016. Basically Microsoft wants to make sure you went through the core conversion in 2012.

Version Upgrade Rights are offered as a Software Assurance (SA) benefit for qualified licenses and allow
customers access to upgrade their deployments at no additional cost. Existing SQL Server 2012 software
licenses covered by SA are automatically upgraded to licenses for the corresponding SQL Server 2016
edition.”

What does this mean? If you had BI edition, your licenses will get converted to Enterprise Edition. If you read further into page 29, you will see that BI edition customers will be treated quite favorably. Any time there is a change like this, it’s usually a good time to negotiate with your Microsoft sales professional about getting a better deal.

Summary

There are no earth shattering changes in SQL Server 2016 licensing. The developer edition changes are probably the most significant, and can reduce your overall costs a great deal.

Did You Know?? R Services is in Standard Edition of SQL Server 2016

While my last post extolled the virtues of SQL Server Standard Edition, this week while doing some client testing  with Microsoft, I learned about another key standard edition feature. The new SQL Server R services is supported in standard edition of SQL Server 2016. While you won’t get access to some of the cool functions from Revolution Analytics and in-line parallelism (there is a @parallel=1 in sp_execute_external_script that automatically parallelizes your operations), you can still use all of the Open Source R functions, and build out your own parallelism (just multiple calls of your procedure). 

 

This is a huge benefit for those of you are who are doing statistical analysis of data and want to integrate with SQL Server. Stay tuned here for more detail for how DBAs can use R to analyze performance.

SQL Server 2016—Standard Edition Doesn’t Suck!

Yesterday Microsoft announced SQL Server 2016 was launching on June 1st. I can say this is the most production ready version of SQL Server I have ever worked on. I have had a customer on production since last August. We have been very happy and stable, and performance has been fantastic, we are using columnstore, availability groups, and lately R integration. All of these features are tested first in Azure, then deployed to the on-premises product which allows you to have a fully tested enterprise class RDBMS on day 1 of GA.

But What About Standard Edition

In my circles, there are number of people who are complaining about the lack of features in standard edition. While I do agree that Always Encrypted should be in every version, as lack of strong data encryption is a problem that continues to confound IT. Putting Always Encrypted in all editions would be a good start to having wide ISV adoption of the Always Encrypted feature.

However, even without Always Encrypted, Microsoft added a LOT of new features to Standard Edition. Let’s list them (no specific order here):

  • Temporal Tables
  • Query Store
  • Basic Availability Groups
  • Row Level Security
  • Dynamic Data Masking
  • Basic R Integration
  • Tabular Mode of Analysis Services
  • JSON Support
    I saw a complaint about the Tabular support only being 16 GB, which equates to (with typical compression) 100-150 GB which a very reasonable size model. I’ve also seen complaints about Standard Edition only addressing 128 GB of RAM. Microsoft is not a charity, their end goal is to make their shareholders money. There are a bunch of smart finance people, who make these calculations. If Microsoft increased the memory limit to standard edition to say 512 GB, this might mean 25% fewer customers (note—I made these numbers up) buy Enterprise Edition. Here is the Microsoft description of the aim of standard edition.

Standard
SQL Server Standard provides core data management and business intelligence capabilities for non-critical workloads with minimal IT resources.

If you are running mission critical workloads, and need high levels of uptime, and the tools that come with Enterprise  Edition, you need to pay the big bucks (and it’s still way the hell cheaper than Oracle, no matter how your calculate it) or there’s another option.

The Other Option—Azure SQL DB

You may have noticed Microsoft is making a push towards cloud computing (sarcasm). Cloud computing is going to be the defining trend of the next decade and beyond. Microsoft has been careful to avoid feature limitations in Azure SQL Database (there are a couple, columnstore and in-memory are only available in Premium, but that’s just because of limited hardware resources). Want features like partitioning, data compression, and online index rebuilds? Use SQL DB and all of those features are available to you. Always Encrypted is available at all service tiers, and has been since last summer. You can even have scale out readable secondaries now, even in Basic edition. Microsoft, through these actions, has incentivized moving your databases into Azure (and with elastic pools, it’s easier than ever).  While there while there will always be some on-premises systems,the cloud is changing the way products are developed and sold, and the way customers deploy. Don’t get left behind!

Letting Go—SQL Server 2005 Deprecation

SQL Server 2005 is deprecated this week, specifically tomorrow. That means if you have a problem, or a new bug is discovered, Microsoft support will require you to upgrade in order for support to help you with your problem. This is particularly a big deal when a new security bug is found, and it is not patched for the release of SQL Server that your organization is running

But I Work in Healthcare, Banking, Government

If you work in government, I don’t really have any advice for you—I”m not familiar with how to get through the government machinations to get off of really old software. However, if you work in health care, or banking you want to make sure your boss, and your quality assurance people know that you are now running an unsupported version of the software. When I worked in pharmaceuticals and device this worked pretty well—what really helped was getting our RBDMS classified as part of infrastructure so it didn’t have to be qualified like application software. If you work in banking or finance, the better option to follow up is the security angle—if there is a new zero day attack, your servers won’t be patched until your upgrade.

SQL Server 2005—A Pretty Awesome Release

As much as I’ve been working on SQL Server 2016 lately (and it really is awesome), but SQL Server 2005 was a groundbreaking release that really gave Microsoft market share in the big enterprise database space. A few of the features that were introduced in SQL 2005 include:

  • Database Mirroring
  • Dynamic Management Views (DMVs)
  • Service Broker
  • SSIS
  • Modern Consistency Checks
  • SQL Server Management Studio

SQL Server 2005 was an excellent release, however if you are using it, you are running effectively 11 year old software to host your applications. That’s a bad idea—remember what cell phones were like in 2005?

image

That’s what your database looks like now. Just remember—if you are doing an upgrade skip 2008, 2008 R2 and 2012, and just go straight to SQL Server 2014 (or 2016 if you are reading this after RTM), there’s no reason to upgrade to software that’s going to be desupported in two years. Also, install it on the latest version of Windows, so you don’t have to do this again when Windows 2008R2 is deprecated.

Challenges of Speaking—Being Ready for Anything..

I have the good fortune to have a job that allows me to travel around the world speaking about technology. I love this—it gives me a way to see new places, experience other cultures, and meet a lot of wonderful people how have similar interests to me. I always say about half of my job is working on computers and databases, and the other half is talking and writing about them. I’ve been speaking professionally for almost 10 years now, and have had a lot of unique situations.

One of the biggest challenges is that I speak a lot about cloud computing—generally speaking this involves connecting over a myriad of ports to internet connections that may or may not be blocked. Here are some tips that I’ve used to deal with getting to the cloud in various bad network configurations.

  • Use a VPN to remote desktop to another machine that can get out
  • When you realize the network is blocking RDP packets use your phone to tether
  • Set up a VM in Azure with RDP running on Port 443 (https port) and RDP to that
  • Finally, use screenshots, if all else fails
    All of these are complicated by foreign countries, different networks, different network policies, etc.
    The one that takes the cake for challenges though is losing the projector. I had this happen for the first time at a SQL Saturday in Houston—I was doing an session on SQL Server High Availablity and Disaster Recovery and projector locked up. It was a session I easily could have done as a white board. However, the staff was able to get me up and running again.
    Recently, I had my machine lock up, during a session at SQL Saturday Cleveland, so while I was trying to get it to work, I calmly pulled out my second machine, fired it up while I got my other machine working, and kept going. I had another failure, so I restarted on my DR machine. I’m not saying you need to have two laptops in your bag, but it is a nice to have.
    Finally, this week while I was teaching in India, I lost video again—this time there weren’t good white boards. My topic was networking in Azure—so I worked with another presnter to do a live session with each of use representing parts of the network. The session went over quite well.

So what are the tips you need to know:

  • Always have a DR plan (this goes for travel, presenting, IT, and life in general)
  • Stay calm, getting overly nervous does you or the audience no good
  • If things go really sideways, don’t be scared to co-present with someone. It’s a good rescue mechanism
  • Your audience wants you to succeed, they aren’t against you. So relax.
    When all else fails, scotch.

Getting Started with Polybase—Incorrect Syntax near ‘EXTERNAL’ Error

I was playing around with configuring Polybase this morning. Nothing to exotic, just installing the feature getting started. I got everything installed and was about ready to created my data source. The command to that is as follows:

CREATE EXTERNAL DATA SOURCE AzureStorage
WITH (           
TYPE = Hadoop,        
LOCATION = ‘wasb://polybase@mystorageaccountnamehere.blob.core.windows.net’,
CREDENTIAL = AzureStorageCredential);

Unfortunately, it returned the following error:

Msg 102, Level 15, State 1, Line 1

Incorrect syntax near ‘EXTERNAL’.

So what was the fix? A reboot after installing Polybase. The official documentation refers to restarting SQL Server, which I did. Nothing warned me to reboot, but rebooting fixed the error.

Using SQL Sentry Plan Explorer to Shed Light on Estimated Execution Plans

Note: While I do have business relationships with many vendors, I do not have one with SQL Sentry aside from attending their great annual party at the PASS Summit.

SQL Server allows us to have access to two types of execution plans, estimated and actual. Actual execution plans give us the real row counts and statistics the optimizer gets when executing the query, while the estimated plan displays the estimates the SQL Server optimizer used based on statistics when it generated the plan. The ideal scenario for performance tuning is to have the actual plan, but when you are dealing with a long running query that runs hours, and a busy procedure cache, that can be a real challenge to get.

So yesterday, I was helping a friend tune a couple of queries, one of which was running hours, and one of them was running in minutes. So I got my hands on the estimated execution plans, and I loaded them quickly in SQL Sentry Plan Explorer (which has a free version). I can’t recommend this tool enough—for simple query plans Management Studio is ok, but the second you have any level of complexity, Plan Explorer allows you to quickly break it down and identify where the problems are. As in this case:

fig1

Figure 1 The Good Query

fig2

Figure 2 The Bad Query

I quickly honed into the fact that the bad query was doing a lazy spool with 6 BILLION rows versus the 229 million in the good query. However, my friend who was looking at Management Studio was asking where I was getting those numbers. This is what the XML from the original plan showed:

<RelOp AvgRowSize=”51″ EstimateCPU=”0.00889634″ EstimateIO=”0.01″ EstimateRebinds=”0″ EstimateRewinds=”139581″ EstimatedExecutionMode=”Row” EstimateRows=”48868″
LogicalOp=”Lazy Spool” NodeId=”55″ Parallel=”true” PhysicalOp=”Table Spool” EstimatedTotalSubtreeCost=”1242.86″>

I made the assumption that Plan Explorer was doing some math on the execution plan, so I contacted my good friend Aaron Betrand (b|t) who works at SQL Sentry and asked him about it. His response was that “Plan Explorer will sometimes adjust the displayed row counts for certain operators where we know execution count is greater than one”. This is a really great use case for getting better data out of estimated execution plans when that is all you have to work with.

Follow

Get every new post delivered to your Inbox.

Join 2,806 other followers

%d bloggers like this: