Crossing the Threshold…

So it’s the usual story. You need to upgrade a machine, the IT group says, “no problem, we can virtualize it, it’ll be better! Don’t worry, yes, it’ll be fewer CPUs, but they’ll be much faster!”

So, you move forward with the upgrade. Twenty-three meetings later, 3 late nights, one OS upgrade, and two new machines forming one new cluster, you’re good. Things go live.  And then Monday happens. Monday of course is the first full day of business and just so happens to be the busiest day of the week.

Users are complaining. You look at the CPU and it’s hitting 100% routinely. Things are NOT distinctly better.

You look at the CPUs and you notices something striking:

cpu not being used

CPU 8 is showing a problem

4 of the CPUs (several are missing on this graphic) are showing virtually no utilization while the other  8 are going like gang-busters.  Then it hits you, the way the IT group setup the virtual CPUs was not what you needed.  They setup 6 sockets with 2 cores each for a total of 12 cores. This shouldn’t be a problem except that SQL Server Standard Edition uses the lower of either 4 sockets or 24 cores. Because your VM has 6 sockets, SQL Server refuses to use two of them.

You confirm the problem by running the following query:

SELECT scheduler_id, cpu_id, status, is_online FROM sys.dm_os_schedulers

This shows only 8 of your 12 CPUs are marked visible_online.

This is fortunately an easy fix.  A quick outage and your VM is reconfigured to 2 sockets with 6 cores a piece. Your CPU graphs now look like:

better CPU

A better CPU distribution

This is closer to what you want to see, but of course since you’re doing your work at night, you’re not seeing a full load. But you’re happier.

Then Monday happens again.  Things are better, but you’re still not happy. The CPUs are running on average at about 80% utilization. This is definitely better than 100%. But your client’s product manager knows they’ll need more processing power in coming months and running at 80% doesn’t give you much growth potential. The product manager would rather not have to buy more licenses.

So, you go to work. And since I’m tired of writing in the 2nd person, I’ll start writing in 1st person moving forward.

There’s a lot of ways to approach a problem like this, but often when I see heavy CPU usage, I want to see what sort of wait stats I’m dealing with. It may not always give me the best answer, but I find them useful.

Here’s the results of one quick query.

Fortunately, this being a new box, it was running SQL Server 2016 with the latest version service pack and CU.  This mean that I had some more useful data.

CXPackets

CXPackets and CXConsumer telling the tale

Note one of the suggestions: Changing the default Cost Threshold for Parallelism based on observed query cost for your entire workload.

Given the load I had observed, I guessed the Cost Threshold was way too low. It was in fact set to 10.  With that during testing I saw a CPU graph that looked like this:

43 percent CPU

43.5% at Cost Threshold of 10

I decided to change the Cost Threshold to 100 and the graph quickly became:

25 percent CPU

25% at Cost Threshold of 100

Dropping from 43.5% to 25.6%. That’s a savings you can take to the bank!

Of course that could have been a fluke, so I ran several 5 minute snapshots where I would set the threshold to 10, collect some data and then to 100 for 5 minutes and collect data.

CXPacket_10      CXPacket_10_Waittime_MS
635533 5611743
684578 4093190
674500 4428671
CXConsumer_10              CXConsumer_10_Waittime_MS
563830 3551016
595943 2661527
588635 2853673
CXPacket_100   CXPacket_100_Waittime_MS
0 0
41 22
1159 8156
CXConsumer_100            CXConsumer_100_Waittime_MS
0 0
13 29443
847 4328

You can see that over 3 runs the difference between having a threshold of 10 versus 100 made a dramatic difference in the total time spent waiting in the 5 minute window.

The other setting that can play a role in how parallelization can impact performance is MAXDOP. In this case testing didn’t show any real performance differences with changing that value.

At the end of the day though, I call this a good day. A few hours of my consulting time saved the client $1,000s of going down the wrong and expensive road of adding more CPUs and SQL licenses. There’s still room for improvement, but going from a box where only 8 of the 12 CPUs were being used and were running at 100% to a box where the average CPU usage is close to 25% is a good start.

What’s your tuning success story?

52

52 is an interesting number.  It’s the number of weeks in the year. It’s the number of cards in a deck. It’s a number of Earths in the DC Multiverse. It’s an untouchable number, something I just learned. It’s the atomic number of tellurium. In fact it has a number of interesting trivia associated with it according to Wikipedia: 52.

It also just happens to be the number of times I’ve been around the Sun, though strictly speaking that depends if you’re counting sidereal or the tropical year and the fact that I was born at night. But I think we’re close enough.

And it just so happens my birthday falls on the day I usually blog. So rather than something technical (though if I can get permission from a client, I may have something fun and technical soon) I thought I’d post some reflections and thoughts.

For me birthdays are both interesting and boring. I’m glad I’ve reached another milestone. But honestly, after age 25 when my car insurance rates went down, I haven’t given individual birthdays much thought. That’s not strictly true, I sometimes think about the fact that I’ve passed the point where statistically I’m looking at fewer days ahead of me than behind me.

I grew up in a small town, Falls Village CT, and parts of me never have left it. When I stop to daydream, my thoughts take me to the town green where we often played, or the woods behind my dad’s house, or the sand quarry behind the depot I grew up in. It was a safe and quiet life. I watched the world move from bell-bottoms to Reagan power ties.

While just a teenager, I and friends ran not one, but two Monopoly marathons to raise money for the Muscular Dystrophy Association. The first year we played for 100 hours (in teams) and the second 150 hours. I was and am still quite proud of the organization that took and the money we raised.

Since then I’ve done a lot.  I got thinking about that last night at the Capital Area SQL Server User Group meeting. I’m proud to lead this group.  I really enjoy, as I’ve noted before, giving back to the community that has helped me so much.

I’m proud to be a Regional Coordinator for the National Cave Rescue Commission. I can literally say the work the NCRC does saves lives. It’s an honor and humbling when folks come up to me and tell me how their training has made a difference in the lives.

I’m proud of much of what I’ve done in as my avocations and vocations, even if at times I’m often a victim of imposter syndrome. There’s still many times when someone will ask me a question and my first thought is, “why are they asking me, I’m just a kid and… oh wait… no I am the expert here and I’m far from being a kid.”  This is especially true when people I look up turn around and ask me for advice.  This happens a lot in the SQL world.

I’m very proud of my family, especially my son and daughter who are growing up to be wonderful adults, capable of critical and deep thinking. They will make an impact on the world and I don’t think as a parent I could want for anything more.

And of course proud of my wife, but I can’t take credit there, she’s a wonderful person in her own right. I just married well.

One common thread I’ve realized in my life that I enjoy is teaching and sharing my knowledge. I also, as anyone knows me, love a good debate.  Bring your facts to the table. Teach me something, change my mind, or be willing to have yours changed. I still recall a debate I had with someone once about a detail of the Constitution. She made one claim, I made another. We finally settled it by finding a copy of the text and realizing who was right. Afterwards there was no rancor or hurt. We both had appreciated the intellectual exercise and the correction of fact.  But even opinions can at times be changed. At Summit I had two pieces of white chocolate from New Zealand. They changed my mind about white chocolate! I enjoyed them.

People often ask me what I want for my birthday and the truth is, I rarely want material things. Honestly, unless it’s a new Tesla (and NOT the truck) I can and will probably buy it for myself.

But here goes:

  • Another 52 years – hey, why not? We’re making medical breakthroughs, and it’s possible I’m wrong and I’ve got more days ahead of me than behind me. Right now, I’d love that.
  • Learn something – challenge yourself in the next year to learn a new skill or a new topic. Don’t get stuck doing the same things all the time.  Earlier this year I finally took the time to start learning about Extended Events. Who knows what I’ll learn in this coming year.
  • Teach someone something – everyone one of you has a skill someone else doesn’t. Share.
  • Related to that: if you’re a caver, or have a friend or family member who is a caver, get them to take the 2020 National Cave Rescue Seminar.
  • Have a friendly debate with someone. Realize it’s not about winning or losing, but an exchange of ideas. Bring your facts to the table and recognize your opinions. Be open-minded.  Be prepared to say, “You know what, you’re right, I was wrong.” This is not losing a debate.  And be prepared to acknowledge someone saying the above to you. Accept the words graciously, don’t lord it over them. This is not about winning a debate.
  • Be kind.  If nothing else in the coming year, be kind.

And that’s it for turning 52.

P.S. Unrelated, but check out my latest article at Redgate’s Simple-talk: Building a Countdown Timer with PowerShell

Kids, get off my lawn!

Change can be hard. But sometimes it’s necessary. And a lot has happened this week.  First, I want to congratulate my fellow #SQLFamily member Cathrine Wihlemsen on one more orbit of the Sun. Apparently, in her honor Microsoft decided to release SQL Server 2019 on her birthday! I’ve been using SQL Server since the 4.21a days. Every version has had new features and required learning something new. As I said recently, it’s easy to fall into the trap of being an old dog and not learning new tricks. This is something we have to avoid. Being trapped in the past can be limiting.

Besides SQL Server 2019 dropping this week, I recently upgraded my phone. I had been using a Windows Phone for about 5 years now. I loved it. Especially when it first came out, it was top of the line and had a bright future. I eagerly downloaded apps and it became part of my life. But alas, we know how well Microsoft did in the Windows Phone market. But I doggedly held on, even as features were deprecated. I couldn’t use the Weather App. The Amtrak App went away. Eventually several features of Cortana stopped work as Microsoft stopped supporting them. Slowly my phone was becoming a brick. I kept debating do I upgrade to one more Windows Phone knowing it’s the end of the line, or what? I kept putting off the decision. After the mapping function failed me on my recent trip to the Hampton Roads User Group Meeting I decided it was time to finally time to replace it with an Android phone. Choosing from the plethora out there was not fun. It was very tempting to go with one of the top of the line models, but spending $1000 or so wasn’t really a fun idea.  I eventually ended up choosing a Samsung A50.

I’m mostly happy with it. Right now I’m struggling with what parts of it are “get off my lawn” because I don’t like change, and what parts are “what the hell is the UI doing now?”  Fortunately, my son has mentioned some of his dislike of certain UI functionalities, so I think not all of it is me simply being an old curmudgeon (are there young ones?) I will say what I’m most happy with is that Microsoft has a number of tools including the Windows Launcher and the Phone Companion, as well as the obvious apps like Outlook and other parts office.

A word about the Phone Companion. This alone has made the upgrade a win. One of the features is that when I’m working at home (I have not yet enabled it on my Surface Pro) is that things like text messages pop-up on my desktop screen. This actually makes life a LOT easier, since I can simply type a reply from a full-size keyboard or copy the numerous soft-tokens I get to log into various client sites without having to pick up my phone. It’s a small detail, but a wonderful one!

The Launcher helps me retain some of the features that I liked about my Windows Phone. Overall, it’s a win.

But the changes in my life aren’t complete. As I mentioned last week I’m at PASS Summit again this week in Seattle. But alas, this is the last year that PASS Summit will be in Seattle. Next year it will be held in Houston. Just as I’ve figured out where the cheapest and most convenient parking for me is, where some decent food places are, and I’m feeling, if not at home in Seattle, at least comfortable, next year is a big change. I won’t be able to stay with my college friends or do our annual Thai pot luck with a bunch of ROC Alumns.

But, I’ll get to explore another city. I’ve been to Houston only once, literally decades ago, to do SQL Server install at Exxon. The server was literally the only Intel computer in a room full of mainframe equipment. I suspect that has changed since then.  That was one of my early experiences installing SQL Server (4.21a for the record).

So, this old dog is still learning and looking forward to new experiences: plus ça change, plus c’est la même chose.

 

Small Victories

Ask most DBAs and they’ll probably tell you they’re not a huge fan of triggers.  They can be useful, but hard to debug.  Events last week reminded me of that. Fortunately a little debugging made a huge difference.

Let me set the scene, but unfortunately since this was work for a client, I can’t really use many screenshots. (or rather to do so would take far too long to sanitize them in the time I allocate to write my weekly blog posts.)

The gist is, my client is working on a process to take data from one system and insert it into their Salesforce system.  To do so, we’re using a 3rd party tool called Pentaho. It’s similar to SSIS in some ways, but based on Java.

Anyway, the process I was debugging was fairly simple. Take account information from the source and upsert it into Salesforce. If the account already existed in Salesforce, great, simply perform an update. If it’s new data, perform an insert.  At the end of the process Pentaho returns a record that contains the original account information and the Salesforce ID.

So far so good. Now, the original author of the system had setup a trigger so when these records are returned it can update the original source account record with the Salesforce ID if it didn’t exist previously. I should note that updating the accounts is just one of many possible transformations the entire process runs.

After working on the Pentaho ETL (extract, transform, load) for a bit and getting it stable, I decided to focus on performance. There appeared to be two main areas of slowness, the upsert to Salesforce and the handling of the returned records. Now, I had no insight into the Salesforce side of things, so I decided to focus on handling the returned records.

The problem of course was that Pentaho was sort of hiding what it was doing. I had to get some insight there. I knew it was doing an Insert into a master table of successful records and then a trigger to update the original account.

Now,  being a 21st Century DBA and taking into account Grant Fritchey’s blog post on Extended Events I had previously setup a Extended Events Session on this database. I had to tweak it a bit, but I got what I wanted in short order.

CREATE EVENT SESSION [Pentaho Trace SalesForceData] ON SERVER
ADD EVENT sqlserver.existing_connection(
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.login(SET collect_options_text=(1)
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.logout(
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.rpc_starting(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.sql_batch_completed(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.sql_batch_starting(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho'))
ADD TARGET package0.ring_buffer(SET max_memory=(1024000))
WITH (MAX_MEMORY=4096 KB,EVENT_RETENTION_MODE=ALLOW_SINGLE_EVENT_LOSS,MAX_DISPATCH_LATENCY=30 SECONDS,MAX_EVENT_SIZE=0 KB,MEMORY_PARTITION_MODE=NONE,TRACK_CAUSALITY=ON,STARTUP_STATE=OFF)
GO

It’s not much, but it lets me watch incoming transactions.

I could then fire off the ETL in question and capture some live data. A typical returned result looked like

exec sp_execute 1,N'SourceData',N'GQF',N'Account',N'1962062',N'a6W4O00000064zbUAA','2019-10-11 13:07:22.8270000',N'neALaRggAlD/Y/T4ign0vOA==L',N'Upsert Success'

Now that’s not much, but I knew what the Insert statement looked like so I could build an insert statement wrapped with a begin tran/rollback around it so I could test the insert without actually changing my data.  I then tossed in some set statistics IO ON and enabled Include Actual Execution Plan so I could see what was happening.

“Wait, what’s this? What’s this 300K rows read? And why is it doing a clustered index scan on this table?”  This was a disconcerting. The field I was comparing was the clustered index, it should be a seek!

So I looked more closely at the trigger. There were two changes I ended up making.

       -- Link Accounts
       --MERGE INTO GQF_AccountMaster T
       --USING Inserted S
       --ON (CAST(T.ClientId AS VARCHAR(255)) = S.External_Id__c
       --AND S.Transformation in ('Account'))
       --WHEN MATCHED THEN UPDATE
       --SET T.SFID = S.Id
       --;
       
       if (select transformation from Inserted) ='Account'
       begin
              MERGE INTO GQF_AccountMaster T
              USING Inserted S
              ON T.ClientId  = S.External_Id__c
              WHEN MATCHED THEN UPDATE
              SET T.SFID = S.Id
       end

An astute DBA will notice that CAST in there.  Given the design, the Inserted table field External_Id__C is sort of a catch all for all sorts of various types of IDs and some in fact could be up to 255 characters. However, in the case of an Account it’s a varchar(10).

The original developer probably put the CAST in there since they didn’t want to blow up the Merge statement if it compared a transformation other than an Account. (From what I can tell, T-SQL does not guarantee short-circuit evaluation, if I’m wrong, please let me know and point me to definitive documentation.) However, the minute you cast that, you lose the ability to seek using the index, you have to use a scan.

So I rewrote the commented section into an IF to guarantee we were only dealing with Account transformations and then I stripped out the cast.

Then I reran and watched. My index scan of 300K rows was down to a seek of 3 rows. The trigger now performed in subsecond time. Not bad for an hour or so of work. That and some other improvements meant that now we could handle a few 1000 inserts and updates in the time it was previously taking to do 10 or so.  It’s one of those days where I like to think my client got their money’s worth out of me.

Slight note: Next week I will be at PASS Summit so not sure if/when I’ll be blogging. But follow me on Twitter @stridergdm.

NY ComicCon

Last week I talked about Kids These Days. This past weekend I went with my daughter to NY ComicCon. It was a late 8th grade graduation present she had requested. Due to me messing things up last year, we missed our chance to go, so I made up for it this year. And it was well worth it, for a couple of reasons. I want to focus on two, one topical and one personal. The topical first.

The topic is in the above photograph.  I apologize for it being blurry, “I’m a DBA Jim, not a photographer.”  But I took it for a reason. This was the panel for a talk titled: Join the Resistance! (Journey to Star Wars: The Rise of Skywalker). It was an interesting panel that talked about the books they wrote that cover the time between The Last Jedi to this December’s The Rise of Skywalker. But partway through listening, something dawned on me about the panel.  Can you figure out what I realized?

It’s there in the picture, but if not, let me list the panelists: authors Rebecca Roanhorse, Justina Ireland, Kevin Shinick, Ethan Sacks, Delilah S. Dawson, audiobook narrator Marc Thompson and moderator Ashley Eckstein.

What strikes you about that list of names? Now compare that to the panels you see at a number of tech events such as various SQL events. Note what it’s not. This is NOT a MANEL!

Science Fiction has for far too long been treated as the domain of boys and then later men. Marketing for decades often focused on boys. It was assumed that every boy wanted to be Han Solo or Luke Skywalker or Captain Kirk.  Women in shows and books were often only there as props for the male characters to react to. Granted, this statement isn’t 100% true, even Princess Leia had some meat to her character in the original Star Wars (back before it was episode IV or A New Hope.) Even then though, she served the role set out in much of mythology as the princess in distress to be rescued. Fortunately her role and the role of women in Star Wars was greatly expanded over the series, to the point now where Rey is our hero.

Ahsoka Tano in triplicate!

Ahsoka Tano in triplicate!

And this panel shows exactly how equitable the Star Wars universe has become. The moderator was Ahsley Eckstein, who voices the character Ahsoka Tano in various animated Star Wars series. Three of the authors on the panel were women. In other words, women were well represented.

Think about this when planning your tech event such as SQL Saturday. Do you have equal representation? “But wait Greg, there’s just not that women doing SQL! I only had 3 women apply to talk and 30 men!” I’m going to give you some advice. Ask for more women. Talk to those three, see if they know anyone who might want to speak, but was too nervous to put in a submission. Talk to Kathi Kellenberger and Rie Irish of the PASS Virtual Group Women in Technology.  Yes, there may not be as many women in tech as men, but I can guarantee that there’s more than you think and that it won’t change without encouragement and representation. If you as a guy get invited to speak on a panel, make sure there’s diversity. Turn down opportunities if it looks like it’s going to be a manel. Call out your fellow community members if they’re engaging in sexist behavior. It’s not always comfortable,especially if it’s a friend or a co-worker, but it needs to be done. Do your part.

If ComicCon can have an equitable panel in regards to Star Wars, you can do the same in regards to SQL or other tech panels.

Now for the personal:

Live Long and Prospoer

Autograph and picture with two amazing women, Nichelle Nichols and my daughter Rebecca.

Two amazing women: Nichelle Nichols is an amazing woman and helped represent African Americans on television in the 1960s and helped inspire people like Whoopi Goldberg and Mae Jemison. And as for my daughter, her future and journey is in front of her.  I will admit to basically being speechless in front of such an icon and here I am, still three days later grinning ear to ear thinking “I was in the presence of Uhura!”

(BTW, for those who recognize it, that’s a 1st edition Star Fleet Technical Manual with her signature. It also contains the signature of George Takei and James Doohan.)

 

Old Dogs and New Tricks

One problem with writing a weekly blog is sometimes you get the same idea in your head and realize you’re about to repeat yourself. I want to write about learning new stuff and started to repeat stuff I said here. I suppose I could just put a pointer to that and be done but… nah.

Several ideas prompted this week’s theme for this blog, the most recent being Grant Fritchey’s blog on using Extended Events. He makes some good points and I want to add my own thoughts.

I’ll start by admitting I still often use Profiler. I know it. It’s “easy”.  Well, no, not really. A better description would be “It’s familiar”. Truth is, I’ve often found the interface a bit clunky and I have to do more steps than I want to narrow down to trace exactly what I want.

But, about two weeks ago, one of my clients had an issue with a run-away query that expanded their tempdb to about 400GB before it rolled back. We wrote it off as a fluke; until it happened again the next day.  Now we had a pattern. I decided that we needed to monitor the server about the same time the next day to see if it would occur again. Now, of course one can sit there and run queries like sp_whoisactive, but I wanted more.  Profiler might work, but I figured it was time for this old dog to learn, or at least practice new tricks. I’ll admit, I basically googled for what I wanted, but I quickly found a trace I could modify and run for my needs.  30 minutes later and I had a nice extended event setup and monitoring the tempdb.

Now, as fate would have it, that run-away query hasn’t shown up since then, so in a sense the trace hasn’t fulfilled it’s goal. But, it’s lightweight enough that I’ve decided to leave it. And, the results are easily accessible to others if I’m out of town.

In the past year or so, it’s really hit me how much my role as a DBA or IT professional has really changed from two or more decades ago. There’s stuff that I do now that wasn’t possible then and there’s stuff then that I would do that I don’t worry about now or can’t even do (who out there recalls the ability to setup SQL Server database “files” on raw partitions for the speed boost?)

Over two decades ago I wrote a fairly impressive batch file that could install a client’s application on the proper drive. It used a lot of commands most people weren’t even aware of in the batch language (this was DOS, so not even as good as what CMD in Windows NT and above gives us).  Nowadays, I’d use PowerShell.

My original programming was in Fortran. Nowadays, I tend to sling code in VB.Net or C#.

It’s ok at times to fall back on what’s familiar. Often it can be faster and easier for a single project. But in the long-term, one really needs to learn what’s new and apply what one can.

My advice, pick a new technology or skill, and use it, even if for one project.  I’m not going to be an expert in Extended Events anytime soon, but now at least I can say, I’ve used it.  I’m far from an expert in PowerShell, but I’ve now been paid for 3 (and soon 4) articles on using it.

If this old dog can learn a new trick so can you.

And, listen to music you didn’t grow up on. As much as I love to listen to the music I’m familiar with while working, I often branch out. Yes, I’ve been known to listen to some Taylor Swift and Charlie XCX. And yesterday was a bit of Imagine Dragons and Meute. Two very different styles of music, but still a good listen.

What new trick will you learn this month?

Challenge Accepted!

Monica Rathbun in a recent blog post commented on how hard it is to write a blog post in under 5 minutes and challenged her readers to try to do it.

The only thing I can say is… challenge accepted.

But what to write about?

How about how I write, or rather how an idea gets into a blog post.

I have to admit, some Tuesdays my mind is blank. I sit at the screen, sometimes for 5 minutes or longer and my mind draws a blank. That’s rare. Fortunately, I often, sometime in the previous 6 days or so get an idea in my head and start to think about what I should write on it. It might have been a particular issue at work I had to solve, so I might be focusing on a more technical SQL or PowerShell focused blog.  Or it might be something I’ve seen that amused me.  This means I mull the thoughts over in my head and often have a basic outline before I put fingers to keyboard. The can help me cut down on the time I spent blogging.

I’ve also got about a dozen drafts saved in WordPress where I simply write a few lines of an idea for future posts. These are my saving graces. When I really can’t think of an idea I’ll go back and pull one of those up and finish them, such as this one which lay in draft status for months.

So, looking I think I failed. I think this one took just over 5 minutes. And to save time, I’m ignoring adding a picture, so you get the default. For now.

 

Busy Weekend Volunteering

As I mentioned previously, I was on vacation for about 10 days and got back to Albany very early Wednesday morning (or late Tuesday night depending on how one looks at it.) And once back from vacation I had to jump right back into two other events I had previously put on my schedule. This meant I didn’t have much time to catch up on work or sleep. But it was worth it.

A confluence of events meant that I ended up being double booked this past weekend. The first event was some special cave rescue training called a Small Party Assisted Rescue (SPAR) class. This was a 3 day class, Friday through Sunday. However, in addition, students had the chance to show up Thursday night in order to test on their skills before participating.  I was both an instructor for this class as well as the site and course organizer. My second event was SQL Saturday Albany, which I had been selected to speak at. I’m also the User Group coordinator that sponsors this event. This double booking meant that I couldn’t instruct at the SPAR on Saturday. I do want to note that at both events there were a number of other volunteers, and some were doing even more work than I was.

Between these two events, it meant I was getting about 6 hours of sleep a night plus putting in a lot of driving. It was a long, tiring, essentially 3.5 day weekend starting on Thursday. Additionally the jet-lag made it seem even longer.

Why do I mention all this? Because, both events are very important to me and cover two large areas of my life. I’ve previously written about some of my SQL Saturday experiences and SQL Pass experiences.  This is part of my professional life. I feel very strongly about volunteering and speaking at these SQL events. I enjoy running our local Capital Area SQL Server User Group (CASSUG) for the same reason. I’m a better DBA because of the shared experiences of my fellow speakers. I’ve written about this previously here and elsewhere in this blog. I hope I’ve helped others.

WP_20190720_002

Deborah Melkin discusses normalized vs. star schemas.

On the other side, as my slide deck often points out, I’m a caver. More critically I’m the Northeastern Regional Coordinator for the National Cave Rescue Commission. I’ve had the privilege of teaching 100s of people how to perform cave rescue, been a media resource during the 2018 Thai rescue, and have spoken and written on the subject. I am by no means an expert, I’m always learning, as are all my fellow instructors. But, we all are not only willing, but want to spend the time and money and effort to teach others. We are passionate about it.  I don’t mean this lightly. For this particular SPAR, while about 1/2 the instructors lived within 2 hours of the event and it was an easy drive, the rest either drove 5-8 hours, or spent all day flying on standby to get here or to get home. None were reimbursed for any of their expenses and in fact had to pay for linens if they wanted them.  They also had to take 2 days or more days off of work to come to New York.

Next summer, I will be the course coordinator for our 2020 National Weeklong here in New York State. This will bring close to 100 people to New York for a week of 14 hour days of teaching and learning cave rescue techniques. Fortunately, I will have a LOT of help organizing this event. But again, all the instructors and staff are volunteers who will travel at their own expense to be here and help teach.

So I spent my weekend volunteering, because I’m passionate about it. How was your weekend?

 

Marshmallows

Though I attended RPI, which is generally considered an engineering school, my degree is a BS in Computer Science. I say that because I consider myself more of a scientist than an engineer at times. And honestly, we all start out as scientists, but many of us lose that along the way.

Anyone who has had a small child has observed a scientist in action. No, they’re not in a lab full of test tubes and beakers and flasks giving off noxious smells. But they are in the biggest lab there is, the world. They also don’t necessarily realize it. Nor do parents. But every time they drop a Cheerio, they’re testing gravity.  Fortunately (or unfortunately depending on your point of view) so far every time they’ve managed to prove that gravity works. This is the most obvious example, but when you stop to think about it, much of the first few years of life is all about experimenting. Most of the time it goes well, but sometimes, as a burnt hand will attest, the experiment has a less than ideal outcome.

And it’s the fear of burned hands that leads to parents to utter that common  refrain, “Don’t touch that!” or the variation “Don’t do that!”.  Soon, over time, our experimentation starts to get reined in until we do very little of it. This can be inhibiting.

Years ago I used to teach an “Introduction to Windows” adult education class. It was I believe a 6 week class and I taught several over the course of a couple of years. It didn’t take me long to realize the biggest constraint on the students ability to succeed in the class was that they had internalized “Don’t do that, you might break something.” Once I realized that, half my teaching pedagogy simply became, “Touch that, you won’t break it, and if you do, it’s not a big deal, and if it is, we’ll fix it anyway.” Seriously, more than anything else, I had to encourage most of my students to experiment with the computer.

More recently I realized I had stopped doing as many experiments in my life as I should be doing. About 1.5 weeks ago I attended a Wilderness Medicine Conference a friend of mine had told me about. At the end of the very wet, cold, rainy day, a bunch of us went outside and tried to start a fire. Starting a fire, let alone in such conditions was something most of the students had never done. I had, but not in years. With some effort, and experimentation, including using the outside box of a single serving size package of Fruit Loops, we finally managed to get the fire going.

But this got me thinking. When I go hiking, I carry a tiny ziplock back in my jacket with some firestarting materials. They’re there in case of an emergency. But, the thing is, I had never actually tried them and realized if I didn’t know how well they worked in practice, I couldn’t rely on them in emergency. So, I went outside, and started a fire. And I learned that yes, my materials ARE adequate, but the dryer lint needed to be pulled apart more than I realized. I tried again later in the week, and added the use of a toilet paper roll to form sort of a chimney so the starting fire would draft better. This, and the better pulling of the lint worked even better and a single match was sufficient this time.  This gave me more confidence that in an emergency, in less than ideal conditions I could get an actual fire going.

But, I wasn’t done! Our microwave broke this weekend. But, before I wrote it off, I wanted to make sure it wasn’t a fluke or something else. So, in this case I decided to get a bag of marshmallows and lay them out inside the microwave to see if I was getting ANY energy out of the magnatron. Turns out, nope, nada, nothing. So, today or tomorrow I will be buying a new microwave. But, it was a fun, and later tasty experiment.

Without delving deep into the scientific method here, I’ll say at a simple level, science is about having a hypothesis and testing it. The testing it is important.

To bring this back to SQL. First, you have a hypothesis that your backups will work. Have you tested that hypothesis? If not, do so immediately. Even if they do, you might learn something now that will be important when you have to do it for real. Perhaps you learn the volume your backups are on only has write access. Or perhaps you learn you need to retrieve your encryption keys and the person who controls access to them is on vacation. Or perhaps your RPO is 4 hours and the restore takes 6 hours.  So, experiment.

query plan

Capture of a random query plan

Recently for one client I’ve spent some time experimenting with various changes to help improve the performance of some queries. Not everything I tried worked, but some things did. So, again experiment.

I’m curious what recent experiments you may have done, SQL or otherwise. What were their outcomes?

 

Redemption

About a year ago I wrote this post: And so it Happened… about my first (and so far only) time I ended up with an empty room at a SQL Saturday. I’ve run into a few other speakers who have had the same experience, so that soothed the bruised ego a bit, but it still left a bit of a mark.

As a result, I set a goal of redeeming myself this year again at the Colorado Springs SQL Saturday. I figured it wouldn’t be that hard to exceed my turnout from last year.  So, I submitted several topics for them to select from and waited. Finally the day came, and I found that I had been selected to speak. There was only one problem. The topic in question was one that while I had submitted, and had a good outline for, I had not actually fully developed into a presentation and was a bit nervous about:
The very Model of a Modern Day Database. I thought it would be a good topic, I just had to develop it.  And of course like any good procrastinator I kept putting off the work. I mean I was making progress, but, well it was slow.

Fortunately, by Friday the 5th, I had run through a complete form of it and had worked out pretty much all the tweaks I wanted and had practiced it a few times to an empty room, you know, just in case of a repeat of last year. Seriously though, I do several run-throughs to make sure I get the timing right and I pretty much know what I was going to say. I’ll let readers in on a little secret, some of the parts of my presentations that look like they’re improvised or impromptu comments or replies, are often rehearsed.

So I felt pretty good going into Saturday.  Then, looking at the schedule, it struck me that my topic was on the System Databases, one of which is known as the TempDB (to my non-SQL readers, that’s a fairly critical database SQL Server uses as sort of a scratch pad) and that a session before lunch (mine was scheduled after lunch) was by Kalen Delany and was an entire hour on just the TempDB. I first heard Kalen speak at SQL Connections conference back in 2005 or so in Orlando and had read a few of her books. To say that she’s well known in the SQL Community and highly respected might be an understatement. Now the impostor syndrome was really starting to kick in! What could little ol’ me say about the TempDB in 15 minutes that would interest people after listening to her?

But then I realized, our topics had a slightly different focus, and while some of our advice was similar (put your TempDB on FAST drives), I covered things in a different way and there would still be something of interest to my attendees. And, it is not a competition after all. Honestly, my goal whenever I teach any topic is to reach at least one student or attendee. If I can get one person to walk away and say, “I learned something” or “That was worth it” I feel like I’ve won. This happened during a week-long cave rescue training course once. On the first day in the field I showed a student a fairly simple but not entirely obvious way to rig a rope. After explaining it to her she looked at me and said, “that’s worth the price of the course right there!”.  I glowed and joked I could now take the rest of the week off; I had achieved my goal.

Anyway, after lunch I was prepared. Lunch was scheduled for 12:30-1:45 and I was in the classroom by 1:40, all setup waiting for folks to show up. And sure enough two people showed up. I was happy. Perhaps not ecstatic, but at least happy I had an audience.  And then two more people showed up, put down their stuff and asked, “mind if we leave this here, we’ll be back.”  I said it was fine, but was a bit confused since the clock was saying 1:44 and I was wondering where they’d be going just before my session started.

But hey, four people, that was four more than last year, even if two weren’t in the room and one of the others admitted they weren’t really a DBA and wasn’t sure if the class was applicable to what they wanted to learn.

At that point, one of the original pair started to shuffle her papers and looked up and said, “you know, it’s weird, the schedule has a 15 minute break between lunch and the first afternoon session. This is supposed to start at 2:00 PM”  I looked and she was right.  As far as I can tell, when the organizers laid out the sessions, they put a 15 minute break between them, and simply did the same for after lunch. This explained why the second pair of people had left with the intent to come back. They wanted good seats for the 2:00 PM start.

Sure enough, by 2:00 PM the room was fairly full and I was off and running. I was in a smaller room than Kalen’s presentation, where she had 40 or more, I had perhaps a dozen. But I was happy and content. And, once it was over, both the room monitor and myself reminded folks to give feedback and this audience was great at that.

A word on feedback. The forms at SQL Saturdays tend to be fairly standard and I think I speak for most presenters when I say, that while it can be gratifying to get all 5s on the top of the form, it’s also kind of useless. But, when folks actually take time at the bottom of the form to give actual written feedback, that’s quite gratifying. This audience gave great written feedback.

I also appreciate feedback in person. At least one person came up afterwards to say, “That was really great, I bet you could do an hour on each System Database.”  So perhaps, I will do an hour presentation on the TempDB someday!

So, I feel redeemed. Due to a variety of reasons it’s unlikely I’ll bid to speak at Colorado Springs next  year, but I’d highly recommend it for anyone in the area. And, if you’re afraid that some other presenter might overshadow you because they’re better known or their topic is similar to yours, don’t despair. Seriously, there’s enough knowledge to go around and enough interest.