Kids, get off my lawn!

Change can be hard. But sometimes it’s necessary. And a lot has happened this week.  First, I want to congratulate my fellow #SQLFamily member Cathrine Wihlemsen on one more orbit of the Sun. Apparently, in her honor Microsoft decided to release SQL Server 2019 on her birthday! I’ve been using SQL Server since the 4.21a days. Every version has had new features and required learning something new. As I said recently, it’s easy to fall into the trap of being an old dog and not learning new tricks. This is something we have to avoid. Being trapped in the past can be limiting.

Besides SQL Server 2019 dropping this week, I recently upgraded my phone. I had been using a Windows Phone for about 5 years now. I loved it. Especially when it first came out, it was top of the line and had a bright future. I eagerly downloaded apps and it became part of my life. But alas, we know how well Microsoft did in the Windows Phone market. But I doggedly held on, even as features were deprecated. I couldn’t use the Weather App. The Amtrak App went away. Eventually several features of Cortana stopped work as Microsoft stopped supporting them. Slowly my phone was becoming a brick. I kept debating do I upgrade to one more Windows Phone knowing it’s the end of the line, or what? I kept putting off the decision. After the mapping function failed me on my recent trip to the Hampton Roads User Group Meeting I decided it was time to finally time to replace it with an Android phone. Choosing from the plethora out there was not fun. It was very tempting to go with one of the top of the line models, but spending $1000 or so wasn’t really a fun idea.  I eventually ended up choosing a Samsung A50.

I’m mostly happy with it. Right now I’m struggling with what parts of it are “get off my lawn” because I don’t like change, and what parts are “what the hell is the UI doing now?”  Fortunately, my son has mentioned some of his dislike of certain UI functionalities, so I think not all of it is me simply being an old curmudgeon (are there young ones?) I will say what I’m most happy with is that Microsoft has a number of tools including the Windows Launcher and the Phone Companion, as well as the obvious apps like Outlook and other parts office.

A word about the Phone Companion. This alone has made the upgrade a win. One of the features is that when I’m working at home (I have not yet enabled it on my Surface Pro) is that things like text messages pop-up on my desktop screen. This actually makes life a LOT easier, since I can simply type a reply from a full-size keyboard or copy the numerous soft-tokens I get to log into various client sites without having to pick up my phone. It’s a small detail, but a wonderful one!

The Launcher helps me retain some of the features that I liked about my Windows Phone. Overall, it’s a win.

But the changes in my life aren’t complete. As I mentioned last week I’m at PASS Summit again this week in Seattle. But alas, this is the last year that PASS Summit will be in Seattle. Next year it will be held in Houston. Just as I’ve figured out where the cheapest and most convenient parking for me is, where some decent food places are, and I’m feeling, if not at home in Seattle, at least comfortable, next year is a big change. I won’t be able to stay with my college friends or do our annual Thai pot luck with a bunch of ROC Alumns.

But, I’ll get to explore another city. I’ve been to Houston only once, literally decades ago, to do SQL Server install at Exxon. The server was literally the only Intel computer in a room full of mainframe equipment. I suspect that has changed since then.  That was one of my early experiences installing SQL Server (4.21a for the record).

So, this old dog is still learning and looking forward to new experiences: plus ça change, plus c’est la même chose.

 

Small Victories

Ask most DBAs and they’ll probably tell you they’re not a huge fan of triggers.  They can be useful, but hard to debug.  Events last week reminded me of that. Fortunately a little debugging made a huge difference.

Let me set the scene, but unfortunately since this was work for a client, I can’t really use many screenshots. (or rather to do so would take far too long to sanitize them in the time I allocate to write my weekly blog posts.)

The gist is, my client is working on a process to take data from one system and insert it into their Salesforce system.  To do so, we’re using a 3rd party tool called Pentaho. It’s similar to SSIS in some ways, but based on Java.

Anyway, the process I was debugging was fairly simple. Take account information from the source and upsert it into Salesforce. If the account already existed in Salesforce, great, simply perform an update. If it’s new data, perform an insert.  At the end of the process Pentaho returns a record that contains the original account information and the Salesforce ID.

So far so good. Now, the original author of the system had setup a trigger so when these records are returned it can update the original source account record with the Salesforce ID if it didn’t exist previously. I should note that updating the accounts is just one of many possible transformations the entire process runs.

After working on the Pentaho ETL (extract, transform, load) for a bit and getting it stable, I decided to focus on performance. There appeared to be two main areas of slowness, the upsert to Salesforce and the handling of the returned records. Now, I had no insight into the Salesforce side of things, so I decided to focus on handling the returned records.

The problem of course was that Pentaho was sort of hiding what it was doing. I had to get some insight there. I knew it was doing an Insert into a master table of successful records and then a trigger to update the original account.

Now,  being a 21st Century DBA and taking into account Grant Fritchey’s blog post on Extended Events I had previously setup a Extended Events Session on this database. I had to tweak it a bit, but I got what I wanted in short order.

CREATE EVENT SESSION [Pentaho Trace SalesForceData] ON SERVER
ADD EVENT sqlserver.existing_connection(
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.login(SET collect_options_text=(1)
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.logout(
    ACTION(sqlserver.session_id)
    WHERE ([sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.rpc_starting(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.sql_batch_completed(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho')),
ADD EVENT sqlserver.sql_batch_starting(
    ACTION(sqlserver.session_id)
    WHERE ([package0].[greater_than_uint64]([sqlserver].[database_id],(4)) AND [package0].[equal_boolean]([sqlserver].[is_system],(0)) AND [sqlserver].[username]=N'TempPentaho'))
ADD TARGET package0.ring_buffer(SET max_memory=(1024000))
WITH (MAX_MEMORY=4096 KB,EVENT_RETENTION_MODE=ALLOW_SINGLE_EVENT_LOSS,MAX_DISPATCH_LATENCY=30 SECONDS,MAX_EVENT_SIZE=0 KB,MEMORY_PARTITION_MODE=NONE,TRACK_CAUSALITY=ON,STARTUP_STATE=OFF)
GO

It’s not much, but it lets me watch incoming transactions.

I could then fire off the ETL in question and capture some live data. A typical returned result looked like

exec sp_execute 1,N'SourceData',N'GQF',N'Account',N'1962062',N'a6W4O00000064zbUAA','2019-10-11 13:07:22.8270000',N'neALaRggAlD/Y/T4ign0vOA==L',N'Upsert Success'

Now that’s not much, but I knew what the Insert statement looked like so I could build an insert statement wrapped with a begin tran/rollback around it so I could test the insert without actually changing my data.  I then tossed in some set statistics IO ON and enabled Include Actual Execution Plan so I could see what was happening.

“Wait, what’s this? What’s this 300K rows read? And why is it doing a clustered index scan on this table?”  This was a disconcerting. The field I was comparing was the clustered index, it should be a seek!

So I looked more closely at the trigger. There were two changes I ended up making.

       -- Link Accounts
       --MERGE INTO GQF_AccountMaster T
       --USING Inserted S
       --ON (CAST(T.ClientId AS VARCHAR(255)) = S.External_Id__c
       --AND S.Transformation in ('Account'))
       --WHEN MATCHED THEN UPDATE
       --SET T.SFID = S.Id
       --;
       
       if (select transformation from Inserted) ='Account'
       begin
              MERGE INTO GQF_AccountMaster T
              USING Inserted S
              ON T.ClientId  = S.External_Id__c
              WHEN MATCHED THEN UPDATE
              SET T.SFID = S.Id
       end

An astute DBA will notice that CAST in there.  Given the design, the Inserted table field External_Id__C is sort of a catch all for all sorts of various types of IDs and some in fact could be up to 255 characters. However, in the case of an Account it’s a varchar(10).

The original developer probably put the CAST in there since they didn’t want to blow up the Merge statement if it compared a transformation other than an Account. (From what I can tell, T-SQL does not guarantee short-circuit evaluation, if I’m wrong, please let me know and point me to definitive documentation.) However, the minute you cast that, you lose the ability to seek using the index, you have to use a scan.

So I rewrote the commented section into an IF to guarantee we were only dealing with Account transformations and then I stripped out the cast.

Then I reran and watched. My index scan of 300K rows was down to a seek of 3 rows. The trigger now performed in subsecond time. Not bad for an hour or so of work. That and some other improvements meant that now we could handle a few 1000 inserts and updates in the time it was previously taking to do 10 or so.  It’s one of those days where I like to think my client got their money’s worth out of me.

Slight note: Next week I will be at PASS Summit so not sure if/when I’ll be blogging. But follow me on Twitter @stridergdm.

Hampton Roads User Group Recap

I’ve talked about how I think it’s important to be part of the #sqlfamily community and how I enjoy talking and giving back. Last week was another example of this. Much earlier this year (it might have even been at Pass Summit last year) I convinced Monica Rathbun to do a quid pro quo. I’d speak at her user group in Virginia Beach if she’d come to upstate NY to speak at my user group. I’d seen Monica speak and knew she would be a great speaker for my group. Fortunately, despite seeing me speak, she apparently felt I’d be good enough for her group.  Seriously though it was a good deal.

My original plan had been to drive down Wednesday, address her group, stay at an AirBnB on the beach and then spend a few nights in the Washington DC area visiting with some friends.  Unfortunately, less than a week before I was ready to head down, my DC plans fell through. This radically changed my travel plans and I scrambled to make various plans to make the trip a practical one and one that wouldn’t break my budget. One of the unfortunate facts of being as consultant is that I don’t have an employer that can cover travel expenses. On the other hand, I often have a lot more flexibility in when and how I travel.

I ended up taking the train to Wilmington Delaware and getting a rental car from there. This allowed the most flexibility, was second in time to flying, and overall the least stressful. I love taking the train because I can sleep (which I did on the Albany to NYC segment) and get work done (which I did on the NYC-Wilmington segment, working on a future article for Redgate Simple-Talk and reviewing my talk) Unfortunately, due to a missed turn, some slow traffic due to the rain and then the rain in general, rather than showing up at 5:30 like I had hoped, I was in the door at 6:15 or so. This gave me time for a single chicken wing before I launched into my talk.

I had been monitoring the Meet-up page to see how many people were expected and at my last count it was 8. I was comfortable with that. I was hoping for more, but hey, I’ll take what I can get.  Imagine my surprise when I walked in and there were closer to 20 people there. Honestly, a great turnout! But, between running late, the usual hardware issues of getting my laptop and the monitor talking, and not being able to get one last run through of a 10 minute section of my talk, I’ve got to say I was a bit flustered.

I love to teach. But I would be lying if I said I don’t love it when I see or hear a student have what I call that “Aha moment!” This is that moment when you explain or demonstrate something and you can see the look in their eyes or the tone in their voice when something just clicks. It might be a small thing to you, but for them you’ve just rocked their world.

A number of years ago while teaching the Level 2 cave rescue class in Colorado, we were doing an instructor lead evolution. During these, the instructors take the lead and guide the students through the problem. It’s usually the first real new teaching experience of the week-long class, before that it’s mostly review. In this case I had a single student working with me and we were charged with setting up two lines to be used as a safety and for another purpose.  I told her to grab a single rope and a carabiner. She looked at me questioning because she knew we needed to have two lines rigged. I then showed her the tree I had selected and told her to basically double the rope, tie what’s known as a high strength tie-off using the middle of the rope, clip it in with the carabiner and toss both ends down. Then the aha moment, “wow, I’d never thought of that. That’s worth the price of the class right there.” I’ve got to say I was proud. My job was done, 2nd day of teaching. I could take the week off. Of course I didn’t.

This time around, I was talking about the Model Database and how most DBAs completely ignore it and overlook it. I was demonstrating how when you put objects in it or change various options in it (such as from Simple Logging to Full Logging) any new databases will pick up those objects or options (unless you override the options using a script.)  As I was bending over the keyboard to type the next demo I heard it, someone in the middle of the classroom suddenly said, “Woah…” and you could tell their world had just been expanded. That alone made the entire 36 hours (including travel time, sleeping etc) of the trip worth it. I knew someone had learned something. I live for those moments.

Don’t get me wrong, I enjoy getting paid as a consultant, but honestly, I speak on SQL Server topics and teach cave rescue for those aha moments, for knowing that I’ve just expanded someone else’s world a bit.

Oh that, and in this case, the free wings!

ehb5jp6w4ae8-at.jpg

Tasty wings at Hampton Roads SQL Server User Group

Just a reminder, I will be at the 2019 PASS Summit in Seattle and look forward to meeting with anyone who wants. My Twitter handle is @stridergdm and I often hang out with the folks at MinionWare (they’ve got a comfortable couch) and will be attending the Birds of the Feather luncheon (undecided where I’ll be sitting) and the Women in Technology Luncheon.

And I’m hoping for my nest article on PowerShell for Redgate’s Simple-Talk to be submitted before then!

 

NY ComicCon

Last week I talked about Kids These Days. This past weekend I went with my daughter to NY ComicCon. It was a late 8th grade graduation present she had requested. Due to me messing things up last year, we missed our chance to go, so I made up for it this year. And it was well worth it, for a couple of reasons. I want to focus on two, one topical and one personal. The topical first.

The topic is in the above photograph.  I apologize for it being blurry, “I’m a DBA Jim, not a photographer.”  But I took it for a reason. This was the panel for a talk titled: Join the Resistance! (Journey to Star Wars: The Rise of Skywalker). It was an interesting panel that talked about the books they wrote that cover the time between The Last Jedi to this December’s The Rise of Skywalker. But partway through listening, something dawned on me about the panel.  Can you figure out what I realized?

It’s there in the picture, but if not, let me list the panelists: authors Rebecca Roanhorse, Justina Ireland, Kevin Shinick, Ethan Sacks, Delilah S. Dawson, audiobook narrator Marc Thompson and moderator Ashley Eckstein.

What strikes you about that list of names? Now compare that to the panels you see at a number of tech events such as various SQL events. Note what it’s not. This is NOT a MANEL!

Science Fiction has for far too long been treated as the domain of boys and then later men. Marketing for decades often focused on boys. It was assumed that every boy wanted to be Han Solo or Luke Skywalker or Captain Kirk.  Women in shows and books were often only there as props for the male characters to react to. Granted, this statement isn’t 100% true, even Princess Leia had some meat to her character in the original Star Wars (back before it was episode IV or A New Hope.) Even then though, she served the role set out in much of mythology as the princess in distress to be rescued. Fortunately her role and the role of women in Star Wars was greatly expanded over the series, to the point now where Rey is our hero.

Ahsoka Tano in triplicate!

Ahsoka Tano in triplicate!

And this panel shows exactly how equitable the Star Wars universe has become. The moderator was Ahsley Eckstein, who voices the character Ahsoka Tano in various animated Star Wars series. Three of the authors on the panel were women. In other words, women were well represented.

Think about this when planning your tech event such as SQL Saturday. Do you have equal representation? “But wait Greg, there’s just not that women doing SQL! I only had 3 women apply to talk and 30 men!” I’m going to give you some advice. Ask for more women. Talk to those three, see if they know anyone who might want to speak, but was too nervous to put in a submission. Talk to Kathi Kellenberger and Rie Irish of the PASS Virtual Group Women in Technology.  Yes, there may not be as many women in tech as men, but I can guarantee that there’s more than you think and that it won’t change without encouragement and representation. If you as a guy get invited to speak on a panel, make sure there’s diversity. Turn down opportunities if it looks like it’s going to be a manel. Call out your fellow community members if they’re engaging in sexist behavior. It’s not always comfortable,especially if it’s a friend or a co-worker, but it needs to be done. Do your part.

If ComicCon can have an equitable panel in regards to Star Wars, you can do the same in regards to SQL or other tech panels.

Now for the personal:

Live Long and Prospoer

Autograph and picture with two amazing women, Nichelle Nichols and my daughter Rebecca.

Two amazing women: Nichelle Nichols is an amazing woman and helped represent African Americans on television in the 1960s and helped inspire people like Whoopi Goldberg and Mae Jemison. And as for my daughter, her future and journey is in front of her.  I will admit to basically being speechless in front of such an icon and here I am, still three days later grinning ear to ear thinking “I was in the presence of Uhura!”

(BTW, for those who recognize it, that’s a 1st edition Star Fleet Technical Manual with her signature. It also contains the signature of George Takei and James Doohan.)

 

Dress for Success?

“Dress for the job you want, not the one you have.” This is advice I once heard years ago. Of course I’m not sure what you do if the job you want is the one you want.

Back around 1999 I mentioned to my dad what I was wearing to work. I think I mentioned something about cargo pants and hiking shoes. He admonished me that perhaps I should dress more appropriately for the office and see what the COO and CFO were wearing as an example.  I replied, “Dad, they wear shorts and sandals without socks to the office. I actually dress up more than the COO and CFO do!”  It took me awhile to convince him that in the new dot-com era, not everyone was wearing a shirt and tie to the office.

This all came to me yesterday afternoon as I was deciding what to wear to the Capital Area SQL Server User Group meeting. Since I’m generally the host, I do want to project a professional, but relaxed atmosphere. So, my usual fallback is khakis. But, I was also the speaker so I figured it wouldn’t hurt to make sure I dressed even a bit more appropriately. Fortunately, having been to a number of SQL Saturdays, I had more than enough choices. I ended up with my SQL Saturday Albany 2016 shirt. But as it is starting to be cool here, I figured tossing a top over that would work, and funny enough, I had my Chicago SQL Saturday 2017 top to toss on over it. I’m nothing if not a shill for SQL Saturday!

My kids will claim I have a certain style when it comes to what I wear and they’re right. And while it may seem I often don’t give much thought to what I wear, the truth is, my choice of clothing, especially t-shirts, is often far more deliberate than it may appear.  I just don’t let on often to that fact.

And the truth is, between t-shirts from SQL Saturday and from National Cave Rescue Commission trainings, I probably have close to a month’s worth of shirts if need be.

Makes me wonder, do I volunteer because I like to give back to my community, or because I need the t-shirts? Hmm.

 

 

Old Dogs and New Tricks

One problem with writing a weekly blog is sometimes you get the same idea in your head and realize you’re about to repeat yourself. I want to write about learning new stuff and started to repeat stuff I said here. I suppose I could just put a pointer to that and be done but… nah.

Several ideas prompted this week’s theme for this blog, the most recent being Grant Fritchey’s blog on using Extended Events. He makes some good points and I want to add my own thoughts.

I’ll start by admitting I still often use Profiler. I know it. It’s “easy”.  Well, no, not really. A better description would be “It’s familiar”. Truth is, I’ve often found the interface a bit clunky and I have to do more steps than I want to narrow down to trace exactly what I want.

But, about two weeks ago, one of my clients had an issue with a run-away query that expanded their tempdb to about 400GB before it rolled back. We wrote it off as a fluke; until it happened again the next day.  Now we had a pattern. I decided that we needed to monitor the server about the same time the next day to see if it would occur again. Now, of course one can sit there and run queries like sp_whoisactive, but I wanted more.  Profiler might work, but I figured it was time for this old dog to learn, or at least practice new tricks. I’ll admit, I basically googled for what I wanted, but I quickly found a trace I could modify and run for my needs.  30 minutes later and I had a nice extended event setup and monitoring the tempdb.

Now, as fate would have it, that run-away query hasn’t shown up since then, so in a sense the trace hasn’t fulfilled it’s goal. But, it’s lightweight enough that I’ve decided to leave it. And, the results are easily accessible to others if I’m out of town.

In the past year or so, it’s really hit me how much my role as a DBA or IT professional has really changed from two or more decades ago. There’s stuff that I do now that wasn’t possible then and there’s stuff then that I would do that I don’t worry about now or can’t even do (who out there recalls the ability to setup SQL Server database “files” on raw partitions for the speed boost?)

Over two decades ago I wrote a fairly impressive batch file that could install a client’s application on the proper drive. It used a lot of commands most people weren’t even aware of in the batch language (this was DOS, so not even as good as what CMD in Windows NT and above gives us).  Nowadays, I’d use PowerShell.

My original programming was in Fortran. Nowadays, I tend to sling code in VB.Net or C#.

It’s ok at times to fall back on what’s familiar. Often it can be faster and easier for a single project. But in the long-term, one really needs to learn what’s new and apply what one can.

My advice, pick a new technology or skill, and use it, even if for one project.  I’m not going to be an expert in Extended Events anytime soon, but now at least I can say, I’ve used it.  I’m far from an expert in PowerShell, but I’ve now been paid for 3 (and soon 4) articles on using it.

If this old dog can learn a new trick so can you.

And, listen to music you didn’t grow up on. As much as I love to listen to the music I’m familiar with while working, I often branch out. Yes, I’ve been known to listen to some Taylor Swift and Charlie XCX. And yesterday was a bit of Imagine Dragons and Meute. Two very different styles of music, but still a good listen.

What new trick will you learn this month?

“Don’t be so Sensitive”

I’ve mentioned in the past that I have an interest both in the SQL Server world, and the caving world. Often these both overlap in different ways, for example disaster planning and the like.

The other day I was reminded of another way in which they overlap: the ratio of men to women in each activity.  In both areas, though I don’t have firm numbers, far more men participate than women. There are a number of reasons for this, but one I’ll call “the good ol’ boys” attitude. I discussed this in a previous post concerning women in the industry. Recently however I was reminded that sexism continues to be a problem in the caving community. On Facebook, I’m a member of a number of groups with a focus on caving. The other day someone saw fit to post a picture in one of the groups I’m a member of. The picture was of a young woman, in a sports bra and short tights wearing a rock-climbing harness, and holding on to a dangling rope.

Now, there were several problems technically wrong with the picture, including the fact that she was wearing a rock-climbing harness and this was a caving group and the fact that the harness was on backwards.  But, that wasn’t the real problem.  The real issue was, this was that it simply was not appropriate for this group.  Several members posted pointing out that this picture, and pictures like this, objectify women and discourage them from caving.

And then it came, a guy saying, “Don’t be so sensitive.”

In four words, he casually dismissed the concerns and feelings of a large number of his fellow cavers.  He said, without realizing it, “I don’t care how you feel. Your feelings and concerns are not important to me.” In my experience, these are the very same men who then complain there aren’t enough women in caving.

Similar comments included, “Oh, now you’re saying she’s wrong to be proud of her body” or “what’s the matter with a hot body”.  Here the underlying subtext is that anyone who expressed an issue with the photograph in that group was a prude.

I’ve seen similar comments at times in the IT community; “What’s the matter if I call her pretty, she should be proud of that!” “What, DBAs can’t be hot too?”

What some of my fellow cavers and IT professionals fail to understand is that the women in these circles want to be considered by the same standards as their male companions, on their skills and accomplishments, not on what their body looks like.  This does not make them prudes.

Nor, when these same women dress up for a cocktail party, or have a beer, or crack a ribald joke does that make them hypocrites. This is also an important concept for many men to understand. The women I know who cave and are DBAs are just as complex and varied as the men. Some like to dress up, some like to tell off-color jokes in the appropriate setting, some like to smoke a cigar, and often do all the same things that their male compatriots do.  But when it comes to caving or IT, they want to be respected for their skills, not judged for these other attributes.

So, next time you’re about to post a hot sexy photo, or make a comment on a woman’s appearance, ask yourself, “is this the appropriate place for this? Would I do this if it was a hot sexy photo of a man? Would I make the same comment towards a man?” And as a hint, if the name of the group is something like, “Professionals in IT” or “Cavers of the World” the answer is almost certainly “no”. Remember, your fellow cavers and IT professionals are judging you.