Starting off the New Year… wrong

Ok, I had plans for a great blog post on time and how we measure it and how it really is arbitrary. But… that will have to wait. (I suppose that brings its own irony about time and waiting and all that. Hmm, now I may need a post about that!)

But, circumstances suggested a different post.

This one started with an email from my boss last year (who will remain nameless for his sake. 🙂  “Greg, I hacked together this web page so my team-members can track their hours. I need you to make a few tweaks to it. And don’t worry, we’ll replace it by the end of the year.”  Yes, that sound you hear is your head hitting the desk like mine, because you KNOW what is coming next. Here we are in January of 2018 and guess what, the web page is still in place.

And of course, it’s not just HIS (now former, he’s moved on) team that is using it. Apparently his boss liked what it could do, so his boss (my boss^2) declared that ALL his teams would use it.  So instead of just 4-5 people using it, there’s close to 30-40 people using it.

So, remember the first rule of development, “Oh don’t worry about this, we’ll replace it soon” never happens.  His original code made a lot of assumptions (which at the time I suppose seemed reasonable). For example, since many schedules are formed around work-weeks, he was originally storing hours by the week (and then day of the week) they were worked in. For example, if someone worked for 6 hours on January 5th of last year, the table stored that as “WEEK 1, day 5”.

Now, before anyone completely thinks this is nuts, do keep in mind, many places, including my last job at GE did everything by Week of the Year.  There’s some advantages to this (that I won’t go into here).

For the most part, the code worked and I didn’t care. But, at one point I was asked to do a report based not on Work Weeks, but on an actual calendar month. So, I had to figure out for example that February 1st occurred during Work Week 5, on the 4th day. And go from there.

So, I hacked that together.  Then where was a request for another report and another. Pretty soon it was getting to be a pain to keep track of all this. And I realized another problem. My boss had gotten lucky in 2017. January 1st occurred on a Sunday, which meant that Work Week 1, Day 1 was a natural starting point.

I started to worry about what would happen when 2018 rolled around. His code would probably break.  So I finally took the plunge and started to refactor the code.  I also added new features as they were requested.  Things were going great.

Until yesterday.  So now we get to another good rule of development: “Use Version Control” So simple. Even for a one person shop like me it’s a good idea. I had put it off for awhile, but finally downloaded the git plug-in for Visual Studio and started to use it.

So yesterday, a user reported that they couldn’t enter hours for last week. I pulled up the app and realized that yes, in fact it was coded in such a way you couldn’t go back into a previous year to enter hours. No problem, I can fix that!

Well, let me add a rule I had sort of missed; “Understand HOW to use Version Control”. I won’t go into details, but let’s just say, I wasn’t committing like I should have been or thought I was. So, in an attempt to get a clean base and all, I merged things… “poorly“. I had the branches I wanted, but had not properly committed stuff to them.

The work of the last few weeks was GONE. I know, you’re saying, “just go to your backups Greg, because of course a person who writes about DR has proper backups, right?”  Yeah, right! Let’s not go there!

Anyway, I spent 4 hours last night and 2 this morning recreating the code. Fortunately, it dawned on me, being .NET code, it was really just CLR and that perhaps with a good decompiler, I could get most of the code back without too much effort. I want to give props to RedGate’s .NET Reflector. Of the two decompilers I tried, it was clearly the better one. While I lost my variable names and the decompiled code isn’t quite what I’d call human written, it was close enough I could in short order recreate hours and hours of work I had done.

In the meantime, I also talked to some of my programming buddies on an RPI chat server (ask me about it sometime) about git and better procedures.

And here’s where I realized my fundamental mistake. It wasn’t just misunderstanding how Visual Studio was interacting with git and the branches I was creating, it was that somewhere in the back of my head I had decided that commits should be used only when major steps were completed. It was almost like I was afraid I’d run out; or perhaps complicate the history with too many of them.  But, you know what? Commits aren’t scarce resources. And I’m the only one reading the history. I don’t really care if I’ve got 10 commits or 100. The more the better in many ways. Had I been making commits much more frequently, I’d have saved myself a lot of work. (And I can discuss having a remote store at some point, etc.)

So really, the point of this hastily, sleep-deprived written blog isn’t so much to talk about dates and apps that never get replaced, but about a much deeper problem I was having. It wasn’t just failing to fully understand git commands, it was failing to understand how I should be using git in the first place.

So, to riff on an old phrase, “Commit Early, Commit often”.

Oh and I did hack together a way for that user to enter hours for last year. It’s good enough. I’m sure I won’t need it a year from now. The code will be all replaced or refactored by then I’m sure!

 

Maps

This is actually about a bit more than simply maps. It’s really about what a map is. It’s an abstraction. This may sound obvious, but it’s an important concept.

“I have a map of the United States… Actual size. It says, ‘Scale: 1 mile = 1 mile.’ I spent last summer folding it. I hardly ever unroll it. People ask me where I live, and I say, ‘E6.” – Stephen Wright

Forget the impracticality of such a map, it makes a fine joke, it’s still just a map. We don’t live on maps.  Piaget and others have done a lot of research into the development of the minds of young children and it’s fascinating. At first, I think one can argue they live in a very solipsistic world. Nothing exists except what they can see or touch. Soon though concepts of object permanence come into being. But they’re still not quite at the concept of true abstraction. At very young ages, if you hold up a picture of a room to a small enough child, they’ll try to grab stuff of the picture. And why not? They can grab things in the physical world.  Soon they start to realize the picture is just that, not the actual room, but a flattened portable version of it.

Around age 5 (I’m going on memory here) we start to ask children to draw maps. It might be a map of their room, or a map of how to get to school. They’re crude and often, to our adult minds wildly inaccurate, but, they’re starting to achieve a truer concept of abstraction.  Their maps aren’t even miniaturized pictures of the real world, they’re completely new, abstract designs of the world.

Eventually, around 8th grade or so, they’re probably introduced to Algebra. Up until then much of their math might have been things like 8 * ___ = 56. They’ll dutifully fill in the 7. It’s a blank, but it’s not really abstract. Then they start to solve problems like “56/x = 8, solve for x”.  Again they’ll write down 7, but now they’ve learned that a letter can represent a number and be used that way later on.

What’s interesting about the above is, many people never stop to realize that * or / or + or many other mathematical symbols are really just abstractions. Yes, numbers are in a sense an abstraction. There’s nothing inherent about the symbol 5 that means it MUST represent five of an object. It’s just a convenience (and it’s shorter than writing ///// hash marks). But the mathematical symbols are even more of an abstraction. If I wrote 8♒︎__ = 56, most of us could probably figure out what ♒︎ represents there.

Anyway, that aside, why am I writing about maps and ultimately abstraction? Because, it is so vitally important to my job and perhaps the jobs of most of my readers and it’s been on my mind lately.

I’ve been working on an SSIS package for a client. The original spec was “Import these 20 or so (out of 240) CSV files out of this ZIP file.”  I thought about it and after teaching myself a bit more about SSIS (I don’t usually do much in it) I realized I could create a container that did most of the work. And I could assign variables to the container. The variables include things like what database connection to use and where the files might be located. And, most importantly, a variable for the NAME of the file/table (by using a table with the same basename as the file, I can use a single variable and line stuff up nicely.) And part of what I had to (re)-learn was the scope of variables within SSIS packages and how to expose the ones I wanted as parameters.

Now, honestly, it probably took me more time to set up this level of abstraction to make it workable than if, for the 20 or so tables I had hard-coded everything.

BUT, I’ve got enough experience to guess at what was coming next; and I’m going to guess you know what’s coming next.

“Hey, this is great, but you know what, we think we want all 240 files in that ZIP file loaded. Can yo do that?”

“Sure!”  And at this point, the problem isn’t the amount of work, it’s the tediousness of it; “create table schema, create a new file connector, copy the SSIS container, update the variable for the table, go in and reconnect the file connect to the new table, test”.  It’s a LOT less work than if I had to hardcode everything by hand.  The container name even automatically is updated based on the table variable. This actually has an unintended, positive side-effect. If for some reason I start to duplicate a container (say I forgot I had already imported table Customer_Val, the container name won’t update using the variable since you can’t have duplicate container names. I caught this very problem the other day. So this was an unintended, but very useful additional side effect!)

And of course, once I move the final code to their UAT environment and then the Prod environment, it’s simply a matter of updating 3-4 variables in the deploy (for the database server, file location and where to set emails) and I’m done.

By taking the time to abstract out as much as I could, I’ve made my life easier. Up front it was definitely a bit more work, but in the long run, has saved me a lot of effort.

Nothing revolutionary about the concept to us developers. BUT, stop and try to think of an environment where maybe variables or such layers of abstraction don’t exist. Such environments exist and there’s some valid reasons for them, but ultimately, being able to abstract stuff can make our lives MUCH easier and our code that much more powerful.

And it appears that abstraction at this level is perhaps the ONE real intellectual advantage we have over other intelligence on this planet. And I’m not even so sure about that!

So I’ll end in pointing out that even the written word is really just a level of abstraction. So this column is about itself. Or something like that.

 

Star Wars: The Last Jedi

Ok, fair warning there may be spoilers ahead.  So unless you’ve seen the movie, or don’t care, postpone reading this particular post for awhile.

So let me start by saying, I loved it. I had some issues with it, but overall I loved it.

But, this post isn’t a movie review per se. It’s more about how a story might be crafted.

I have a daughter, 14 now, who was 12 when The Force Awakens came out. I can’t recall when she became a Star Wars fan, but she’s a huge fan and I love that about her.  I introduced her to the original trilogy and, only because the movies are about the good and the bad, the prequels. I still recall while watching The Phantom Menace she turned to me and said, “Dad, this makes no sense.”  She was right. But that didn’t stop her from watching all of them, The Clone Wars and pretty much anything else she could get her hands on that involved Star Wars.

So, 2 years ago, she told me for my birthday she wanted to take me to The Force Awakens. We went after school a day or two after it opened. The theater we went to was nearly empty, and honestly, not a great one. (I’ve come to really enjoy stadium seating, if only so it reduces the chances of someone’s head in front of me).

The theater darkened, the previews passed and that iconic chord rang through my ears and the scroll started; and I was looking at my daughter’s face. Suddenly, I was transported back to 1977 and I was 9 and I was watching Star Wars (it had no subtitle back then, let alone episode number) for the first time, all over again. I realized that at that point it didn’t matter if the movie was good or not (it was) I was being given a gift she didn’t realize she was giving. I was watching Star Wars through the eyes of a child.

That said, I did watch the movie. But I did turn to look at her one more time; the scene where Han is confronting is son, telling him to do what he knows he must do.  I knew what was coming. I also knew my daughter didn’t. The look of shock on her face as Kylo committed patricide was clear and apparent. She was shocked and upset.

We left the theater and she was clearly upset. “How could Han die! He’s a major character.” We stopped for dinner at Moe’s Southwest Grill and we talked. I explained how the scene almost certainly HAD to happen; that the movie was about Poe, Rey, Finn; that in a sense, the past had to fade to the past or even be killed off so that the story could advance. I also explained how Kylo’s character had to develop and that this was one very definite way to do so.  She wasn’t happy then.  A day later though she understood my point.

I think this was the point where she went beyond simply watching the story and trying to understand WHY a story is crafted the way it is. Sometimes it’s obvious, sometimes, not so obvious. Or a story may go in a direction the viewer may not like.

<SPOILER COMING>

Last warning!

So, again, this year she took me to see a Star Wars movie for my birthday. It’s a tradition I definitely enjoy. On the way to The Last Jedi, we talked about what Luke would do when she handed him his original light saber. I jokingly, but only 1/2, said, “eh, he’s going to throw it over his shoulder.”  So of course when the scene happened, I watched her face. It was worth it.  I say only 1/2 jokingly because again, to advance the plot and to make it more successful, I felt the movie had to break with the past. Him simply becoming a Jedi Master again much like Obi-Wan would have just made that part of the movie a rehash. Instead, we explored new ground. What happens when the Master doesn’t want to be? How does the apprentice react? How do WE as an audience react?

Luke asks Rey several times, “Why are YOU here.” She continues to respond with “Leia sent me”. Luke doesn’t accept that answer and neither do we.  We even know Rey knows it’s not the honest answer.  She finally gives the honest answer. But it’s not until much later in the movie she finally learns what is important; something far more important than anything a Jedi Master can teacher her, and it’s the one lesson that Kylo learns in the negative.  And as any well crafted movie, they learn their opposite lessons from both the exact same place and from very different places. But I won’t say more.

BUT, as I do try to tie these blog posts into something relevant to my overarching goal of this blog:

Ask yourself; “Why am I here?” “Why am I doing this job?” I once interviewed a candidate for a position. His answer was, “For the money.” While important, it wasn’t the answer I needed to hear. There’s always another, higher paying job down the street.  If you’re in it for only the money, you’ll never be content because you’ll always want a bit more. (that said, I don’t suggest working for less than you’re worth either.)

And when facing a problem. Don’t simply ask, “what happened” “Oh, Kylo killed Han”. No, ask yourself, “Why did it happen?” or “Why did it have to happen THAT particular way.” “Kylo killed Han to advance the plot and his character development.”

Ask the question behind the question. Get to the truth.

And every once in awhile, take the time to see the world again through the eyes of youth.

SQL Saturday DC 2017 Follow-up

Last week I wrote about heading towards SQL Saturday DC. This week I figured I’d write a follow-up.

I want to start by saying thanks to everyone who organized and set things up and ran it. I’ve only been on the periphery of working the Albany SQL Saturday, but I have an inkling of what it takes to run an event like this. There’s so much work, much of it obvious, but also a myriad of small details, many that never get seen or noticed.

Since I like to talk about “thinking” in this blog I’m going to mention one such detail that I’m sure no one gave much though about, but was there. Chris Bell is one of the principal organizers of the DC event (but far from the only one who works it). I’m going to give away one of his secrets (and I hope he’s ok with that!)

So one of the decisions an event organizer has to decide is how long to make the sessions. Generally they run 60 minutes or 75 minutes. (Occasionally you might see some that run 90 minutes). 15 minutes may not seem like much, but over 4-5 sessions, it can mean the difference between adding or removing an entire session during the course of the day.

Now as a speaker, I can’t say I have a real preference. A 60 minute session often feels like I have to rush a bit, but a 75 minute session tends to mean I may have to add content, OR perhaps end a bit early, especially if there’s not a lot of questions at the end.

And that last detail is a critical detail that Chris took advantage of.

See, one of the logistical issues any major event has to worry about is feeding people. At a large event like PASS Summit, they have an entire and huge room just dedicated for lunch with rows and rows of serving tables. They can, if they had to, get everyone through the lunch line and served in a short period of time. However, at smaller events, with a smaller budget, especially 1 day events, it makes little sense to rent a space that large.

So, if you have limited space and limited time, how do you handle lunch, and especially trying to get your speakers and staff through first (so they can be prepared for their next session)?

You take advantage of the fact that every seminar before lunch isn’t necessarily going to use the full 75 minutes! You start lunch say, 15 minutes before the END of the pre-lunch sessions. Some sessions will end 15 minutes early, so those folks can get in line right away. 1-2 might end 10 minutes early and they can get in line after the previous folks are just finishing getting their lunches.  And so on.

So, you end up serving everyone in a reasonable amount of time, but you don’t have huge long lines all of a sudden. You have a much more reasonable distribution of people across your available time.

So, that’s something to think about when scheduling events. Oh by the way, if you’re doing massive maintenance or dataloads to your SQL Server database, you might want to see if you can spread out your I/O in a similar way. (Hmm, I had to get SOME SQL Server content in here, right?)

So, remember, if you think outside the box, sometimes you can get more done than otherwise!

P.S. I couldn’t finish this blog post without a huge shout-out to the @SQLSpouse and stealth Princess Gigi Bell and say thanks again for the card!

SQL Saturday DC 2017 Prologue

I’ll apologize upfront, not every blog post is mind-shattering and deep. This is one such lightweight one.

Once again I’ll be heading to a SQL Saturday, this time in Washington DC. I actually had two to choose from, one in Providence Rhode Island and this one. Mostly because I have friends in DC from the days I worked there, I choose to submit some sessions to present in DC.

Submitting presentations is always a bit nerve-wracking for me. You feel confident you’ve got a good session and you hope it is what what the organizers want. I’m not really sure how many SQL Saturdays I submitted sessions to before I was finally selected for my first one in NYC. My topic then (and still one I still present from time to time) was “Tips that Saved my Bacon.” I even made bacon cookies to hand out. (Never again, they didn’t come out great.) I knew I was off to a good start when I saw Micron Technologies handing out t-shirts with a bacon motif and even a scent of bacon. Must be kismet.

I get to my room, I get setup, I’m all set. I’m a bit excited and waited for the people to pour in and fill the room. Ok, then I waited for anyone to walk into the room. Finally some people showed up. Great, first hurdle cleared, I had an audience. Not as large as I had hoped with which to share my amazing wisdom, but an adequate one nonetheless.

Before I started, a question, from a woman I believe who was Muslim, “what is bacon?” Of course, with pork being harām to most observant Muslims, she may have never even encountered it. But I realized too, I had a bigger issue. She didn’t understand the idiom!  I was really off to a great start now.  I did the best to explain both what it was and the idiom.  She seemed satisfied. And then we were off.

Fortunately I sort of had a plant in the audience; a friend of mine who was attending who had wanted to hear my talk.  He claims he got a lot out of it.  I hope so.

Since then I’ve given that talk several times and fortunately had better turnouts and better results.

After that I’ve had pretty good luck in getting selected to speak, but sometimes you still get the rejection. I had previously put in to speak at DC at least twice before and turned down both time. You take it in stride and try to not take it personally.

So, again, this year I put in to speak at DC. I submitted three different possible presentations, hoping at least one would be accepted.

The final date for submissions came and went. And nothing. Then I saw they had reopened the period for submissions. This didn’t instill hope in me getting selected.

And still nothing. Oh well, I decided I’d go to SQL Saturday DC anyway, just to attend and to see my friends in DC.

Then I was at the RedGate After Party at Pass Summit and Chris Hyde walked up to me and said, “Congratulations. I really wanted to see your talk in DC, but apparently we’re presenting at the same time.” So, I had to go through my emails and find the one from DC. (That day I had a system that was giving me some warning messages, so I had to sort through about 100 messages to find the one from SQL Saturday DC, hence why I had originally missed it.)

But, as Chris didn’t say WHICH of my presentations he was going to miss, I pulled out my phone, logged into the site. And lo and behold, I discovered I wasn’t presenting just once, but TWICE! I was completely shocked. And… now in a wee bit of trouble.

You see my talk on Who’s flying the plane? What IT can learn from plane crashes, is one of my favorites and one I’ve given multiple times before (and the one apparently Chris will miss). But, my 2nd talk Presently Presenting…. Presenting was one that had hadn’t quite fully written. Ok, I had the outline in my head, but hadn’t written it at all! I generally do NOT recommend this style of doing things. I really like to present at a smaller group first (say my local user group) but I figured this was a good way to give me a kick in the pants and get the talk written. And I was right. The presentation is written, I’ve run through it a few times (and will run through it a few times more before Saturday) and I’m quite happy with it at this point.

So, this coming Saturday, I’ll be giving not just one, but two talks at SQL Saturday in DC. If you’re reading this and already signed up, I’d love to see you there. If we know each other, of course say hi. If we don’t, introduce yourself. I always enjoy meeting new folks.

And if you haven’t signed up, there is unfortunately a wait-list, but you can still add your name to it and if folks cancel, get in.

So, I hope to see you there!

P.S. I’ll give one piece of advice that’ll be in my talk on presenting. If you DO get turned down, don’t take it personally. Take it with grace. SQL Saturday organizers face a lot of challenges in picking presenters and are often overwhelmed with the number of submissions. Trying to argue with them or worse calling them names or getting upset with them is a sure fire way to guarantee you do NOT get selected in the future. And, organizers talk to each other. You do NOT want to get tagged with being “that person”. If you get turned down, don’t take it personally and move on.

Too Secure 2

A quick followup to my blog post from the other day.

So, today I tried to update a service at the client. But of course, with IE locked down and cookies not allowed, I can’t update the service. Hmm. Tell me how that’s more secure?

And my wife just came back from work last night, talking about how she’s no longer able to get to a website critical for her job; because the firewall rules changed.  All this in the name of security.

Yes, we can be too secure!

Too Secure

There’s an old joke in IT that the Security Office’s job isn’t done until you can’t do yours.

There’s unfortunately at times some truth to that.  And it can be a bigger problem than you might initially think.

A recent example comes to mind. I have one client that has setup fairly strict security precautions. I’m generally in favor of most of them, even if at times they’re inconvenient. But recently, they made some changes that were, frustrating to say the least and potentially problematic.  Let me explain.

Basically, at times I have to transfer a file created on a secured VM I control to one of their servers (that in theory is a sandbox in their environment that I can play in). Now, I obviously can’t just cut and paste it. Or perhaps that’s not so obvious, but yeah, for various reasons, through their VDI, they have C&P disabled. I’m ok with that. It does lessen the chance of someone accidentally cutting and pasting the wrong file to the wrong machine.

So what I previously did was something that seemed strange, but worked. I’d email the file to myself and then open a browser session on the said machine and get the file there. Not ideal and I’ll admit there are security implications, but it does cause the file to get virus scanned at at least two places and I’m very unlikely to send myself a dangerous file.

Now, for my webclient on this machine, I tended to use Firefox. It was kept up to date and as far as I know, up until recently, on their approved list of programs.  Great. This worked for well over a year.

Then, one day last week, I go to the server in question and there’s no Firefox. I realized this was related to an email I had seen earlier in the week about their security team removing Firefox from a different server, “for security reasons”. Now arguably that server didn’t need Firefox, but still, my server was technically MY sandbox. So, I was stuck with IE. Yes, their security team thinks IE is more secure than Firefox.  Ok, no problem I’ll use IE.

I go ahead, enter my userid and supersecret password. Nothing happens. Try a few times since maybe I got the password wrong. Nope. Nothing.  So I tried something different to confirm my theory and get the dreaded, “Your browser does not support cookies” error. Aha, now I’m on to something.

I jump into the settings and try several different things to enable cookies completely. I figure I can return things to the way they want after I get my file. No joy. Despite enabling every applicable options, it wouldn’t override the domain settings and cookies remained disabled.  ARGH.

So, next I figured I’d re-download FF and use that. It’s my box after all (in theory).

I get the install downloaded, click on it and it starts to install. Great! What was supposed to be a 5 minute problem of getting the file I needed to the server is about done. It’s only taken me an hour or two, but I can smell success.

Well, turns out what I was smelling was more frustration. Half-way through the install it locks up. I kill the process and go back to the file I downloaded and try again. BUT, the file isn’t there. I realize after some digging that their security software is automatically deleting certain downloads, such as the Firefox install.

So I’m back to dead in the water.

I know, I’ll try to use Dropbox or OneDrive. But… both require cookies to get setup.  So much for that.

I’ve now spend close to 3 hours trying to get this file to their server.  I was at a loss as to how to solve this. So I did what I often do in situations like this. I jumped in the shower to think.

Now, I finally DID manage to find a way, but I’m actually not going to mention it here. The how isn’t important (though keeping the details private are probably at least a bit important.)

Anyway, here’s the thing. I agree with trying to make servers secure. We in IT have too many data breaches as it is. BUT, there is definitely a problem with making things TOO secure. Actually two problems. The first is the old joke about how a computer encased in cement at the bottom of the ocean is extremely secure. But also unusable.  So, their security measures almost got us to the state of making an extremely secure  but useless computer.

But the other problem is more subtle. If you make things too secure, your users are going to do what they can to bypass your security in order to get their job done. They’re not trying to be malicious, but they may end up making things MORE risky by enabling services that shouldn’t be installed or by installing software you didn’t authorize, thus leaving you in an unknown security state (for the record, I didn’t do either of the above.)

Also, I find it frustrating when steps like the above are taken, but some of the servers in their environment don’t have the latest service packs or security fixes. So, they’re fixing surface issues, but ignoring deeper problems. While I was “nice” in what I did; i.e. I technically didn’t violate any of their security measures in the end, I did work to bypass them. A true hacker most likely isn’t going to be nice. They’re going to go for the gold and go through one of at least a dozen unpatched security holes to gain control of the system in question. So as much as I can live with their security precautions of locking down certain software, I’d also like to see them actually patch the machines.

So, security is important, but let’s not make it so tight people go to extremes to by pass it.