Security: Close isn’t good enough!

I was going to write about something else and just happened to see a tweet from Grant Fritchey that prompted a change in topics.

I’ve written in the past about good and bad password and security polices. And yes, often bad security can be worse than no security, but generally no security is the worst option of all.

Grant’s comment reminded me of two incidents I’ve been involved with over the years that didn’t end well for others.

In the first case, during the first dot-com bubble, I was asked to partake in the due diligence of a company we were looking to acquire. I expected to spend a lot of time on the project, but literally spent about 30 minutes before I sent an email saying it wasn’t worth going further.

Like all dot-com companies, they had a website. That is after all, sort of a requirement to be a dot-com. And it was obvious it was backed by a database server (which I knew was SQL Server, which sped up my process, but only by a few minutes). So, I did the obvious thing and got the IP address of the web-site. Then, I simply tried to connect to SQL Server from my desktop to that IP address and one or two on either side of it. On my second attempt, the IP address right before the one of the website replied to my attempt to reach SQL Server. That was not a good sign. The reply meant there was no effective firewall in place. Note, had they not been using SQL Server, but some other tech, it might have taken me another 10-15 minutes to find the right client to connect. So knowing it was SQL Server wasn’t overly important.

But of course at least they had a password right? Well, back then, the latest and greatest version of SQL Server was 2000 which still did not require a password when you set it up.  I asked myself, “it couldn’t be that easy could it?”

Sure enough it was. Within minutes I had logged in as sa without a password. I now had complete control of their SQL Server. But even more so, back then SQL Server allowed unfettered access to xp_cmdshell. In theory at that point I could have done anything I wanted on the box, including installing remote access software and creating and giving myself administrator access.  I didn’t. But, my email to my boss was short and sweet. I explained how there was absolutely no way we could acquire their platform without a complete top to bottom review of it for any signs of malware. If it took me only 30 minutes or less to get in, I was almost certain their system was owned.

We never acquired that company. I’ve wondered since then what happened to them. My guess is, like many dot-com companies they folded. I can’t say it would have been because of their lack of security, but I can say that the lack of security played a huge factor in us NOT acquiring them. (and for the record, the company I worked for at the time ended up acquiring 1-2 other companies, merging with a 3rd and finally being acquired by a 4th, which is still around. So we were doing something mostly right.)

The second incident that comes to mind was about 8 years later at another start-up. I was asked by the COO to do some due diligence on the setup in another division’s datacenter setup. Again, I didn’t do anything fancy. I knew they weren’t running SQL Server, but I figured I could still do some probing. This time what I found was a bit different. It wasn’t software per se, but rather their iSCSI switch. Sure enough not only did it have a public facing IP address, but, the CTO of that division had failed to change the default password. I was very tempted at the time to give the IP address to my 8 year old son, without any other details and asking him to try to log in. Given his skills, even at that age, I’m 99% sure he’d have figured how to Google the required information and get in. But I figured I didn’t really need to do that to make my point.

That and other factors later lead to the CTO leaving the company.

Moral of the story: Make sure your sensitive information is under some form of lock and key and don’t use blank or factory default passwords, let you or your company end up in a headline like this one: Evisort Data Exposed.

Punditry

We’re all experts on everything. Don’t think so? Go to any middle school or high school soccer game and you’ll be amazed at how many parents are suddenly experts on soccer. It’s also amazing at how many parents are parents of future NCAA Division I scholarship soccer players.

Seriously though, we’re all guilty of this from time to time. I’ve done it and if you’re honest, you’ll admit you’ve done it.

Yesterday the world suffered a loss, the near destruction of Notre Dame.  Early during the fire our President tweeted:

“Perhaps flying water tankers could be used to put it out. Must act quickly!”

As many have pointed out, this was actually a terrible idea. The idea of dropping 100s of kilograms of water onto an already collapsing roof is most likely to do more damage than not. But, while I think it’s easy to mock the President for his tweet, I won’t. In some ways it reminds me of the various suggestions that were made last summer during the Thai Cave Rescue. We all want to help and often will blurt out the first idea that comes to mind.  I think it’s human nature to want to help.

But, here’s the thing: there really are experts in the field (or to use a term I see in my industry that I dislike at times: SME (it just sounds bad) Subject Matter Expert.)

And sometimes, being a SME does allow you to have some knowledge into other domains and you can give some useful insight. But, one thing I’ve found is that no matter how much I know on any subject, there’s probably someone who knows more. I’ve written about plane crashes and believe I have a more than passing familiarity in the area. Perhaps a lot more than the average person. But, there’s still a lot I don’t know and if I were asked to comment by a news organization on a recent plane crash, I’d probably demur to people with far more experience than I have.

Having done construction (from concrete work in basements to putting the cap of a roof on), I again, have more than a passing familiarity with construction techniques and how fire can have an impact. That said, I’ll leave the real building and fire fighting techniques to the experts.

And I will add another note: even experts can disagree at times. Whether it’s attending a SQL Saturday or the PASS Conference itself, or sitting in a room with my fellow cave rescue instructors, it can be quite enlightening to see the different takes people will have on a particular question. Often no one is wrong, but they bring different knowledge to the table or different experiences.

And finally, you know what, sometimes the non-expert CAN see the problem, or a solution in a way that an expert can’t. But that said, at the end of the day, I’ll tend to trust the experts.

And that’s the truth because I’m an expert on punditry.

Shouldn’t that be plugged in?

That was the question a friend of mine in 6th grade asked. As a result I developed what I call the Charlie M. rule after my friend. It was sort of Show and Tell day in 6th grade and we were supposed to talk about our hobbies. I brought in a circle of HO scale track (18″ radius for those interested) and my locomotive (a model GP-38) and some cars and of course the transformer to power it all.

I set it all up in front of the class and dutifully tried to demonstrate it. Nothing moved. I checked to make sure the engine was properly on the tracks: check. I made sure the wires were connected to the transformer: check. I made sure the wires were connected to the track: check.  I was stumped: check. Finally Charlie raised his hand and asked, “Shouldn’t that be plugged in?”  Ayup, in all my nervousness and being hurried, I had forgot the most basic step, of plugging in the transformer.

I try to keep this in mind when troubleshooting: check the obvious. I ran into this again over the weekend when trying to get my BMW Z3 running again. (Side note: no, consulting does not pay that well. This is one of the few tangible items I have left from my dad’s estate). It had stopped running late last fall and at the time I spent a little time trying to make it run, without much success. Finally, with the family’s help I pushed and pulled it into a shelter for the winter and then left it for the winter.

I wasn’t planning on worrying about it until later this month, but then… well let’s just say when I put the large box with metal corners into the rear of the Subaru, I forgot to check the obvious and slammed the rear hatch down on the box. Well, the box, realizing it didn’t have enough room, decided to take advantage of the metal corner and proceeded to make more room by punching out the rear window of the Subaru. Oops.  Such a simple mistake, but a large one.

So, while waiting for the Subaru to get fixed, I decided it was time to get the BMW on the road.

Now due to the symptoms, I knew it wasn’t a dead battery or bad gas. So taking advantage of what I call my extended brain, I asked others for help.  We had narrowed the problem down to either the clutch interlock switch or the starter. Neither looked like it would be an easy self-service and I was getting frustrated. I finally decided that perhaps checking the ODB-II codes might yield more information. Strangely though, the reader didn’t power up; there were no codes to read. That struck me as a strange. So here I did check the obvious: I took the reader to the Subaru and made sure the reader worked. And it worked fine on the Subaru. I went back to my extended brain and mentioned that.

“Oh, have you checked the fuses?”

“Nah I thought about it, but everything seems to have power.”

“You sure, sounds like the onboard computer fuse might be blown.”

So, I trudged out and took off the fuse cover.  Now, I don’t really believe in fate or signs from God, but it was weird, in the list of about 40 fuses, the first one my eyes fell on was Computer. “Nah, can’t be.”

I pulled it, and sure enough, it was burned out. I pulled it and replaced it. Got in the car and thought, “it can’t be that easy, can it?” A turn of the key and the next thing I knew, the 6 cylinders were purring.

All that work and frustration because I had overlooked the basics.

This is far from the first time I’ve overlooked the basics. And I bet you’ve done the same thing. I have a theory about why we do this, and it is in part because the basics ARE so fundamental that we assume it has to be something else. In my model train example, dirty track and loose wires, especially in an ad-hoc setup are arguably a more common issue than forgetting to plug in the transformer. In my BMW case, because literally everything else worked, I assumed the power was getting to the computer. And honestly, even now, thinking about it, I’m surprised the dash light startup didn’t change at all because of a lack of computer.

I’ve seen this in databases and elsewhere. I was recently trying to do a quick restore of a database from one machine to another and the obvious wasn’t working. It took me a bit to remember the client’s new security setup prevented this specific case for these two machines. Once I remembered that, the problem and subsequent solution were obvious.

This in part goes back to why I like using a rubber-duck at times. It can force you to review your assumptions and check the basics.

Having a problem? Employ the Charlie M. rule and check the basics.

 

JOBS THAT BEAT THE CARING OUT OF YOU

Let me start by saying this is NOT an April Fool’s Joke. This is a true story.

I do lay the ‘blame’ for this post squarely two members of my #SQLFamily: first on the heels of Grant Fritchey and his post by he same name. He in turn lays blame on Jen McCown’s post by the same name.

I mention elsewhere in my blog I prefer to be intelligiently lazy, so rather than retype, I’ll post the content from a Quora answer I wrote.  Technically I was just a consultant, and after twice getting a late check I made it clear to them that if they stopped paying me on time, I would stop working.  Apparently they liked me enough that a quick call to the CFO would get me a check cut that day.

So with that:

Let me give you an example of a client I once had. When I started with them, people loved working there and they were expanding and successful. So successful the company got bought.

Then… things changed.

Sales people were finding their expense checks weren’t getting paid (more on that later). Did you know, even if you try to explain to the credit card company that it’s a “company card” if it’s in your name and the company doesn’t pay it, you’ll ruin your credit score? Yes, it’s pretty difficult to be a sales person who can’t travel because no one will give you a credit card any more!

Then, to cut costs, an office move was proposed. Quite frankly, had I not been involved as their IT guy, it would have been a disaster for a variety of reasons. Fortunately for them, besides my IT skills, I could read blueprints. It was quite obvious to me that 2 outlets would not serve an office of 20–25 people with computers and printers. It took me nearly kidnapping the CFO on a day he visited and dragging him to the office to make clear how much more work the office needed. They simply assumed, “oh, it’ll have enough power.”

Meanwhile the previous owner had started a new company (in a completely different industry) and was growing and expanding at a furious rate. Also, my wife was a recruiter at another local company (in a different industry also). The only thing all three of these companies had in common was they all were software related, but the fields they served were completely different.

At one point, the top sales person from the failing company left to go get a job a with the new company. Within days the former company sent a cease and desist letter to the new company insisting they stop poaching employees and if they continued, they’d sue the owner for violating the non-compete clause. Now, keep in mind the owner was very much NOT approaching employees of the old company, but even if he were, the non-compete only applied if he had founded a new company in the same industry. he hadn’t. We had a good laugh at the old company.

Now, meanwhile, my wife, while not exactly poaching, knew that almost any offer she made would be accepted since morale was so bad at the old company.

Then… this happened. I was there for the meeting and sat in on it. It’s the closest I’ve come to “beatings will continue until morale improves” ever.

The CFO and CEO came into town for an all-hands meeting. Their goal was to address, among other things, the late employee expense checks issue.

I will say, they had some pretty looking slides. The slides showed things like cash-flow, moving towards profitability and some other items. But the message was quite clear, “We will continue to pay YOUR expense checks as late as possible because it helps our cash flow. And you should be grateful for this.” They very much could NOT understand why employees were furious that their expenses were basically being used as no-interest loans by the company. The rate of exits accelerated after that.

What had been a thriving company became a dying, decaying shell of a company in under a year because of the management.

One Postscript:

One of the developers who left the old company ended up at the new company. He submitted his expense check. He was reasonable, he knew it would probably hit his next pay cycle. He was OK with that. I still recall the look on his face when later that day someone from finance walked in with his expense check. They were under no obligation to turn it around that fast and he certainly wasn’t expecting it. But they did so. They “bought” his loyalty that day by a simple gesture.

So, if people are leaving, trying to force them to stay will backfire. Figure out what you’re doing wrong and fix it.

Nothing to Prove

This past weekend was a busy weekend for me. Thursday night I helped do the offensive door for my wife’s hockey team in the first game of a 3 game tournament. Despite only having 9 players (plus a goalie) they tied it 2-2 against a far larger squad. Quite impressive really seeing them up their level of play. They in fact got accused later of stacking the team with college students. Truth is, about half the team was in their 40s or above and the others late 20s or 30s. Not bad.

Friday night the family went to see Captain Marvel. I’ve mentioned before my daughter’s love of Star Wars. She’s also a fan of superheros, both in the comics and movies. We’ve seen a number of them together, including Wonder Woman and now Captain Marvel.

As a father I’m glad for movies like The Last Jedi, Wonder Woman and Captain Marvel. The last two might be called superhero movies, but they’re really about being a hero in general. You don’t need superpowers to be a hero. You need to be a person who decides to do the right thing at the right time. When Wonder Woman decides to cross the No-Man’s Land, she does rely on her superpowers to keep her safe, but she acts because she knows she has the power to make things better. Similarly, Carol Danvers is faced with a choice of doing what is easy and she has been taught to do and what is right. She’s a hero, again not because of her powers, but because she chooses to do the right thing.

Wonder Woman and Captain Marvel are great movies for a number of reasons; but I think foremost because they truly focus on their stars. Steve Trevor and Nick Fury are supporting partners, not just in theory, but in the way they often take their lead from the main characters. Both are strong and powerful people in their own right, but recognize they’re among their betters. And, aiding their partners doesn’t hurt their masculinity, and they realize that.

I enjoy these movies because, while, as far as I know, my daughter doesn’t have a magical lasso, nor can she shoot photon bursts out of her hands, she has role models. While yes, her favorite comic book superhero might be Batman, she really enjoys having these superheros as models.

On Saturday, she and I drove 2.5 hours to Binghamton NY for her team’s competition in something called “Odyssey of the Mind“. While her team didn’t do as well has hoped, it was still a great time to bond and talk, including about Captain Marvel and role models.

My wife had her second game of the tournament on Saturday and when she called to give me the score, I could pick up on her excitement even before she said they had lost, 1-0. But this time with only 8 players instead of 9, it was in some ways even more impressive than the Thursday game.

Sunday rolled around and since we had to take our son to the train station to go back to college, it was easier to bring him to the game first. So, he got to watch his mom’s team, again with only 8 players actually win, 3-1 and come in 3rd out of 6 teams!

It was a fitting cap to a weekend of watching powerful women.

Oh, and besides Wonder Woman, Captain Marvel, and my wife’s hockey team be great role models for women, I think they’re great role models for men.  Men, you can have powerful women in your life and it doesn’t weaken you or make attack your masculinity.  If anything, it can help you be a better man.  So don’t just take your daughters to see Captain Marvel, take your sons.

 

Design Thoughts

Ever look at a product and wonder, “why did they design it that way?” I know I have, and I have some examples I want to bring up.

Years ago, over dinner, I had a programmer from our Wisconsin office basically ask, “why the hell is your file system for your web servers setup the way it is?”  It was a fair question. It wasn’t something one would normally see.  But before I explain that…

Like any modern American, I’m physically incapable of being more than 10′ from a flat screen TV in my house.  We have several, including one in my office and one in the kitchen. I couldn’t tell you the brand of the one in the kitchen (well I could, but I’m too lazy to go downstairs and find out) and the only reason I can tell you the brand of the one in my office is because I can see it from here. It’s an Inginsia brand.

Both serve the same function: they allow me to watch TV. But both have design quirks.

Their button layouts are a bit different (note the layout of the numbers and the volume/channel control buttons.)

Kitchen TV Remote

Kitchen TV Remote

Office TV Remote

Office TV Remote

The kitchen TV also has a built-in DVD player, so it has additional controls for that.

So obviously, there’s different design philosophies and requirements here. But I want to go a step deeper and talk a bit about functionality.

The kitchen TV remote, if you mistype a number, you can hit the Vol – button and it will essentially backspace and delete the number. Actually a handy feature.  The Office remote has no such functionality, though hitting EXIT will remove the entire channel already entered.  Score one for Dynex. (Ok, I did go downstairs so I could grab the remote and take a photo).

But, the Dynex has one annoying quirk I’ve never figured out. When I hit the OFF button, there’s a noticeable delay of 1-2 seconds before it actually turns off. For the life of me, I have NO idea why. I mean I’m turning off a TV. It’s not like I’m shutting down a computer where it has to write the contents of memory to disk and perform other tasks. Sure, maybe it has to save the last channel I was tuned to, but it could do that right after I tuned into that station. Same with the volume.  Every other TV in the house, including my office one, when you hit the off button, turns off instantly.

I’m reminded a bit of early computers that had the big red switch. There was something satisfying about turning off an early PC. You knew it was instantly off. There was no two questions about it. Now, shutting off a PC is a far more complex operation and can take sometime.  But a TV? I’d love to know why the kitchen TV takes a long time to turn off.

Now back to the file design the programmer was asking me about. Essentially we had 5-6 web front ends, each with a virtual directory in IIS pointing to a NAS. Not an entirely awful setup, but uncommon at the time.  We were offering a web platform to newspapers so they could publish their content. Originally we tried using a 3rd party package to make sure the content on all the servers was always in synch (since a newspaper could upload content at any time to any of the servers and wanted it available instantly). What we found was sometimes we’d get into race conditions where files could actually end up erasing themselves. The 3rd party company kept assuring us they had the solution. Well after a desperate call at 4:00 AM call from my on-duty NOC person, I drove into the office, scrambling to figure out a better solution. On the drive, the idea of using the virtual directories to the NAS occurred to me. We implemented it in about 30 minutes and solved our problems. It was supposed to be a temporary solution until we came up with a more robust, permanent solution. But, 18 months later it was still in place, working great and I was explaining it to our out of town programmer. He went from, “that’s nuts” to “Hey, that makes a lot of sense.”

So, I like to think that when there’s a design I don’t understand, the designers at the time had their reasons. But, to be honest, I’m not always sure.

For example, the photo that should be heading up this article, of a shampoo bottle and a bottle of conditioner, both from the same manufacturer, both designed to be cap down, are printed the opposite way. The only reason I can think of that makes sense is so that in a befuddled, sleep deprived state, I can more easily determine which is which. But even if that is the answer, why this way, and not the other? Inquiring minds want to know!

And yes, the shampoo bottle can be placed cap up, but the conditioner bottle can’t be. Again, why? The viscosity of the two aren’t that different. Again, inquiring minds want to know.

Shampoo/Conditioner bottles

One of these is upside down!

 

Don’t…

call yourself an ally.

Just don’t.  You may think you’re an ally. You might actually be one, at times. But, don’t call yourself an ally.

Note, I didn’t say you can’t be one, nor did I say you can’t strive to be one. I’m simply saying don’t call yourself one.

I don’t care if you volunteer for the local LGBTA+ outreach group, if you serve on a women in tech diversity panel, or have all the “right” stickers on your car or laptop. You can do all the right things and be an ally, but don’t call yourself one.

Now, if members of the groups you’re helping want to call you an ally, that’s great. You’re doing good work. You’re doing something right. And it’s ok to enjoy the praise and thanks, a bit. But, still, don’t call yourself an ally.

Here’s the thing, I’ve got about every privilege box I can check in the US. I’m white, cis-het male, with a decent income. I have friends and family members that don’t quite hit all those boxes. I like to think I use my privilege to help others.

And it’s true, I’ve found in online debates I can say almost the exact same thing one of the people identifying as female says and somehow my words get taken with more gravity.

And yes, in the IT field, I’ve seen multiple times my coworkers talk over or ignore someone who wasn’t cis-het male, despite the other person’s knowledge and wealth of experience.

I’ve used my privilege to try to bring equality into the IT spaces I’m in. Sometimes I’ve succeeded, sometimes I’ve failed.

I’ve called others out for homophobic jokes, cat-calling and more.

But, as I’ve grown older, and I like to think wiser, I’ve realized even more, how I can’t call myself an ally.

It’s not for lack of effort. Let’s be clear, NO amount of effort will allow me to call myself an ally. And here’s why:

I’m not a member of the groups I’m trying to help. I’ve never truly experienced the discrimination and bigotry they’ve faced. Even when I’ve been associated with them, I’ve come to realize I’m “other”. This isn’t a fault or a failing, simply my reality. I can be among a group of gay men celebrating a friend’s birthday at the Green Lantern in Washington DC, and be perfectly comfortable, but know I’m “other”. And they know it too. The next day at work, even if I encountered a homophobic coworker, I can still disassociate myself from that weekend’s activities. It’s not a core part of WHO I am, it’s simply a part of something I did.

If I support women in tech, and I’d like to think I do, no matter how well I listen, I won’t truly have the gestalt experience of walking to my car at night wondering, if something happens, will someone’s first question be, “Well, what where you wearing?”  I’ll never be in a meeting and have an idea shot down and wonder, “was my idea dismissed because it was bad, or because of my gender presentation?”

I will always be “other”.

In a related manner, I can, intentionally, or unintentionally stop being an ally in an instant. I can intentionally choose to sit down and not be an ally.  Or, I can make a misstep and fail as an ally. And I don’t get to decide if I’ve succeeded or failed.

I will give an example of this: via Twitter I saw one woman speaker comment on how she felt offended that she had been asked, in part because of her gender, to speak at a conference. The person approaching her had made it clear he wanted more women to speak. Generally, this could be seen as being a good ally; making sure conferences aren’t full of manels and/or only have a slate of male speakers. Other women stepped in and said they wouldn’t be offended at all, that they appreciated the effort to include more women, even if at times it came across as ham-fisted or overly obvious.

So here’s the thing, the person asking probably thought of himself as an ally and might have called himself one. But, clearly the first woman didn’t agree, but others did. This is why he can’t call himself an ally, but others can. Perspective makes a huge difference here.

So in conclusion, let me end with what I opened with: not only can you work to be an ally, I would in fact encourage you to work to help others obtain the privileges and opportunities you have. BE an ally. But let OTHERS determine if you’re their ally. If they call you an ally, great! Keep it up.

If they however tell you that your actions or words are not helpful, listen to them. They are in the best position to determine what helps them. Unless they ask for input, don’t tell them what they’re doing wrong or why they’re wrong to not accept their help.