It’s too late?

I want to start with a sobering thought. It’s too late to contain this pandemic. I’m watching the news as slowly more and more states in the US issue versions of “shelter in place” or “stay at home” orders. But I think in most cases, it’s too late. The virus has probably already spread so much that self-isolation won’t be nearly as effective as it would have been had the states issued the same orders a week or two earlier. That said, it’s most likely still better than doing nothing.

Human beings at times are lousy with risk analysis. If a risk is immediate, we can react well, but the longer it stretches out or the further away it is, the harder it is to get people to react. Almost any climate scientist who has studied anthropogenic global warming has known for a decade or more we have a problem and we have a very quickly narrowing window for solving it, and the longer we wait, the harder it will become.

Yet too many of us put off the problem for another day.

So it is with the Covid-19 virus. “Oh we don’t have to lock down just yet, let’s wait another day.” And I’ll admit, sitting in the state that is the center of the virus outbreak here in the US, I’m tempted to say, “25,000 isn’t TOO bad, we can manage that.”  But that’s the lizard part of my brain reacting. It’s the emotional part. Then I kick in the rational part. If we use one of the numbers bandied about, doubling every 4 days, that means by this weekend, in New York State alone, it will be 50,000. By April 1st, 100,000. By the end of April, it could be the entire state.  Those numbers are hard to comprehend.

That said, I’m also hopeful. Modelling pandemics is pretty much pure math, but reality is more complex and often luck can play a huge factor. Let me try to explain.

First, we need to heed the words of experts like Dr. Fauci and others who are basing their remarks and recommendations on the inexorable exponential rise in expected infections. They are giving basically the worst case scenario if their recommendations are followed. And that’s proper. That’s really what you have to plan for.

Let me take a little side trip and mention a cave rescue in Vermont several years ago. By the time I had gotten the call to show up and to call out other rescues, the injured party had been in the cave for several hours. I didn’t know much about the extent of their injuries other than it was a fall and that it was in a Vermont cave, which almost certainly meant operating in tight quarters. I grabbed a box of Freihofer cookies, a lawn chair (my fellow cave rescuers will understand the reference), a contact list of other potential rescuers, and my son. While I drove, he’d read off a name and I’d say “yes call” or “Nope, next name.”  On the hour plus drive to the rescue we managed to contact at least two other people who could get there. (It turns out, as I surmised, several of the folks I wanted to call were members of the original caving party.)

Once there, my son and I were driven partway to the cave entrance and trudged the rest of the way. I talked with the folks on the scene to gather information and then dressed to go into the cave to gather first hand information. I still hadn’t gained too much information other than to know it was potentially shaping up to be a serious rescue. The person had been climbing a cable ladder when they fell and injured themselves. This meant, based on the information at hand, a worst case scenario of an evac through tight passages with the patient in a SKED stretcher.  I was playing the role of Dr. Fauci at that point, preparing for the worse based on the information I had.

Fortunately, literally at the moment I was about to enter the cave, one of the members of the original caving party crawled out and said, “he’s right behind me, he’ll be out in a minute or so.”  It turns out his injuries were fairly minor and with the members of his own caving party, he was able to get out of the cave under his own power.

I got back to Incident Command about an hour later and was informed, “oh, by the way, you’ve got at least 3 cavers who showed up to help. We held them at the bottom of the road. What should we tell them?”  My answer was simply, “Thanks and to go home.”

I relate this story not so much to talk about cave rescue specifically but to point out that even when planning for the worst, you may get a lucky break. But you can’t rely on them. Let me give an alternate scenario. Let’s say I had not called out the other rescuers and had gotten to the cave and crawled in, realized the situation was a worst case scenario, crawled back out and then initiated a call-out. It would have at that point probably meant at least an extra 90 minutes before the extra resources would have been on the scene. It would have meant the patient was exposed to hypothermic condition for another 90 minutes. It would have meant 90 more minutes of pain. It would have meant fewer brains working to solve the problem.

Getting back to Covid-19. Will we get lucky? I don’t know. I actually suspect we might. One “advantage” of an increasing population of sick people is we can better model it and we can also perform more drug trials. We may discover certain populations react differently to the disease than others and be able to incorporate that into the treatment plan. I don’t know. But I do know, we need to plan for the worst, and hope for a bit of luck. In the meantime, hunker down and let’s flatten the curve.

And if you’ve read this far and want to know how to make some pita bread, I’ll let you in on the two secrets I’ve learned: VERY hot oven (I typically bake mine at about 500-550F for 2 minutes on 1 side, and 1 minute on the other) and roll it out thinner than you might think.

Them’s the brakes…

I need brake work done on my car.  I scheduled an appointment with my mechanic for Thursday.

Now normally brake work might not be worth writing about, but this time it got me thinking. About 9 years ago I taught myself to change the brakes on my Subaru. It wasn’t that hard, but it definitely took longer than I would have liked. I’ve learned how to do it much faster since then.

Back then I did it because I was between jobs and had more time than money. It made sense at the time. Since then I’ve generally continued to do them myself. It doesn’t take long and as I’ve said in the past, sometimes I like working with my hands. It reminds me not all of life is simply databases and servers.

So why this time? The simple answer is, my garage floor is cold!  Ayup, it’s a funny thing, but in the winter, things get cold! And honestly, I just don’t feel like sitting/laying on the concrete while I do the work. So, I’ll pay an expert who can pop it up on a lift and do the work in probably half the time I can.  I remind myself it’s why I work, so I can pay others to do work I don’t care to do, or that they can do better than I can.

Years back, at my last job as the Director, and later VP of IT, I changed our email provider from one company to another (don’t ask me who they were, I can’t recall over a decade later).  My team didn’t like this. They kept insisting they could run the mail servers themselves. I had no doubt that they could. Running an Exchange server isn’t the most difficult thing in the world. That said, I kept saying no.  For a simple reason: “we weren’t an email company!” Providing reliable email for a business like we were meant at the very least strong spam protection and of course high-availability with ideally geographic redundancy. This meant at least two Exchange servers, spam filters and more. This alone would have cost more than what we paid annually. Now add in the time my team would have spent on email issues, it just wasn’t going to be worth it.

And, we’re seeing that more and more with services such as Azure. Sure, many businesses run change their own brakes, err, run their own SQL Server. But more and more are outsourcing to platforms such as Azure. But there are still a number of companies that for various reasons don’t do that. Fortunately for them, there are consultants like me to help them with their servers. SQL Server has always been easier to maintain than some of its competition, especially when it was first available. But that has never meant it needed no intervention. Someone still needs to make sure that jobs are being run, that backups are available and more.

So, some days I change my own brakes because it’s fun and easy, some days I pay someone.  Some days my clients handle their own SQL Server issues, and other days, they pay me.

No real revelation here. No real advice on when to draw the line on outsourcing, just an observation. But in my case, my concrete is too cold, so I’ll pay someone else to do the dirty work.

“Do What You Love…

And you’ll never have to work a day in your life.” – Confucius, Mark Antony, Mark Twain.

Honestly, I think that’s some of the worst advice ever. It’s a sure way to end up hating something you love doing. Or if you do follow it, make sure you understand what it is that you love.

I first realized this in high school. I had signed up to do JV soccer, something I enjoyed, but I can’t say I loved. Before school started, we had a day of orientation. It included a hike or run up the mountain behind the school. I loved the woods and I loved (and still do at times) running through the woods. Somewhere along the way, a fellow student saw me running and suggested if I enjoyed running through nature so much, that I consider doing Cross-Country instead as my fall sport. I took their advice; snd hated it for two fall semesters in a row.

I realized what had happened was that I had replaced what to me was a fun, non-competitive activity and turned it into something where I had to perform at a specific level every single time. What I loved was running through the woods, not running competitively.

People assume I love working with computers.  That’s not entirely true. I ENJOY working with computers. I enjoy solving problems and computers are one way I can express that joy. When I make a query run 10x faster, or automate a process that previously took someone an hour a day to do, I enjoy that. But do I love it? Probably not. And I’m actually grateful for that. Because if I loved it, it would mean those days of drudgery where I bang my head against the wall all day trying to solve a problem, or I’m up until 3:00 AM recovering a failed server would turn something I love into something I dread. Something I loved would become a chore.

I love to teach caving. I get a real thrill out of it. But, I suspect that if I spend 40 hours a week, 50 weeks a year doing it, it would soon become a chore.

As my kids started their college journey, I’ve advised them, “find something you enjoy, not something you love. Keep the something you love for your own personal time so it doesn’t become a chore.” And that’s my advice to anyone.

But hey, I could be wrong. What are your thoughts? Did you pick your job because you love it and if so, do you still love it, or did you end up resenting it at all? Or do you enjoy your job?

Being Observant

I was busy last week teaching cave rescue at the annual National Cave Rescue Commission Weeklong training seminar, hence my lack of a post last week.

As always, I had a great time helping to teach a class of 19 students. Our teaching includes classroom time as well as time in the field. We use both caves (obviously) and cliffs for out in the field instruction. The cliffs give us an opportunity to focus on rigging and lets everyone see what’s going on; mostly. In a cave, often it’s too dark and small to let everyone have a good few.  But, you may notice that I said mostly above.

Let me interrupt with a quick video.  Can you count the number of times the players dressed in white pass the basketball? If you got 15, congrats.

Now back to the class being on the cliffs. As part of our exercises, we usually do several iterations of the same thing, such as lowering a patient and then raising them up. Each time we may change a detail or two, such as where they can rig, what haul system to use, etc. The idea is that repetition helps them learn the skill. Often though, at the last iteration we toss in a wrinkle: they’re not allowed to speak, at all. Generally this means, they have to start with a pile of rope and hardware on the ground and without saying a word, rig a safe system to lower and raise a patient.  Generally they use a variety of hand signals and various body motions. While at times it can be quite entertaining, it can also be instructional. It’s often the faster iteration of the day, in part because they’ve honed their skills but also due in a large part because there’s no ideal chit-chat.

The point of the exercise is multi-fold. Partly it’s a great challenge for them that they end up really enjoying. It also illustrates that communication is still possible when you can’t verbalize things. This is actually often common in caves where rushing water or echos can make vocalizations useless. It finally shows that they’ve come together as a team and can work effectively.

That said, sometimes they can miss details. As instructors we ensure though that no safety issues are missed. We will stop the exercise if we see something unsafe and help them make it safe.  Part of this includes an instructor on a separate rope to hang over the edge so they can watch the patient (generally another instructor) go down and come back up. This instructor watches the entire process for safety reasons.

This time around, we decided to add a bit of a twist to the exercise.  Generally the safety instructor stays at the top. This time however, he rappelled over the edge, all the way to the bottom. Nothing overly unusual about that.

However, once he was at the bottom, along with the patient, who happened to be me, I very quickly detached the ropes connected to my harness and connected them to his harness. I then moved over to the line he had been on and started to ascend as the students began to haul him up.

They eventually got him to the top and congratulated themselves on a job well done. Other than the student at the edge of the cliff who had watched the process, none of them appeared to notice that they had lowered me and raised him.

This actually isn’t totally surprising. We like to think of the human brain as a computer, but in many ways it’s not. Or at least it’s not a digital one with perfect recall. We actually ignore a lot of “noise” in order to focus on what’s important at the time. The fact that the person they hauled up was different from the person they had lowered counts as noise in this case because it’s so improbable that the brain won’t bother processing it, lest it take away from more important tasks, like ensure the ropes were handled safely.

And for those of who you didn’t watch the video, the answer is 15, but that’s not what’s important. Go watch it.

Today’s takeaway, our brains are fuzzy and work “well enough” most of the time, but can miss details. And generally, that’s ok.

Security: Close isn’t good enough!

I was going to write about something else and just happened to see a tweet from Grant Fritchey that prompted a change in topics.

I’ve written in the past about good and bad password and security polices. And yes, often bad security can be worse than no security, but generally no security is the worst option of all.

Grant’s comment reminded me of two incidents I’ve been involved with over the years that didn’t end well for others.

In the first case, during the first dot-com bubble, I was asked to partake in the due diligence of a company we were looking to acquire. I expected to spend a lot of time on the project, but literally spent about 30 minutes before I sent an email saying it wasn’t worth going further.

Like all dot-com companies, they had a website. That is after all, sort of a requirement to be a dot-com. And it was obvious it was backed by a database server (which I knew was SQL Server, which sped up my process, but only by a few minutes). So, I did the obvious thing and got the IP address of the web-site. Then, I simply tried to connect to SQL Server from my desktop to that IP address and one or two on either side of it. On my second attempt, the IP address right before the one of the website replied to my attempt to reach SQL Server. That was not a good sign. The reply meant there was no effective firewall in place. Note, had they not been using SQL Server, but some other tech, it might have taken me another 10-15 minutes to find the right client to connect. So knowing it was SQL Server wasn’t overly important.

But of course at least they had a password right? Well, back then, the latest and greatest version of SQL Server was 2000 which still did not require a password when you set it up.  I asked myself, “it couldn’t be that easy could it?”

Sure enough it was. Within minutes I had logged in as sa without a password. I now had complete control of their SQL Server. But even more so, back then SQL Server allowed unfettered access to xp_cmdshell. In theory at that point I could have done anything I wanted on the box, including installing remote access software and creating and giving myself administrator access.  I didn’t. But, my email to my boss was short and sweet. I explained how there was absolutely no way we could acquire their platform without a complete top to bottom review of it for any signs of malware. If it took me only 30 minutes or less to get in, I was almost certain their system was owned.

We never acquired that company. I’ve wondered since then what happened to them. My guess is, like many dot-com companies they folded. I can’t say it would have been because of their lack of security, but I can say that the lack of security played a huge factor in us NOT acquiring them. (and for the record, the company I worked for at the time ended up acquiring 1-2 other companies, merging with a 3rd and finally being acquired by a 4th, which is still around. So we were doing something mostly right.)

The second incident that comes to mind was about 8 years later at another start-up. I was asked by the COO to do some due diligence on the setup in another division’s datacenter setup. Again, I didn’t do anything fancy. I knew they weren’t running SQL Server, but I figured I could still do some probing. This time what I found was a bit different. It wasn’t software per se, but rather their iSCSI switch. Sure enough not only did it have a public facing IP address, but, the CTO of that division had failed to change the default password. I was very tempted at the time to give the IP address to my 8 year old son, without any other details and asking him to try to log in. Given his skills, even at that age, I’m 99% sure he’d have figured how to Google the required information and get in. But I figured I didn’t really need to do that to make my point.

That and other factors later lead to the CTO leaving the company.

Moral of the story: Make sure your sensitive information is under some form of lock and key and don’t use blank or factory default passwords, let you or your company end up in a headline like this one: Evisort Data Exposed.

JOBS THAT BEAT THE CARING OUT OF YOU

Let me start by saying this is NOT an April Fool’s Joke. This is a true story.

I do lay the ‘blame’ for this post squarely two members of my #SQLFamily: first on the heels of Grant Fritchey and his post by he same name. He in turn lays blame on Jen McCown’s post by the same name.

I mention elsewhere in my blog I prefer to be intelligiently lazy, so rather than retype, I’ll post the content from a Quora answer I wrote.  Technically I was just a consultant, and after twice getting a late check I made it clear to them that if they stopped paying me on time, I would stop working.  Apparently they liked me enough that a quick call to the CFO would get me a check cut that day.

So with that:

Let me give you an example of a client I once had. When I started with them, people loved working there and they were expanding and successful. So successful the company got bought.

Then… things changed.

Sales people were finding their expense checks weren’t getting paid (more on that later). Did you know, even if you try to explain to the credit card company that it’s a “company card” if it’s in your name and the company doesn’t pay it, you’ll ruin your credit score? Yes, it’s pretty difficult to be a sales person who can’t travel because no one will give you a credit card any more!

Then, to cut costs, an office move was proposed. Quite frankly, had I not been involved as their IT guy, it would have been a disaster for a variety of reasons. Fortunately for them, besides my IT skills, I could read blueprints. It was quite obvious to me that 2 outlets would not serve an office of 20–25 people with computers and printers. It took me nearly kidnapping the CFO on a day he visited and dragging him to the office to make clear how much more work the office needed. They simply assumed, “oh, it’ll have enough power.”

Meanwhile the previous owner had started a new company (in a completely different industry) and was growing and expanding at a furious rate. Also, my wife was a recruiter at another local company (in a different industry also). The only thing all three of these companies had in common was they all were software related, but the fields they served were completely different.

At one point, the top sales person from the failing company left to go get a job a with the new company. Within days the former company sent a cease and desist letter to the new company insisting they stop poaching employees and if they continued, they’d sue the owner for violating the non-compete clause. Now, keep in mind the owner was very much NOT approaching employees of the old company, but even if he were, the non-compete only applied if he had founded a new company in the same industry. he hadn’t. We had a good laugh at the old company.

Now, meanwhile, my wife, while not exactly poaching, knew that almost any offer she made would be accepted since morale was so bad at the old company.

Then… this happened. I was there for the meeting and sat in on it. It’s the closest I’ve come to “beatings will continue until morale improves” ever.

The CFO and CEO came into town for an all-hands meeting. Their goal was to address, among other things, the late employee expense checks issue.

I will say, they had some pretty looking slides. The slides showed things like cash-flow, moving towards profitability and some other items. But the message was quite clear, “We will continue to pay YOUR expense checks as late as possible because it helps our cash flow. And you should be grateful for this.” They very much could NOT understand why employees were furious that their expenses were basically being used as no-interest loans by the company. The rate of exits accelerated after that.

What had been a thriving company became a dying, decaying shell of a company in under a year because of the management.

One Postscript:

One of the developers who left the old company ended up at the new company. He submitted his expense check. He was reasonable, he knew it would probably hit his next pay cycle. He was OK with that. I still recall the look on his face when later that day someone from finance walked in with his expense check. They were under no obligation to turn it around that fast and he certainly wasn’t expecting it. But they did so. They “bought” his loyalty that day by a simple gesture.

So, if people are leaving, trying to force them to stay will backfire. Figure out what you’re doing wrong and fix it.

Followers and CPR/First Aid

Yesterday, I performed a little social experiment and was pleased to find it worked. I’ve got to say, sometimes it’s the small things that make me happy.

Despite the below zero (Fahrenheit, so really cold, not that warm-cold of 0°C) temperatures, my son and I decided to head up to a local state park and do a hike.  Surprisingly, OK, maybe not, when we arrived, the parking lot was completely empty.  It had been plowed, but there was still a layer of snow over the entire thing, so it was impossible to see where the parking lines were. Now in the summer, this parking lot can be completely full, but I wasn’t too worried about that occurring when the temp was about -4°F.

So, which way to park? Well, there was some sun, so I figured I’d park so that the windshield would get the most sun and hopefully warm up the car just a bit while we hiked. I was sure at the time and later confirmed, this was at a 90° angle to the way the parking lines run. Ironically it was also about 90° colder than the summer temps!

Even when we started hiking, no one else had shown up. But, I have to admit, in the back of my mind I had to wonder if I would start a trend.

Sure enough, 1.5 hours later, when we arrived back at the car there were 3 other cars.  Not only were they parked in the same orientation, they were all parked right next to my car.  This parking lot probably covers 3 acres. They could have parked pretty much any place they wanted in any direction they wanted. But, because I had randomly picked a spot (and not so randomly a direction) 1 car was parked next to me in the same orientation and the other 2 parked facing us.

So what does this little experiment have to do with First Aid or CPR? Have you ever been at an event when someone has a medical event and at first no one reacts? It’s actually fairly common.  Everyone is standing around waiting for someone else to react. But once someone reacts, others tend to follow.  Be that person that others follow.  Learn CPR and learn First Aid so that when something happens, you can be the first to react. Sometimes people just need a leader to follow; and often they don’t necessarily realize it.

There’s no good reason anyone else parked just like I did, and yet they did. But there is a good reason for people to follow you if you can be the first to react in an emergency.  And you don’t have to be an expert. Obviously it didn’t take “expertise” to park yesterday, but people followed anyway. You don’t have to be an EMT or paramedic to react at a medical emergency. You can be the person that simply shouts, “Call 911” and gets people reacting.

That said, I still highly recommend taking a CPR and First Aid course. Not only do you learn very useful medical response skills, it will help you be that person that reacts first.

And stay warm!