Category Archives: Management

Feeling Good but…

Posted on by
Reply

I think it would be fair to say that like everyone, I’m a bit sick of Covid (thankfully not sick from it.) I just got my booster on Friday and then I’m hearing about the Omicron variant.

I submitted talks for SQLBits in the UK for next year, hoping to present in person. And I’m hearing about numbers rising.

I’m planning a mini-vacation/cave rescue training trip to Hawai’i next year and making sure everything is refundable. Just in case.

So I’m feeling god but…

At the start of each year, I set some financial goals for myself. Some include what things I may pay off, save, or how I’ll spend it (now admittedly most of those are fixed, such as knowing I’ll tax property taxes, etc.) As a contractor I also set a couple of various goals for new work and how much I’ll hopefully earn in the coming year. I find these are important as they help keep me focused and moving forward.

The good news is, financially I’ve hit all my goals, and then some, this year. The downside, with that, and with Covid continually popping up its ugly head, I’ve lost some of my motivation for the rest of the year.

Fortunately, this has freed up some time for some projects around the house. Almost two years ago, with help from the kids, I started on a project to replace some leaking pipes and replace the resulting damaged drywall in the basement. I’m proud to say I’ve finally gotten around to taping and painting the drywall in the basement and patching around where I put in the new bathroom fan. Things get done, albeit slowly.

I’m also feeling good because a major project for one of my clients is mostly completed. But it also came very close to burning me out and I’ll admit I even considered walking away from the client over it. The strange part is that it wasn’t a particularly complicated project, though it did involve a combination of SQL, PowerShell, and using a product called Pentaho. Technically it was fairly straightforward. But, for awhile, the project management was absent and the then lead was actually another agency who, I think it’s safe to say didn’t clearly understand the full scope of the project. With the addition of the client adding their own PM and working with a different agency taking over a bunch of the work, things have gone much more smoothly. Now we’re simply dealing with small niggling details that got missed before.

What kept me from walking away (besides it being my largest client) was a sense of responsibility to the client. Without my efforts, I think the project would have easily been set back a month as they would have had to bring someone else up to speed on my efforts.

Now the upside is that because of the overtime required (and it’s still ongoing) I met my financial goals for the year (and hence now have time for the house projects). So that’s a good thing.

But it did highlight how frustrating being a single-person consulting agency can be at times. It’s made me re-evaluate my goals for 2022. I’ll be writing more about this in a future blog, but it has got me thinking more about getting back to working for an company as a full-time employee, ideally in a management position. Strangely one thing I’ve come to realize is I actually enjoy making decisions and I enjoy managing. I sort of miss it.

And perhaps after nearly 2 years of Covid (and nearly a decade of pure consulting), it’s time I get out of the house more and travel a bit and interact face to face with people.

We’ll see.

But that’s it for today. I’m feeling good but…

P.S. One thing I did finally accomplish is submitting my latest article to Redgate’s Simple-Talk.

The World Wonders

Posted on by
2

I mentioned recently that I had picked up a copy of the book The Last Stand of the Tin Can Sailors. I just finished it and would highly recommend it. The author, James Hornfischer does an excellent job of interweaving the fates of the ships and their crews over the course of several chapters. There’s an excellent sense of the fear and sense of duty among the sailors. He also includes several maps to help one orient themselves as they read about the battle unfolding. He appears to have done his research, which includes numerous interviews with the survivors, reading of the ships logs and more. The one area of missing information, and he admits it, is an adequate understanding of the Japanese side of the battle. This appears in part to be due to a lack of access to such logs and I suspect a language barrier and the difficulty of travelling to Japan.

I mention this because it’s important to understand that the story he writes, nearly 60 years after the battle gives a far fuller picture of what happened than any of the participants had that day. But even now that story is missing pieces.

To quickly recap, the Imperial Japanese Navy was given the mission of breaking up MacArthur’s landings on Leyte in order to reclaim the Philippines. Like many Japanese naval plans it was audacious but also required meticulous planning and timing. And it involved a decoy fleet. This is an important element to what precipitated the last stand. At this point in the war (late 1944), the Japanese Navy had few planes and few experienced pilots, so their aircraft carriers were not an effective force. This despite the fact that the Japanese had shown at Pearl Harbor that the future of surface naval warfare was almost exclusively to be done via aircraft. So they decided to use their aircraft carriers as bait for the Third Fleet commanded by Admiral Halsey. A bait he took; hook, line and sinker.

This left the northern edge of the Seventh Fleet, guarding the San Bernardino Strait basically undefended except for 3 task forces, Taffies 1-3, with just a slew of “jeep” carriers and destroyers and destroyer escorts. Taffy 3 was the northernmost of these and the ones directly engaged by the Japanese fleet. They were soon to be met be the IJN Yamato and and the rest of Admiral Kurita’s fleet of battleships and cruisers. By any measure, Taffy 3 was outgunned and outmatched. Yet, by the end of the day, despite the loss of 2 destroyers, 1 destroyer escort, and 2 escort carriers, the Japanese fleet had lost 3 heavy cruisers, 3 more damaged, a destroyer damaged and the loss of 52 aircraft (compared to the US losing 23) and was in full retreat.

At this point, and for the last 77 years one could reasonably ask, “why?” What drove Admiral Kurita’s decision to withdraw. Unfortunately, most of the answers are predicated on guesswork, educated guesswork, but still guesswork all the same. The simple answer appears to be two fold. For one, he didn’t know if Admiral Halsey had taken the bait, and in fact it appears that he didn’t think Halsey had, and that he was in fact attacking the fleet carriers, not escort carriers, and hence a much larger American fleet than was actually present. But despite his erroneous belief about the American Third Fleet’s position, he was most likely correct in his appraisal of the future of the mission: he did not believe he could continue forward and disrupt the landings. Since that was the primary goal of his mission and it most likely would fail, it appears he saw no point in risking the rest of his fleet and withdrew.

One can speculate what would have happened had he continued on with the battle. My personal, and mostly uneducated guess, is that he probably would have succeeded in sinking the other 2 carriers of Taffy 3 and perhaps the rest of the destroyers and destroyer escorts. However, his position was extremely precarious with the growing number of American aircraft starting to make sorties from Taffy 2 and from an improvised airstrip the Army had prepared and the pilots from Taffy 3 had basically taken over. It’s most likely he would have ended up with several more of his own ships on the ocean floor, including the Yamato.

So, he made what he thought was the best decision based on the information he had at the time. As did Halsey when he took the bait of the Northern Force of the basically defanged Japanese carriers.

So why do I recap all of this? Because I think it’s topical to a lot of what we do at times. This past weekend I was upgrading a SQL Server for a customer. Fairly routine work. And I ran into problems. Things I wasn’t expecting. It threw me off. Fortunately I was able to work around the issues, but it got me thinking about other upgrades and projects I’ve done.

The reality is, in IT (as well as life) we make plans to get things done. Sometimes they’re well thought out plans with lots of research done prior to the plan and everything is written down in detail to make sure nothing is forgotten.

And then… something unexpected happens. The local internet glitches. It turns out there’s a patch missing you had been told was there. Or there’s a patch there you didn’t know was there. Or a manager unexpectedly powers down the server during your data center move without telling you (yes, that happened to me once).

When things go majorly wrong, we’ll do a post-mortem. We’ll look back and say “Oh, that’s where things went wrong.” But we have to remind ourselves, at the time, we didn’t know better. We may not have had all the information on hand. When reviewing decisions, one has to separate “what do we know now” from “what did they know then.”

Now we know, “…Halsey acted stupidly” to quote a famous movie. He shouldn’t have taken the bait. We know Kurita probably should have turned back earlier (since the other half of the pincer had been turned back by the Seventh Fleet, putting the Japanese plan in serious jeopardy, or perhaps pressed on a bit longer before turning back (and taking out a few more escort carriers). But we shouldn’t judge their decisions based on what we know, but only on what they knew then.

Finally, I’m going to end with a quote from the battle. Spoken by Lieutenant Commander Robert W. Copeland of the USS Samuel B. Roberts (DE-413) to his crew over the 1MC “This will be a fight against overwhelming odds from which survival cannot be expected. We will do what damage we can.” And that they did. Among other things they launched their torpedoes at the IJN heavy cruiser Chōkai, hitting and disabling it and then took on another Japanese cruiser with their 5″ guns until finally a shell took out their remaining engine room and they ended up dead in the water.

I can’t begin to fathom the heroism and bravery of the men of Taffy 3 that day. If you can, find the time to get a copy of the book and to read it.

P.S. The title of this post has an interesting story of its own, and I know at least one reader will know it all to well.

… Other Duties as Assigned

Posted on by
Reply

I’ve mentioned once before that at one of my clients I describe my job as “DBA and other duties as assigned.”

This phrase has really been on my mind this week, especially during a phone call with another client yesterday. This second client is a local consulting company that has hired me a few times to back them with my skills in SQL Server and MS Access. This time around the work they’re looking for is definitely SQL Server related. It was refreshing.

But it reminded me of my last two weeks with two of my other clients. One is having an issue with their app (that they always call “the database”) that is most likely a design issue that I need to dig into. This is a perfect example of what I call “software archeology” where I at times have to shift through “pot shards” to determine what the original developer was thinking. At times it can be fun and interesting, at other times, frustrating. I’ll be shifting through more pot shards in the near future to get to the bottom of this problem.

For my largest client, I spent most of my hours with them last week trying to true up a file with some financial data in it. In this case it’s part of an ETL process where I receive data, compile it and send it to a vendor. The process uses a combination of PowerShell and Pentaho. So while they interact with the database, the work I was doing wasn’t in T-SQL or directly on the database server.

The numbers weren’t adding up. There was an undercurrent of “Greg, your numbers are wrong” or “You’re filtering on the wrong criteria.” I kept pointing out that “I simply add up the numbers you give me.” Eventually the problem was narrowed down to the fact that in the source system, which is the system of record, they had deleted rows. Arguably, one should never be deleting rows in such a system, but rather issuing a 2nd row (a credit if you want to reverse a debit, or a debit to reverse a credit) and this was typically what was done. But in this case the maintainers of the source of record decided to wholesale delete these rows. I explained that from day one, since deletions are never supposed to happen (and given the way the system works, extremely hard to detect) all I do is either insert new rows, or update existing rows. In any event, with one minor schema change, some updates to the rows in question and an updated PowerShell script, I was able to make the numbers come out to match with theirs. So, is that really DBA work? Not in the traditional sense. But it’s definitely other duties as assigned.

Now that’s not to say I didn’t do what some might consider actual DBA work. On Saturday morning I patched one of their servers. And at one point during the week, I deployed a script to production. So, out of 18 hours of work for the customer last week, I think I can say maybe 1-2 total was “dba work” or about 5%.

Now, I want to be clear. This is not a rant or a complaint. I’ll admit I tend to prefer to work directly with SQL Server, but I was reminded of a quick discussion I had with a fellow DBA over the weekend about how they probably needed to start to learn PowerShell for their job.

I’ve been arguing for years that the role of a DBA has changed, and will continue to change dramatically over the next few years. Once where we might spend days head down slinging T-SQL code, setting up backups and restores, tuning indices, etc. now much of that is automated or at least far easier to do. Which is a good thing. In years past, a DBA might be responsible for a dozen machines or so at the most. If it was more than that, we’d feel sorry for them. That’s no longer uniformly true. I know a DBA who is responsible for over 100 machines. They’re the soul DBA. But, through PowerShell and other modern tools, it’s generally not an overwhelming job.

However, like the online presentation from the Atlanta Azure Data User Group I attended last night on SQL Database Edge, there is a growing list of things DBAs need to learn. Steve Jones recently posted about whether DBAs need to learn Linux? The short take away is not necessarily, but it’s probably a good idea, but we definitely need to learn about containers.

I have heard for years, “Microsoft will automate everything and the DBA’s job will go away.” Not only is that not true in my experience, the exact opposite is. I think being a successful DBA is in some ways harder than it was a decade ago. There’s so much more to be aware of and to learn.

Off the top of my head, without any real priority I came up with the list below of technologies that a modern DBA might find useful to know. This is not to say I know them all, or that one has to be an expert in all of them. And I will note, this is far from an inclusive list. I also left out third-party tools which are so common place. But I think it illustrates just how broad the required skillset of a good DBA is these days.

  • T-SQL
  • PowerShell
  • Query Store
  • Linux – at least at the most basic level
  • Containers
  • SSIS
  • SSAS
  • SSRS
  • Storage – (at least how different types can impact performance and the advantages and disadvantages of each)
  • Azure
  • SQL Database Edge
  • git or some form of version control

In conclusion, I’ll say, I’m not going to make any predictions about where the Microsoft data platform will be a decade from now, but I can tell you that DBAs will still be needed but their skillset will be as different from today as today is from a decade ago.

And post conclusion, I’ll add I’ll continue to rely on #sqlfamily and all my fellow DBAs to help me out. And continue to help them.

Take 5 Minutes

Posted on by
2

This weekend I had the pleasure of moderating Brandon Leach‘s session at Data Saturday Southwest. The topic was “A DBA’s Guide to the Proper Handling of Corruption”. There were some great takeaways and if you get a chance, I recommend you catch it the next time he presents it.

But there was one thing that stood out that he mentioned that I wanted to write about: taking 5 minutes in an emergency. The idea is that sometimes the best thing you can do in an emergency is take 5 minutes. Doing this can save a lot of time and effort down the road.

Now, obviously, there are times when you can’t take 5 minutes. If you’re in an airplane and you lose both engines on takeoff while departing La Guardia, you don’t have 5 minutes. If your office is on fire, I would not suggest taking 5 minutes before deciding to leave the building. But other than the immediate life-threatening emergencies, I’m a huge fan of taking 5 minutes. Or as I’ve put it, “make yourself a cup of tea.” (note I don’t drink tea!) Or have a cookie!

Years ago, when the web was young (and I was younger) I wrote sort of a first-aid quiz web-page. Nothing fancy or formal, just a bunch of questions with hyperlinks to the bottom. It was self-graded. I don’t recall the exact wording of one of the questions but it was something along the lines of “You’re hiking and someone stumbles and breaks their leg, how long should you wait before you run off to get help.” The answer was basically “after you make some tea.”

This came about after hearing a talk from Dr. Frank Hubbell, the founder of SOLO talk about an incident in the White Mountains of New Hampshire where the leader of a Boy Scout troop passed out during breakfast. Immediately two scouts started to run down the trail to get help. While doing so, one slipped and fell off a bridge and broke his leg. Turns out the leader simply had passed out from low blood sugar and once he woke up and had some breakfast was fine. The pour scout with the broken leg though wasn’t quite so fine. If they had waited 5 minutes, the outcome would have been different.

The above is an example of what some call “Go Fever”. Our adrenaline starts pumping and we feel like we have to do something. Sitting still can feel very unnatural. This can happen even when we know rationally it’s NOT an emergency. Years ago during a mock cave rescue training exercise, a student was so pumped up that he started to back up and ran his car into another student’s motorcycle. There was zero reason to rush, and yet he had let go fever hit him.

Taking the extra 5 minutes has a number of benefits. It gives you the opportunity to catch your breath and organize the thoughts in your head. It gives you time to collect more data. It also sometimes gives the situation itself time to resolve.

But, and Brandon touched upon this a bit, and I’ve talked about it in my own talk “Who’s Flying the Plane”, often for this, you need strong support from management. Management obviously wants problems fixed, as quickly as possible. This often means management puts pressure on us IT folks to jump into action. This can lead to bad outcomes. I once had a manager who told my team (without me realizing it at the time) to reboot a SQL Server because it was acting very slowly. This was while I was in the middle of remotely trying to diagnosis it. Not only did this not solve the problem, it made things worse because a rebooting server is exactly 100% not responsive, but even when it comes up, it has to load a lot of pages into cache and will have a slow response after reboot. And in this case, as I was pretty sure would happen, the reboot didn’t solve the problem (we were hitting a flaw in our code that was resulting in huge table scans). While non-fatal, taking an extra 5 minutes would have eliminated that outage and gotten us that much closer to solving the problem.

Brandon also gave a great example of a corrupted index and how easy it can be to solve. If your boss is pressuring you for a solution NOW and you don’t have the opportunity to take those 5 minutes, you might make a poor decision that leads to a larger issue.

My take away for today is three fold:

  1. Be prepared to take 5 minutes in an emergency
  2. Take 5 minutes today, to talk to your manager about taking 5 minutes in an emergency. Let them know NOW that you plan on taking those 5 minutes to calm down, regroup, maybe discuss with others what’s going on and THEN you will respond. This isn’t you being a slacker or ignoring the impact on the business, but you being proactive to ensure you don’t make a hasty decision that has a larger impact. It’s far easier to have this conversation today, than in the middle of a crisis.
  3. If you’re a manager, tell your reports, that you expect them to take 5 minutes in an emergency.

Stuck, with Responsibility

Posted on by
1

So, by now, you may have all heard about the vehicle that got stuck trying to go through a somewhat narrow passage. No, I’m not talking about the container ship known as Ever Green. Rather I’m talking my car and the entrance to my garage!

Yes, due to circumstances I’ll elucidate, for a few minutes the driver’s side of my car and the left side of my garage door opening attempted to occupy the same spot in space and time. It did not end well. The one consolation is that this mishap was not visible from space!

Now I could argue, “but it wasn’t my fault! My daughter was driving.” But that’s not really accurate or fair. Yes, she was driving, but it was my fault. She’s still on her learner’s permit. This requires among other things, a licensed driver (that would be me) in the vehicle and observing what she was doing. She did great on the 8 mile drive home from high school. So great in fact that when she paused and asked about pulling into my garage, I said “go for it.”

To understand her hesitation, I have to explain that the garage is perpendicular to the driveway and a fairly tight turn. It’s certainly NOT a straight shot to get in. I’ve done it hundreds of times in the last 5 years (when the garage was added to the house) and so I’ve got it down. Generally my biggest concern is the passenger side front bumper “sweeping” into the garage door opening or the wall as I enter. I don’t actually give much thought on the driver’s side.

So, I gave her the guidance I thought necessary: “Ok, stay to the far right on the driveway, this gives you more room to turn.” “Ok good, start turning. Great. Ok. Ayup, you’ve cleared the door there, start to straighten out.” “Ok you’re doing…” Here the rest of the cockpit voice recorder transcript will be redacted other than for the two sounds, a “thunk” and then a “crunch”. The rest of the transcript is decidedly not family friendly.

The investigator, upon reviewing the scene and endlessly replaying the sounds in his head, came to the following conclusions:

  • The “thunk” was the sound of the fold-way mirror impacting the door frame and doing as was intended, folding away.
  • The “crunch” was the sound of the doors (yes, both driver’s side doors) impacting the said door frame.
  • Both the driver and the adult in charge were more focused on the front passenger bumper than they were on distance between the driver’s side and the door frame. Remedial training needs to be done here.

Anyway, I write all this because, despite what I said earlier, in a way this is a bit about the Ever Green and other incidents. Yes, my daughter was driving, but ultimately, it was my responsibility for the safe movement of the vehicle. Now, if she had had her license, then I might feel differently. But the fact is, I failed. So, as bad as she felt, I felt worse.

In the case of the Ever Green, it’s a bit more complex: the captain of a ship is ultimately responsible for the safe operation of their vessel. But also, in areas such as the Suez Canal, ships take on pilots who are in theory more familiar with the currents and winds and other factors that are local to that specific area that the captain may not be. I suspect there will be a bit of finger pointing. Ultimately though, someone was in charge and had ultimate responsibility. That said, their situation was different and I’m not about to claim it was simply oversight like mine. My car wasn’t being blown about by the wind, subject to currents or what’s known as the bank effect.

What’s the take take-away? At the end of day, in my opinion and experience, the best leaders are the ones that give the credit and take the blame. As a former manager, that was always my policy. There were times when things went great and I made sure my team got the credit. And when things went sideways, is when I stood up and took the blame. When a datacenter move at a previous job went sideways, I stepped up and took the blame. I was the guy in charge. And honestly, I think doing that helped me get my next job. I recall in the interview when the interviewer asked me about the previous job and I explained what happened and my responsibility for it. I think my forthrightness impressed him and helped lead to the hiring decision. The funny part is, when I was let go from the previous job, my boss also took responsibility for his failures in the operation. It’s one reason I still maintained a lot of respect for him.

So yes, my car doors have dents in them that can be repaired. The trim on my garage door needs some work. And next time BOTH my daughter and I will be more careful. But at the end of the day, no one was injured or killed and this mistake wasn’t visible from space.

Stuff happens. Take responsibility and move on.

The Customer is Always Right?

Posted on by
Reply

You’ve heard this adage before. Some often believe it. And honestly, there’s a bit of truth to it. But the truth is, the customer pays your bills and if they stop paying your bills, they’re no longer your customer.

I was reminded of this actually during the testimony of Dr. Fauci before the Senate a few weeks ago. To be clear, neither the Senate nor the CDC is a customer here, nor is the President of the United States. But, I think it ties into the thesis I want to make.

Over the past few months there’s been a lot of discourse over whether states should shut down, for how long and how and when they should open up. At the extreme ends you have folks who seem to argue for a continual shutdown to save as many lives as possible and the people who seem to argue that the economy is far more important and that any shutdown is a bad idea.

Dr. Fauci has been accused of wanting to “quarantine the entire country” and is the subject of a hashtag campaign, #faucifraud. During the Senate hearing, Senator Rand Paul took several swipes at Dr. Fauci including a statement implying that some people might treat Dr. Fauci as the end-all. Finally, with only 32 seconds left, Dr. Fauci gave his reply:

“Well, first of all, Senator Paul, thank you for your comments. I have never made myself out to be the end all and only voice in this. I’m a scientist, a physician, and a public health official. I give advice, according to the best scientific evidence. There are a number of other people who come into that and give advice that are more related to the things that you spoke about, about the need to get the country back open again, and economically. I don’t give advice about economic things. I don’t get advice about anything other than public health. So I wanted to respond to that.”

I think this was an incredible reply and one that I think behooves any consultant to keep in mind. Dr. Fauci politely but firmly refutes Senator Paul’s comment about being the end all and then points out what his qualifications are. He then suggests that there are other experts, in other fields, who should be consulted. He then reiterates the limitations of his advice.

As a consultant, this mirrors my own experience. A client may ask me to recommend a HA/DR strategy for them. I might go ahead and recommend some sort of Always On Availability Group with multiple synchronous replicas in one data center and then an asynchronous replica to a second data center. I might then recommend daily backups to tape with the tapes taken off-site. Everything would be encrypted and we would test failovers on a regular basis. With that, the proper selection of hardware, a proper deployment setup, and a completely developed runbook for various scenarios, I could probably guarantee them nearly perfect uptime.

Then, the CFO steps in and points out that their budget is only 1/20th of what I had planned around and that trying to spend more would bankrupt the company.

Then the VP of Sales points out that the business model of the company is such that in reality, they could operate for several hours of downtime and while it might hurt business a little bit, it wouldn’t bring them to a halt.  In fact, they suggest that the order system just be done with a bunch of Excel spreadsheets that accounting can true up at the end of the month. After all, they just want to focus on sales, not on entering data into the system.

Finally the CEO steps in. They decide that it’s true, the company can’t afford a 24/7 HA/DR setup that is the envy of NASA, at least not at this time. Nor do they think the VP of Sales plan has much merit since it won’t allow future growth into online sales and while it might be easier for the salespeople to just jot down stuff, it would mean hiring more people in accounting to figure out the data at the end of each month.

Instead, they direct the CTO to work with all the parties involved to develop a system that can have 3 hours of downtime, but costs no more than 1/10th of what I proposed, and  that also incorporates features that allow them to move to a more advanced HA/DR setup down the road and also will eventually allow for online sales.

So who was right? Me at my extreme of a huge investment in hardware, licenses and resources, or the VP of Sales who wanted to do the whole thing using some Excel spreadsheets.

Both and neither. Either of our ways would have worked, but neither was the best solution for the company.

I think good experts realize exactly where their expertise begins and ends. My role as a consultant is to provide the best advice I can to a company and hope that they take my advice, at least as it is applicable. And I should understand that every situation is different. In these cases, the customer is always right. Their final decision might not be what recommended, but ideally they’ve taken it into consideration.

Finally though, I have to recognize that there are situations where I may have to withdraw myself from the situation. In the above scenario, I crafted a situation where compromise is not only a viable option, it is perhaps the best option. But there are times when compromise is not an option. If a potential client came to me and they were dealing with PII data and refused all advice in regards to encrypting data and other forms of data security, it would be in my best interests to simply say, “here, the customer is wrong” and recuse myself. So in some cases, the customer may be outright wrong, but they should also stop being my customer at that point and would no longer be paying me.

So, do we re-open the country completely or do we shutdown everything until the fall? Honestly, I’m glad I’m not the one making that decision. I don’t think there’s a single answer for every community. But I think the best leaders will take into account the best advice they can from a variety of experts and synthesize the best answer that they can and adjust it as more data and experience come to light. It’s simply not practical to prevent every possible COVID-19 death. But it’s also not ethical to re-open without a plan or even thought as to the impact.

Neither extreme is fully right.

 

Advanced Braining

Posted on by
1

I’m currently reading the tome The Power Broker by Robert Caro. For those not familiar with it, it’s the Pulitzer Prize winning biography of Robert Moses. “Robert who?” you may be asking? Robert Moses, perhaps more than any single person literally shaped New York City in the mid-20th Century. Due to his power, he was responsible in NYC alone, for getting the Triborough Bridge, Brooklyn-Battery Tunnel, West Side Highway, Cross-Bronx Expressway, and many other large scale projects built. He outlived a number of borough presidents, mayors, governors and even Presidents. Arguably, for decades he was the most powerful man in NYC, at least in terms of how many was spent and what projects were completed. In many ways he was a visionary.

However, as the chapter I’m currently in discusses, he also could be extremely short-sighted. I’ll come back to that in a moment.

In the past week, several small incidents occurred in my life. Separately, they don’t necessarily mean much, but taken together, I realized there was a little theme associated with them.

Last Tuesday I posted an update on my dryer repair and an issue at one of my clients. I described the work incident as an example of the normalization of deviance. A few hours later, someone I’ve known for decades, originally online, but have since met in person, Derek Lyons (who has a great blog of his own on anime, a subject about which I know nothing) posted a reply to me on Facebook and said he had read my article, liked it, but thought I was wrong. I was intrigued. You can see his comment and my reply at the bottom of last week’s post. The general point though is I think he showed my thinking was incomplete, or at least my explanation was. In either case, it made the overall article a better one.

Then on Wednesday, my editor at Redgate, Kathi Kellenberger  emailed me with changes to my most recently submitted article. One of the changes was to the title of the article. Now, I’ve come to value Kathi’s input, but I wasn’t keen on the title change, so I suggested something different. She wrote back and recommended we go with hers, How to Add Help to PowerShell Scripts because she said “How to…” generates more hits on search engines and in fact a previous article of mine How to Use Parameters in PowerShell was one of their most read articles at the time (106K hits and climbing). I went with her advice.

Yesterday, a friend contacted me. He was in the middle of doing grading for his students and the numbers on his Excel spreadsheet weren’t quite making sense. The errors weren’t huge, but just enough to make him go “hmmm”. So, he reached out to me to take a look. After a few minutes of digging I understood what was happening and able to write back to him and give him a better solution.

All these have something in common: the final product was better because of collaboration. This is a common theme of mine: I’ve talked about the chat system I use at RPI, I’ve talked about making mistakes. In general, I think that when trying to solve a problem, getting additional input is often valuable.

So back to Robert Moses. In the early part of his career, before his efforts focused mostly on NYC itself, he was responsible for other projects, such as the Northern State Parkway and the Southern State Parkway and Jones Beach on Long Island. He started his career in a time when cars were mostly a vehicle of the well-off and driving a parkway was expected to be a pleasant experience (hence the name). His efforts were built around more and more parkways and highways.

By the 1950s though, it was becoming apparent to most everyone else that additional highways actually generated more traffic than they routed away from the area surface roads. What was originally considered a blessing in disguise, where a bridge, such as the Triborough would quickly generate more traffic (and hence more tolls) than expected, was soon seen as a curse. For every bridge or tunnel built in or around NYC, traffic increased far more than expected. And this came at a price. Urban planners around the country were starting to see the effects. Efforts to build more bridges or highways to ease traffic congestion were actually creating more. Even in NYC as Moses was planning for his next large projects opposition was slowly building. However, Robert Moses was blind to the problem. By the 1950s and 60s he had so surrounded himself by “yes men” that no dissidence was permitted. In addition, opposition outside of this offices was silenced by almost any means Moses could use, including apparently the use of private detectives to dig up dirt on opponents.

In the current chapter I’m reading, Caro, the author, details exactly how much money the Triborough Bridge Authority (which was in practice, though not theory, under Moses absolute control) and the Port Authority had available for upcoming projects, including the planned Verrazzano-Narrows Bridge. He goes on to explain how badly the infrastructure of the NY Subway system and the LIRR had fallen into disrepair. Caro suggests how much better things could have been had just a portion of the money the TBA and PA had at their authority had been spent on things like the Second Avenue Subway (something that is only now coming to fruition and will take possibly decades more to complete). Part of the issues with the subway system can be lain directly at the feet of Moses due to earlier efforts of his to get the city to fund his other projects. The issues with the LIRR however were more an indirect result of his highway building out into Long Island.

I suspect some of Caro’s claims are a bit idealistic and would have cost more than the projections at the time (like most projects) and while I think most of the projects he touches upon probably should have been built in the 50s (the Second Avenue Subway being one of them and the LIRR East Side Access being another) they weren’t because of a single man who brooked no disagreement and was unwilling to reconsider his plans.

Robert Moses was a man who got things done. Oftentimes that’s a good thing. And honestly, I think a number of his achievements are remarkable and worthy of praise.

But I have to wonder, how much better of a city could New York be, had Robert Moses listened to others, especially in the 1950s and 60s.

Today’s takeaway? Take the time to listen to input and ask for it. You may end up with a better solution in the long wrong.

 

 

 

It’s too late?

Posted on by
4

I want to start with a sobering thought. It’s too late to contain this pandemic. I’m watching the news as slowly more and more states in the US issue versions of “shelter in place” or “stay at home” orders. But I think in most cases, it’s too late. The virus has probably already spread so much that self-isolation won’t be nearly as effective as it would have been had the states issued the same orders a week or two earlier. That said, it’s most likely still better than doing nothing.

Human beings at times are lousy with risk analysis. If a risk is immediate, we can react well, but the longer it stretches out or the further away it is, the harder it is to get people to react. Almost any climate scientist who has studied anthropogenic global warming has known for a decade or more we have a problem and we have a very quickly narrowing window for solving it, and the longer we wait, the harder it will become.

Yet too many of us put off the problem for another day.

So it is with the Covid-19 virus. “Oh we don’t have to lock down just yet, let’s wait another day.” And I’ll admit, sitting in the state that is the center of the virus outbreak here in the US, I’m tempted to say, “25,000 isn’t TOO bad, we can manage that.”  But that’s the lizard part of my brain reacting. It’s the emotional part. Then I kick in the rational part. If we use one of the numbers bandied about, doubling every 4 days, that means by this weekend, in New York State alone, it will be 50,000. By April 1st, 100,000. By the end of April, it could be the entire state.  Those numbers are hard to comprehend.

That said, I’m also hopeful. Modelling pandemics is pretty much pure math, but reality is more complex and often luck can play a huge factor. Let me try to explain.

First, we need to heed the words of experts like Dr. Fauci and others who are basing their remarks and recommendations on the inexorable exponential rise in expected infections. They are giving basically the worst case scenario if their recommendations are followed. And that’s proper. That’s really what you have to plan for.

Let me take a little side trip and mention a cave rescue in Vermont several years ago. By the time I had gotten the call to show up and to call out other rescues, the injured party had been in the cave for several hours. I didn’t know much about the extent of their injuries other than it was a fall and that it was in a Vermont cave, which almost certainly meant operating in tight quarters. I grabbed a box of Freihofer cookies, a lawn chair (my fellow cave rescuers will understand the reference), a contact list of other potential rescuers, and my son. While I drove, he’d read off a name and I’d say “yes call” or “Nope, next name.”  On the hour plus drive to the rescue we managed to contact at least two other people who could get there. (It turns out, as I surmised, several of the folks I wanted to call were members of the original caving party.)

Once there, my son and I were driven partway to the cave entrance and trudged the rest of the way. I talked with the folks on the scene to gather information and then dressed to go into the cave to gather first hand information. I still hadn’t gained too much information other than to know it was potentially shaping up to be a serious rescue. The person had been climbing a cable ladder when they fell and injured themselves. This meant, based on the information at hand, a worst case scenario of an evac through tight passages with the patient in a SKED stretcher.  I was playing the role of Dr. Fauci at that point, preparing for the worst based on the information I had.

Fortunately, literally at the moment I was about to enter the cave, one of the members of the original caving party crawled out and said, “he’s right behind me, he’ll be out in a minute or so.”  It turns out his injuries were fairly minor and with the members of his own caving party, he was able to get out of the cave under his own power.

I got back to Incident Command about an hour later and was informed, “oh, by the way, you’ve got at least 3 cavers who showed up to help. We held them at the bottom of the road. What should we tell them?”  My answer was simply, “Thanks and to go home.”

I relate this story not so much to talk about cave rescue specifically but to point out that even when planning for the worst, you may get a lucky break. But you can’t rely on them. Let me give an alternate scenario. Let’s say I had not called out the other rescuers and had gotten to the cave and crawled in, realized the situation was a worst case scenario, crawled back out and then initiated a call-out. It would have at that point probably meant at least an extra 90 minutes before the extra resources would have been on the scene. It would have meant the patient was exposed to hypothermic condition for another 90 minutes. It would have meant 90 more minutes of pain. It would have meant fewer brains working to solve the problem.

Getting back to Covid-19. Will we get lucky? I don’t know. I actually suspect we might. One “advantage” of an increasing population of sick people is we can better model it and we can also perform more drug trials. We may discover certain populations react differently to the disease than others and be able to incorporate that into the treatment plan. I don’t know. But I do know, we need to plan for the worst, and hope for a bit of luck. In the meantime, hunker down and let’s flatten the curve.

And if you’ve read this far and want to know how to make some pita bread, I’ll let you in on the two secrets I’ve learned: VERY hot oven (I typically bake mine at about 500-550F for 2 minutes on 1 side, and 1 minute on the other) and roll it out thinner than you might think.

Them’s the brakes…

Posted on by
4

I need brake work done on my car.  I scheduled an appointment with my mechanic for Thursday.

Now normally brake work might not be worth writing about, but this time it got me thinking. About 9 years ago I taught myself to change the brakes on my Subaru. It wasn’t that hard, but it definitely took longer than I would have liked. I’ve learned how to do it much faster since then.

Back then I did it because I was between jobs and had more time than money. It made sense at the time. Since then I’ve generally continued to do them myself. It doesn’t take long and as I’ve said in the past, sometimes I like working with my hands. It reminds me not all of life is simply databases and servers.

So why this time? The simple answer is, my garage floor is cold!  Ayup, it’s a funny thing, but in the winter, things get cold! And honestly, I just don’t feel like sitting/laying on the concrete while I do the work. So, I’ll pay an expert who can pop it up on a lift and do the work in probably half the time I can.  I remind myself it’s why I work, so I can pay others to do work I don’t care to do, or that they can do better than I can.

Years back, at my last job as the Director, and later VP of IT, I changed our email provider from one company to another (don’t ask me who they were, I can’t recall over a decade later).  My team didn’t like this. They kept insisting they could run the mail servers themselves. I had no doubt that they could. Running an Exchange server isn’t the most difficult thing in the world. That said, I kept saying no.  For a simple reason: “we weren’t an email company!” Providing reliable email for a business like we were meant at the very least strong spam protection and of course high-availability with ideally geographic redundancy. This meant at least two Exchange servers, spam filters and more. This alone would have cost more than what we paid annually. Now add in the time my team would have spent on email issues, it just wasn’t going to be worth it.

And, we’re seeing that more and more with services such as Azure. Sure, many businesses run change their own brakes, err, run their own SQL Server. But more and more are outsourcing to platforms such as Azure. But there are still a number of companies that for various reasons don’t do that. Fortunately for them, there are consultants like me to help them with their servers. SQL Server has always been easier to maintain than some of its competition, especially when it was first available. But that has never meant it needed no intervention. Someone still needs to make sure that jobs are being run, that backups are available and more.

So, some days I change my own brakes because it’s fun and easy, some days I pay someone.  Some days my clients handle their own SQL Server issues, and other days, they pay me.

No real revelation here. No real advice on when to draw the line on outsourcing, just an observation. But in my case, my concrete is too cold, so I’ll pay someone else to do the dirty work.

“Do What You Love…

Posted on by
Reply

And you’ll never have to work a day in your life.” – Confucius, Mark Antony, Mark Twain.

Honestly, I think that’s some of the worst advice ever. It’s a sure way to end up hating something you love doing. Or if you do follow it, make sure you understand what it is that you love.

I first realized this in high school. I had signed up to do JV soccer, something I enjoyed, but I can’t say I loved. Before school started, we had a day of orientation. It included a hike or run up the mountain behind the school. I loved the woods and I loved (and still do at times) running through the woods. Somewhere along the way, a fellow student saw me running and suggested if I enjoyed running through nature so much, that I consider doing Cross-Country instead as my fall sport. I took their advice; snd hated it for two fall semesters in a row.

I realized what had happened was that I had replaced what to me was a fun, non-competitive activity and turned it into something where I had to perform at a specific level every single time. What I loved was running through the woods, not running competitively.

People assume I love working with computers.  That’s not entirely true. I ENJOY working with computers. I enjoy solving problems and computers are one way I can express that joy. When I make a query run 10x faster, or automate a process that previously took someone an hour a day to do, I enjoy that. But do I love it? Probably not. And I’m actually grateful for that. Because if I loved it, it would mean those days of drudgery where I bang my head against the wall all day trying to solve a problem, or I’m up until 3:00 AM recovering a failed server would turn something I love into something I dread. Something I loved would become a chore.

I love to teach caving. I get a real thrill out of it. But, I suspect that if I spend 40 hours a week, 50 weeks a year doing it, it would soon become a chore.

As my kids started their college journey, I’ve advised them, “find something you enjoy, not something you love. Keep the something you love for your own personal time so it doesn’t become a chore.” And that’s my advice to anyone.

But hey, I could be wrong. What are your thoughts? Did you pick your job because you love it and if so, do you still love it, or did you end up resenting it at all? Or do you enjoy your job?