Before You Analyze Data, Understand the Context of the Problem
I had lunch the other day with a colleague who was interested in meeting to discuss more about by blog post The Middle Level Leader. We both share a passion for data in business decision-making and wanted to discuss how leadership, including middle level leaders, could better use data to drive decisions and move the business forward. We both have had similar roles in our career, although mostly different companies, so it was fun to talk about the same problems seen across organizations and across industries. One of those problems we discussed was how data is often misused or misunderstood in businesses, especially at the leadership level.
I used the following example one time when taking a “Data and Decision Making” course as part of my graduate studies. The question posed to the class was “Why is it first important to understand what the data will be used for being pulling the data?” Most people said “So you pull the right data,” but it is much more complicated than that. Consider my example, which I lived on a regular basis when I managed IT operations for a large healthcare system’s operating rooms: “What was our operating room volume?”
One might think this is a straightforward question, but anyone in surgical administration knows it is much more complicated. Depending on the target audience and what problem they were trying to solve (or what message they were trying to convey) would dictate how I would pull the data. The challenge many business face, often not even realizing it, is people many times don’t spend the time understanding the “why” before pulling the data. This lack of understanding can lead to incorrect assumptions or intrepretations of what data is actually needed. Let’s break down the potential audiences of my OR volume problem so we can start seeing the importance of context when pulling data.
The finance person: Finance cares about what makes money so often times they didn’t care about cancelled surgeries, so I had to take those out per their request. But when they said don’t count cancelled surgeries, I would have to ask: Do you mean don’t include patients who didn’t even go into the operating room? What about patients who went into the operating room but then their surgery was cancelled before they had anesthesia? What about after anesthesia but before the first incision? What about after the first incision but before the surgery was actually considered “complete?” All of these examples have “cost” while some do have potential charges to the patent or insurer. The other interesting thing about finance is they often really just wanted to know “surgeries” and not “procedures” like pain management or gastroenterology (GI) cases. So often I had to take out “procedures” from the surgical volume. Depending on the facility, this could swing the gross volume by 25% or more.
The OR manager: The OR manager often cares a lot about staffing, so that gross volume I just mentioned was critical. So the pain management and GI cases finance didn’t care about, the OR manager definitly did. Often times the staff that could be used in surgery were instead used for these procedures, so it impacts the OR manager’s budget and staffing abilities. Also, some insurance companies have contracts based on your operating room volume, but they rarely specified what kind of cases had to be done in the operating room, so you would want to include every procedure done in the four walls of an operating room when reporting back to them.
The surgeon: The surgeon is different but similar to the OR manager for their OR volume data needs. They care about completed cases because that is what they are going to get credit for, but they also care about cancelled cases because it impacts their schedule. A cancelled case is lost revenue and missed opportunity – they can’t bring in new outpatients in for surgery nor could they do any office appointments at the last-minute.
The anesthesiologist: This group cared about any case that anesthesia was involved in, so if there was a procedure or surgery in the operating rooms without anesthesia, they didn’t care. Finance, the surgeon, and the OR manager did care about these cases though, a long as it met their other criteria. Anesthesia often asked about the non-OR cases they were involved in such as Labor and Delivery not performed in an operating room, because they needed to justify their staffing in all areas of the hospital. The surgeon, OR manager, and finance didn’t care about those numbers, because it wasn’t impacting them, in their area of the hospital.
The supply chain manager: They care about all cases, completed and performed. This is because they have to prepare for the surgery and put away all the unused supplies after the surgery, regardless if the surgical case was actually completed or was cancelled.
So whenever I was asked for “OR Volume” numbers, I went through my standard set of questions for understanding who the target audience was, what decisions were going to be made from the data, and if it was going to be compared to any historical data (i.e. is it possible that they may compare apples to oranges?). What I couldn’t control and what I spent a lot of time fighting people about, is when people did compare those apples to oranges. I would get emails about reports that were sent out by someone claiming one OR volume number while the other person had another number. I was always being asked which one was “right” and which was “wrong:” the problem was, they were both right! The two reports, however, were showing data for different problems and therefore could not be compared even though they both had the label of “OR Volume.”
Context around a problem, or understanding ‘why’ the data is needed before pulling it is critical for your business’s success. If you are not careful, you will create more work for yourself chasing around “Wrong numbers.” If you are the one getting the data, you will have a lot of upset customers because you didn’t understand their need before providing them with the “Correct” data.