Brute force can be better...
You have to understand something about me. I was one of those lucky kids that grew up with technology. I wanted a computer since I was 8. I got my first computer when I was 12. I was hacking assembler by the time I was 13 and C when I was 15. By the time I graduated high school I was fluent in BASIC, C, Pascal (of the Turbo variety), and three different types of assembler (6502, 6809, and 8086). I wrote a program that was part of a commercially sold product when I was 17. Ask my high school teachers -- I was more fluent in C than I was in written English. Technology, as far as I was concerned, was the answer to everything.
The media coverage of the recent election and a question that we needed to deal with at one of my customers made me recall the first time I realized that technology, for all of its wonders, does not always improve our lives.
My first job was doing "tech work" for a small medical billing VAR in Southern California. As a four person shop, "tech work" meant just about anything. There was days when I put together new systems and installed SCO Unix and days when I physically cleaned Wyse terminals. By the time I was 19 I had added custom billing reports (C, AWK, and SQL) as well as database conversions to my resume. Good times.
One of the first tasks that I was put to was database monitoring, the ugly task assigned to newbies. Each month, a few dozen faxes would come in with reports showing how big every table had become. The print outs were 132 column and about 50 tables per page. You had to look at past tables for the same doctor and figure out whether the growth rate of the database meant that a resizing needed to be scheduled. Old school databases -- tables were a fixed size and needed to be manually enlarged. Nothing quite says "Party time!" like calculating growth rates and manually resizing databases.
The task was tedious as hell. I tended to keep the work for Friday afternoons when I was getting antsy for the weekend, support volume was low, and there generally weren't other things that needed to be urgently dealt with. What immediately frustrated me about the whole thing was that I knew it could be automated -- we were looking at three months of historical data and extrapolating a rate of growth. Take the rate, add it to the current month and see if it crossed the 90% threshold for the table capacity. Throw in a few heuristics to deal with things like growth spikes on certain tables. For example, a spike in the billing line items combined with growth in the doctors table meant a new doctor joined the group full time. Translation: drop the three month average and go with the last month only.
The bottom line was that this process could be automated. It should be automated. Damnit, I hated doing this task so I would automate this.
Except for one silly problem.
There was no reliable way to transfer the table size report from the doctor's machines to me without a human faxing it. Many doctors hadn't moved to electronic billing yet and thus had no modem. We didn't have HIPAA at the time, but we were sensitive enough to appreciate that you were very careful with who saw what. Auto-dialing another computer and sending database reports was dangerous.
The conclusion I came to was that each computer would run the program and then tell the office manager when to call us. In a fit of excitement, I told the owner of the VAR. "Roxanne, I can automate the database table checking. Think of the time we'll save!" Roxanne thought about this for a minute and then insisted on asking questions. Questions like...
"How do you plan on getting these programs on everyone's computer?"
"How do you plan to test this?"
"How do you plan on sending updates when new versions of the software change the database?"
"How do you plan to support multiple versions of software that have different databases?"
"Who does the office manager call if there is a problem with the program?"
"How are you documenting this so other people in this office can answer questions?"
[...]
"Finally, how long is all of this going to take versus just getting the faxes once a month?"
Pesky... pesky... questions.
I went back to think about the answers. Each answer made me increasingly depressed and the final answer was the topper. There was no practical way that I could beat spending an hour or two a month to just brute force the solution. Even if we grew significantly and I got to do more interesting things, it was more likely that we'd just hire another lackey to do it.
I didn't appreciate the business lesson in there at the time, but I certainly appreciated the lesson in brute force. There are times when it's just faster, easier, and/or more reliable to do it by hand. Technology, it turned out, wasn't the answer to everything.
The media coverage of the recent election and a question that we needed to deal with at one of my customers made me recall the first time I realized that technology, for all of its wonders, does not always improve our lives.
My first job was doing "tech work" for a small medical billing VAR in Southern California. As a four person shop, "tech work" meant just about anything. There was days when I put together new systems and installed SCO Unix and days when I physically cleaned Wyse terminals. By the time I was 19 I had added custom billing reports (C, AWK, and SQL) as well as database conversions to my resume. Good times.
One of the first tasks that I was put to was database monitoring, the ugly task assigned to newbies. Each month, a few dozen faxes would come in with reports showing how big every table had become. The print outs were 132 column and about 50 tables per page. You had to look at past tables for the same doctor and figure out whether the growth rate of the database meant that a resizing needed to be scheduled. Old school databases -- tables were a fixed size and needed to be manually enlarged. Nothing quite says "Party time!" like calculating growth rates and manually resizing databases.
The task was tedious as hell. I tended to keep the work for Friday afternoons when I was getting antsy for the weekend, support volume was low, and there generally weren't other things that needed to be urgently dealt with. What immediately frustrated me about the whole thing was that I knew it could be automated -- we were looking at three months of historical data and extrapolating a rate of growth. Take the rate, add it to the current month and see if it crossed the 90% threshold for the table capacity. Throw in a few heuristics to deal with things like growth spikes on certain tables. For example, a spike in the billing line items combined with growth in the doctors table meant a new doctor joined the group full time. Translation: drop the three month average and go with the last month only.
The bottom line was that this process could be automated. It should be automated. Damnit, I hated doing this task so I would automate this.
Except for one silly problem.
There was no reliable way to transfer the table size report from the doctor's machines to me without a human faxing it. Many doctors hadn't moved to electronic billing yet and thus had no modem. We didn't have HIPAA at the time, but we were sensitive enough to appreciate that you were very careful with who saw what. Auto-dialing another computer and sending database reports was dangerous.
The conclusion I came to was that each computer would run the program and then tell the office manager when to call us. In a fit of excitement, I told the owner of the VAR. "Roxanne, I can automate the database table checking. Think of the time we'll save!" Roxanne thought about this for a minute and then insisted on asking questions. Questions like...
"How do you plan on getting these programs on everyone's computer?"
"How do you plan to test this?"
"How do you plan on sending updates when new versions of the software change the database?"
"How do you plan to support multiple versions of software that have different databases?"
"Who does the office manager call if there is a problem with the program?"
"How are you documenting this so other people in this office can answer questions?"
[...]
"Finally, how long is all of this going to take versus just getting the faxes once a month?"
Pesky... pesky... questions.
I went back to think about the answers. Each answer made me increasingly depressed and the final answer was the topper. There was no practical way that I could beat spending an hour or two a month to just brute force the solution. Even if we grew significantly and I got to do more interesting things, it was more likely that we'd just hire another lackey to do it.
I didn't appreciate the business lesson in there at the time, but I certainly appreciated the lesson in brute force. There are times when it's just faster, easier, and/or more reliable to do it by hand. Technology, it turned out, wasn't the answer to everything.
posted by Steve Shah at 5:56 AM
0 Comments:
Post a Comment
<< Home