I’m A DumBAss
I’m a DBA. That’s an acronym for Database Administrator. I tried getting them to call me GOD, Guardian Of Data, but the idea has been met with something less than warm enthusiasm.
I’m in the middle of my on call week. I and six of my comrades take turns once every seven weeks strapped to our Blackberries 24 x 7, to triage any email alerts that come in from our computer systems. My employer provides educational services to over 65,000 students and we have hundreds of servers in production, test and development environments. These machines do a lot of squawking. Ten percent of the time the squawking requires immediate intervention. The other 90 % of the alerts are along the lines of “I’ve got a momentary problem, but I’ll be OK in a minute or two.” No matter…every alert must be looked at and evaluated, whether it comes at 8:00 PM or 3:00 AM. We have fifteen minutes to evaluate the message and send out a status to the rest of the team.
I’d say I clean the sleep boogies out of my eyes to check out alerts four or five times a night. I’m the type of person that usually can’t get back to sleep at night once I wake up. Also, there is the constant terror of missing an alert. There are many times during the night when I wake up, snorting loudly, wiping the drool off my chin, grabbing blindly for my Blackberry because I had the temerity to sleep for an entire 20 minutes straight. The individual in charge of our department, Madame Vice President, is an operations fiend. She stays connected all the time. Woe be the on call guy who sleeps through an alert and gets the 4:00 AM phone call from Madam Vice President. The walk of shame to her office is not a pleasant one….or so I’ve heard 
.
Recent on call hilarity? Let’s see. The last time I was on call one of the computers holding all our company’s files went down at 3:00 AM on a Saturday . This is one of our more critical servers and it had to be brought up so that the CEO could access his financial reports or his downloaded Girls Gone Wild videos when he awoke Saturday morning. So, I tried logging into that computer without success. That meant a trip to the office, six miles away, to eyeball the little bastard. As I passed a bank on the way in, the temperature on the sign registered thirteen degrees.
One of the cool things about our company is that the corporate offices are spread across a number of 19th century buildings. My problem child was sitting in a building that used to be a hospital, in what used to be the morgue. I parked, entered and immediately a stentorian woman’s voice began alerting me that the alarm had to be disarmed or dire, unspecified, consequences would follow. I don’t go into this building a lot and was unfamiliar with the placement of the keypad, so when I finally disarmed the alarm, I’d done so three seconds too late. I sat down on the steps, in case the cops came. I didn’t want to be walking up from the basement, where the morgue was located, and be shot as an intruder. After fifteen minutes of waiting in this increasingly creepy building, I went down to the morgue and found that the computer was plugged into a power source that didn’t work. I took care of that problem and drove home, unmolested by police or the ghosts of patients who didn’t leave the hospital upright.
On call week begins on Friday at 2:30 PM, at the On Call Handoff Meeting. The entire team gathers to hear a briefing from the guy going off on call. Sleep deprived and with bags under his eyes, this fellow is almost insane with giddiness. He gives his briefing and then, with a flourish, makes a great show of unholstering his Blackberry and turning it to “Phone Only”. The person going on call is mocked and taunted with things that may go horribly wrong with the computer systems that week.
Friday evening through Monday morning sucks, because you’re truly on call 24 X 7. During the weekdays its better because during work hours, they guys in charge of the system fix anything that breaks. On weekends, though, you’re on that wall alone. Monday, Tuesday and Wednesday nights are a death march. Either you’re up responding or afraid you’re going to miss something, so decent REM sleep is out of the question. Yes, it’s good to have made it through the weekend, but Friday is a looooong way off. Thursday morning there is a slight bounce in your step with the realization that you are about to spend your final evening on call. Friday morning, no matter what the weather is actually like outside, you pull up the blinds to find nothing but sunshine, birds singing and, yes, perhaps even the smell of fresh baked apple pie in the air. There is nothing quite like the feeling on a Friday morning knowing that some other poor nimrod will be in charge of walking the wire at 2:30 PM.
Alas, DBA in my case also stands for DumBAss. Why? Because I volunteered for on call. The network and system’s engineers and administrators have to do it: it’s part of the job description. I’m a senior individual in my organization and my skill set his pretty narrow and, fortunately for me, in high demand. If I so desired, I could have made myself available by phone if anything in my area of responsibility went wrong (which rarely happens). I wasn’t in the on call rotation when I joined the company almost three years ago. At the time I was on my own little island while groups within the operations department fought each other. Network guys had their own on call schedule because they didn’t want the systems guys to touch any of their stuff, and visa versa. The entire atmosphere was toxic. My current boss, Madame Vice President, joined the company and the hammer came down. Everyone was to play nice, learn each other’s job and everyone would be in the same on call rotation. There was castigation, there were firings. In the spirit of building a big, happy team, I volunteered to be a part of the new on call structure. Although I would have to rely heavily on documentation to resolve any problems that came up, at least I could provide the guys on my team with a little more shut eye. Madame Vice President points to this selfless act as the start of our team becoming a team. Then she, and the other guys on my team, point to the dark circles under my eyes and mock me.
That’s ok, though. My turn to mock is coming…in 72 hours, 51 minutes and 14 seconds.
