Why doesn’t the taxi driver use the definite article here? He talks about the definite place, doesn’t he?
Just for interest’s sake, in the western USA we don’t say “I’m at hospital” although I am aware of this usage from having read it in books from the UK. I’d use “the” in that sentence.
Ok, Jingle, good to know. At school, we were primarily taught British English, so I’m familiar with “I’m at hospital”. 
I would be borderline shocked to hear ‘I’m at hospital’ from a native speaker. Unless the name of the hospital is ‘Hospital,’ that is very incorrect.
I have heard ‘I’m in hospital,’ said by patients, while ‘I’m at the hospital’ is ambiguous, and could mean you’re a patient, or you’re visiting a patient, or anything else.
‘In town’ is curious, it’s a very common expression, and to say ‘in the town’ would be a bit unusual. As well, instead of saying ‘out of the town,’ people just say ‘out of town.’