(Why bother even writing it? Why not just leave it as the GPO Box number?)
Because once upon a time it wasn't the case. As told to me by an old PO tech: there was always an arrangement where gov't depts were allocated common box numbers across all GPOs, but it was very ad-hoc* & started falling apart with the rationalisation & decentralisation of government services in the 80's. IIRC, it changed in the mid-90's - Auspost were upgrading their sorting machines, cottoned onto the fact that it was a potential revenue raising premium service, & added the 9800-9999 range of common boxes.
I like to think the choice of "IN YOUR CAPITAL CITY" was down to someone with a twisted sense of humour noticing how stupid some people really were.
FWIW, according to the AusPost guidelines for common boxes & LMB you can only use those addresses on preprinted barcoded domestic reply envelopes - hand addressed mail is supposed to use a full box-city-postcode address. Doesn't stop the ATO & everyone else telling people to write "IN YOUR CAPITAL CITY" though...
(* They had a lot of weird semi-official ad-hoc stuff like that - anyone else remember SBS's old "PO Box 0-28, Crows Nest" address?)