Did the joke with Oswald really happen? We've only got Leavelle's words. So, the question is: Suppose Leavelle invented the joke with Oswald, WHY would he do that, and, more significantly, WHY would he have Oswald laugh about it? Here's my two-reason theory:
1. To let the public believe the DP arrested the 'right' guy. (Oswald, like you said Ralph, supposedly didn't deny any shooting to Leavelle. No, supposedly, he laughed)
2. To let the public believe that the DP was innocent of killing Oswald. (adding more credibility to the unexpectedness of the event, by jokingly making up 'a shooter as good as you are')
PS I wouldn't be just outraged, I would be scared! I would be scared of getting shot right there and then! It was a cruel joke.