Re: Rounding, truncating

From Stanford Wong's BJ21

Posted by Pete Moss on 21 Nov 1997, at 12:37 p.m., in response to Re: Rounding, truncating, posted by Halver on 20 Nov 1997, at 8:27 p.m.

Well, I asked for people to point out mistakes, and I think you found one. Is the "truncation" scenario viable? I don't think so! What do you do about the indices that are zero under the "round down" scenario? If you leave them as zero, there is the problem you point out. If you move them up a notch, along with the negative ones, then you get the wrong decision with positive indices that truncate to zero.

It can be stated very simply and intuitively in this way: If you use rounding to the nearest whole number to form the true-count categories, every category has a span of 1. No method of building the table can compensate for the fact that truncating creates a category around zero with a span of 2. I had thought the sense of the index could act as a switch, but as you point out, zero is in no-man's land.

Pete