Rebound is the stuff that bounces off during spraying and doesn't stick. Slightly different from material carved off during shaping, but they're equally bad, and yes should be removed to the extent that it's practical.
The rebound under the rebar on the floor is very difficult to remove, but...