Does anything adverse happen to your garden if you take all of the color out of the place before a Brother arrives? And does it screw up the future production of your trees if they come and are not repelled by a trap? In other words, should I be using my traps even if I am leaving no color behind?

No, there is no need to use traps in this case.

Cool. It's good to know that I'm not screwing up my gardens by leaving them unguarded. Thanks for the info.