An interesting question indeed and without proper testing under controlled circumstances, an accurate answer may not be viable. But I would like to offer a theory on possibly reasons for the slowing of performance.
Its possible the forums are structured in a way that if you have boards ignored, each time you're on any forum page, it has to search the contents of the page to determine if any of the ignored boards are present and to block them from your view. It seems like an inefficient way of blocking a topic but is possibly the performance drag you're experiencing.
Hope that helps?
M&M
