One can use a different pooling strategy or a combination of strategies, such as learning a single pooling structure for bottom-up features while maintaining the original top-down architecture. This would allow for preserving the benefits of incorporating top-down information without incurring the performance costs associated with certain pooling methods.