Propose an architectural change to your RNN design that would allow it to better capture long-term dependencies and learn from past events without being overly sensitive to noise or recent changes. Explain the tradeoffs involved in this change.

Replace standard sigmoid units with leaky units, which can maintain a memory of past values and adapt to changing conditions over time. This will allow the RNN to capture long-term dependencies and improve its ability to generalize to new scenarios, but may require more computational resources and memory.