Consider using online learning methods, such as incremental gradient descent, which can adapt to new data in real-time without requiring multiple passes through the entire training set. This approach will allow for more efficient use of compute resources while still enabling frequent updates to the model parameters. However, be aware that this may come at the cost of slower convergence rates compared to traditional batch optimization methods.
