When approaching a performance problem this method of train and test is, sadly, not yet fully applicable. This is why we have to keep our software as scalable as possible, to enable it to fit on the go with increasing (or falling) demand.
The ability to train on real time data (or learn from mistakes) using a closed feedback loop is a common method to do adaptive machine learning, and is exactly the same as we use to determine how many instances of our software currently needs to be up.
In Storm, circles in topologies are usually looked upon as a perversion, but with a couple of simple tricks you can always move the model out of Storm to a datastore (Redis maybe?), and separate the training bolt from the predicting (or classifying, or testing, depending on what stage you are) bolt.