Am I looking for small changes between consecutive generations, or am I looking for low overall values?
@mcgeochd The actual value of Shannon entropy is arbitrary. What you should be looking for is the value to stabilize after a certain number of batches. Take a look at this notebook to see an example for a real problem.