Re-implementation of ActiveLearningBatchGenerator to avoid memory allocation problems
The current implementation of ActiveLearningBatchGenerator reads all data into memory. This is clearly undesirable for large data sets. To avoid duplication of effort, it would also be beneficial to delegete the batch creation process to the BatchGenerator class, so that ActiveLearningBatchGenerator only handles bookkeeping. Finally, it would be desirable to have a more modular and transparent implemention of some of the class methods, e.g., _get_new()
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information