Compression

Using compress_pickle, we can override pyndb's default save_pickle and load_pickle functions (which are imported from pickle). Here's how it's done:

from compress_pickle import dump, load
import pyndb
pyndb.save_pickle = lambda obj, fn, *args: dump(obj, fn)  # pyndb will try to send HIGHEST_PROTOCOL
pyndb.load_pickle = lambda fn, *args: load(fn)

Now, when opening a file, append the extension of the format you would like to use, for example:

db = pyndb.PYNDatabase("mnist.pyndb.gz", filetype="pickled")

Make sure to specify filetype="pickled" because pyndb does not recognize .gz by default.

To open the file, the same code will work.