Thanks! I ended up doing something similar to what you suggested. I went into the boxcryptor drive so that the filenames aren't encrypted, copied them, then pasted to another folder I created (now that I don't have filename encryption checked). This created copies that aren't filename encrypted, so I just deleted the old ones and kept the new ones.
I did see the option in the folder to recursively decrypt. Didn't see it initially at first since it seems if at least 1 file in the folder is decrypted already, then the option turns into "enable file name encrypt" on the folder level, even though most of the file names are encrypted already.