Shocking performance uploading via Boxcryptor - six times slower!

Diewrecked

Hello,

I bought the Unlimited Business version of Boxcryptor so I could use it commercially; going forward if I can get this problem sorted then I will be using Boxcryptor in my generic service offering to all my clients - this could mean a lot of potential new customers and a lot of potential revenue! However the performance I am currently experiencing is a show-stopper - I can't offer BoxCryptor as a solution if I don't get it sorted.

I set a 6Gb upload to my cloud storage (I tried a Google Drive and an S3 bucket one after the other to compare) via a drive letter I had mapped with the Gladinet client. Both came out about two hours, give or take.

I did the same 6Gb upload via my Boxcryptor drive and the performance was shocking - literally six times slower (it's been going for six hours now and it's barely 1/4 of the way). There's no bandwidth throttling, there's no CPU topping out, there's a lot of idle time.

What's going on here and what can be done about it please? Like i said this is a show-stopper, I have clients I want to go in front of with data sets from 600Gb to 6Tb - there's no way I can use Boxcryptor if it's going to make everything take six times longer, can you imagine how long the upload would be with this amount of data?

Please advise how I can get this fixed. There's a lot of headroom both in terms of CPU and bandwidth, to be honest client-side encryption shouldn't affect the upload speed whatsoever it should be completely transparent.

Many thanks

Plerry

I don't know what you mean when you write:
"I did the same 6Gb upload via my Boxcryptor drive ..."
I am not sure you are using the Google drive / Boxcryptor combination properly.
To work via Google Drive, your Boxcryptor drive(s) must be one or more subfolders in your Google Drive.

Obviously, (locally) "moving" or "copying" files to or from the Boxcryptor drive involves encryption respectively decryption, and is substantially slower than a "normal" move or copy.
For exactly the same reasons, (locally) accessing files in the Boxcryptor drive from applications can be noticeably slower than when accessing unencrypted files.

However, when synchronizing, Google Drive will simply synchronize the content of all its subfolders “as is”.
If one or more of those subfolders actually is a Boxcryptor drive, the content of those subfolders will be synchronized in its encrypted form, without any involvement of Boxcryptor.
Boxcryptor encrypts at file level and does not increase file-size. When using the Boxcryptor-option of file-name encryption, file-names will become about 1.5 times longer, but this will only have a minimal effect on synchronization-size and thus time.

I don’t know about S3, but at least for Google Drive, when used properly, having part of your Google Drive content consisting of Boxcryptor drives should (and from experience: does) not affect the Google Drive synchronization times.

Plerry

Diewrecked

Hi Plerry

Thanks for coming back so quickly.

To confirm, your usage model ("your Boxcryptor drive(s) must be one or more subfolders in your Google Drive") is exactly what I'm doing. I'm not using the Google Drive app per se but it is going into a Google Drive.

I'm using the Boxcryptor application to map an encrypted Boxcryptor folder in my Google Drive to a drive letter, as per standard Boxcryptor model. The difference between the "standard" way of doing it and what I'm doing is that instead of using the Google Drive app, which is a synchronisation app, I'm using Gladinet Client, which is a "file storage" app - it allows you to use Google Drive (and S3 and various other cloud providers) as a standard mapped drive rather than a folder that is being continually synchronised.

However, the fact remains that when I upload 6Gb via the Gladinet drive mapping direct to the Google Drive it takes about two hours and when I do the same upload via the BoxCryptor drive mapping (which points to an encrypted folder on the same Google Drive) it's six times slower.

I'm not convinced about your comments regarding encryption/decryption causing upload/download to be "noticeably slower". I've been using such as SafeBoot, McAfee Endpoint Encryption, Truecrypt etc for years and seen no noticeable performance overhead whatsoever. Indeed, if it was about the encryption/decryption on the fly and the amount of processing it takes, why is there so little CPU overhead? That's one of the things I observed in my original post.

Really hope we can get this sorted out! What sort of performance are you seeing when you use it?

Thanks

FrankGfromW

@Diewrecked: Encrypted data is usually hard to compress (takes more cpu and gives much less compression than clear text). Just my guess: Your upload does compress data, at least during transport. This might partially explain the slower speed, although I would expect 6x only for easily compressible data like txt files.

Diewrecked

Hi Frank - thanks for coming back. I understand what you're saying but there is a lot of idle time in the upload and there is a lot of CPU idle time too - I don't understand how any overhead, be it bandwidth or CPU processing, could cause a 600% increase in upload time when it's not being maxed out.

Robert

Just a short notice from my side (as I haven't used Gladinet yet):

Using BoxCryptor increases the number of file operations (like get file information, open a file, get directory listing, etc.). For example a task which would involve "normally" 5 file operations in plaintext may require 10 (or even more) file operations when performed through BoxCryptor. One reason for this is filename encryption where we need different access patterns.

This is not a problem at all if the BoxCryptor Folder is located on your local hard disk because access and response times are so fast (1-50 ms) that it makes no difference if 5 file operations are executed or 20.

However, if the BoxCryptor Folder is located on a remote network location, access times are greatly higher (> 100ms) and also the overhead to execute a file operation is a lot bigger. In this case additional file operations can greatly reduce the performance.

That's why we recommend to use a cloud storage provider with a true syncing process where the BoxCryptor Folder can be located on a local hard disk - as most of our users do. I assume this is not the case when you use Gladinet (which then stores the data in Google Drive). It would be better to directly use the Google Drive sync client (at least for the first data import).

I don't think that encryption causes your slower performance (we have ~ 40-50 MB/sec raw encryption speed on a Core i5/7 processor). I think it's rather the increase in the number of file operations in combination with the high latency over network which causes this performance troubles - so the bottleneck is not your cpu or bandwith, but actually the latency.

Robert

PS: Encryption could be a reason for slower transfer speeds if Gladinet uses compression before the transfer (which many are doing) and you transfer data with a good compression ratio (e.g. text files). Ciphertext is almost "uncompressable" due to its random nature, so if the plaintext could have a 50% compression ratio, transfer times would double for ciphertext. But I don't think this is the reason in your case.

Diewrecked

Hi Robert

Thanks for coming back.

You're right regarding performance of a local Boxcryptor folder. I created a new folder on the NAS on my LAN and C&P'd my 6Gb data into it - it took about 20 minutes to encrypt the whole lot on the fly including filenames. Then I C&P'd the folder into my Google Drive via Gladinet and it was pretty much the same time as the unencrypted files, about three hours.

So we know the performance is there and we know it works.

The problem I now have is that I want to use the Boxcryptor folder as primary storage in cloud, be it Google Drive, Amazon S3, whatever (Gladinet supports them all transparently) so that people in the office can all use it the same as they would a local file server, but equally people remote on either laptops or iPads can also all use it remotely on their respective devices, with collaboration in real time (like with MS Office 365 etc). Equally I want one or more offsite and disparate backups available (one in the office, one in the CEO's house etc) which would be created by synching in the Gladinet client to some local storage path.

How can I do this?

The perfect solution would be of course to do the encrypting locally, ie, you download the file onto the box when it's opened, and when it's saved it's encrypted before it's saved then uploaded back to cloud storage. That would effectively do what my two-step process (encrypt first then upload) did in one, yes? Is that workable?

I'm so close with Boxcryptor I can taste it - I just need to get this performance issue sorted out!

Really hope you can help with the implementation here!

Cheers

Plerry

Perhaps not the proper question to ask in the BoxCryptor forum, but nevertheless:

If you insist on using Gladinet, why not use the data-encryption of Gladinet?
It is stated to offer "AES-256 bit Data At Rest Encryption".
I am not sure what this means exactly in case of Gladinet, but it might be adequate for you.

Plerry