When using and object store as primary storage and using the default
encryption module at the same time, any encrypted file would be truncated
when read, and a text error message added to the end.
This was caused by a combination of the reliance of the read functions on
on knowing the unencrypted file size, and a bug in the function which
calculated the unencrypted file size for a given file.
In order to calculate the unencrypted file size, the function would first
skip the header block, then use fseek to skip to the last encrypted block
in the file. Because there was a corresponence between the encrypted and
unencrypted blocks, this would also be the last encrypted block. It would
then read the final block and decrypt it to get the unencrypted length of
the last block. With that, the number of blocks, and the unencrypted block
size, it could calculate the unencrypted file size.
The trouble was that when using an object store, an fread call doesn't
always get you the number of bytes you asked for, even if they are
available. To resolve this I adapted the stream_read_block function from
lib/private/Files/Streams/Encryption.php to work here. This function
wraps the fread call in a loop and repeats until it has the entire set of
bytes that were requested, or there are no more to get.
This fixes the imediate bug, and should (with luck) allow people to get
their encrypted files out of Nextcloud now. (The problem was purely on
the decryption side). In the future it would be nice to do some
refactoring here.
I have tested this with image files ranging from 1kb to 10mb using
Nextcloud version 22.1.0 (the nextcloud:22.1-apache docker image), with
sqlite and a Linode object store as the primary storage.
Signed-off-by: Alan Meeson <alan@carefullycalculated.co.uk>
The current phpdoc of IStorage#file_put_contents doesnt corresponds to
it's actual usage in code, e.g.
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.
While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)
With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.
Signed-off-by: Robin Appelman <robin@icewind.nl>
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.
This also removes and empty lines from method/function bodies at the
beginning and end.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
in order to create a 1:1 copy of a file if a version gets created
we need to store this information on copyBetweenStorage(). This
allows us to by-pass the encryption wrapper if we read the source file.
The code path called when using external storage with WebDAV is using `\OC\Files\Storage\Wrapper\Encryption::getMetaData` which did not contain the actual encrypted version inside the cache entry version. This lead to the following:
1. User uploaded a file
2. File is created and `\OC\Files\Storage\Wrapper\Encryption::getMetaData` is called. It has an empty `encryptedVersion` but sets `encrypted` to either `true` or `false`.
3. The call when updating the file cache will use the old version.
In case of a move operation from an unencrypted to an encrypted
storage the old encrypted version would stay with "0" while the
correct value would be "1". Thus we manually set the value to "1"
for those cases.
See also https://github.com/owncloud/core/issues/23078
When calling `\OC\Files\View::copy` we should also keep the version to ensure that the file will always have the correct version attached and can be successfully decrypted.
To test this the following steps are necessary (from https://github.com/owncloud/core/issues/22781#issuecomment-191328982):
1. setup a new ownCloud 9.0 beta2
2. enable encryption
2. upload a docx (5.7MB large)
3. upload the same file again and overwrite the existing file
4. I can download the original file and the first version
5. I restore the first version
6. restored version can no longer be downloaded with the error described above
The manual cache operation in `\OCA\Files_Versions\Storage` is unfortunately necessary since `\OCA\Files_Versions\Storage::copyFileContents` is not using `\OCP\Files\Storage::moveFromStorage` in the case when an object storage is used. Due to the workaround added in 54cea05271 the stream is directly copied and thus bypassing the FS.
Some move operations when cross-storage will be replaced by copy and
delete. Before attempting this, first check whether the source storage
has delete permissions.
This also prevents renaming system-wide external storages.
alanmeeson
Daniel Kesselberg
acsfer
John Molakvoæ (skjnldsv)
Robin Appelman
Jasper Knockaert
Christoph Wurst
Tigran Mkrtchyan
J0WI
Morris Jobke
Roeland Jago Douma
Florian Schunk
Bjoern Schiessle
Arthur Schiwon
Lukas Reschke
Piotr M
Joas Schilling
Lukas Reschke
Roeland Jago Douma
Björn Schießle
Vincent Petry