goldstift's 2 cent on technology and software development

Navigation

Owncloud Updates between multiple Major Versions

In case you also come into this great situation "I upgraded my Ubuntu to 16.04 LTS and suddenly my owncloud installation is a mess", don’t panic. I’m about to share my mess with you so you can pick the perils.

It all started with Ubuntu 14.04LTS an innocent do-release-upgrade …​ which didn’t want to be aborted - twice. So it convinced to "yes, ok, let’s do it now…​". After the normal upgrade process and a reboot my system was back and working fine - I thought, until I noticed mails from my monitoring system, that the owncloud webUI wasn’t available.

Double checking confirmed "yes, the webUI is not available". And even after the hard part of fixing the nginx configuration - yes, I’m running owncloud behind nginx - and manually installing php7.0-fpm - did I mention Ubuntu 16.04 replaced php5 with php7 EXCEPT FOR THE FPM PART?! -, all websites worked, but Owncloud wouldn’t! Taking a very close look into nginx error.log and I suddenly noticed that the index.php was missing from /var/www/owncloud! In fact, it seems during the Ubuntu Upgrade process for an unreproducible reason all .php files in /var/www/owncloud got deleted! But hey, no worries, let’s apt-get install --reinstall! I double checked that my sources list was correctly pointing to owncloud stable for Ubuntu 16.04 (see https://download.owncloud.org/download/repositories/stable/owncloud/) and apt-get installed owncloud 9.1.3 on my machine without mentioning any problems.

Unfortunately my monitoring system was still unhappy. "Ah, of course, the 'manual upgrade' process", I thought, and ran sudo -u www-data /var/www/owncloud/occ upgrade only to get this wonderful output

ownCloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Exception: Updates between multiple major versions and downgrades are unsupported.
Update failed
Maintenance mode is kept active
Reset log level

Opening the webUI had the same error message for me but additionally included a link to the immensely helpful https://central.owncloud.org/t/updates-between-multiple-major-versions-are-unsupported/815 which pointed out, that I had indeed skipped the 9.0 version!

Ok, let’s cut this short: It occurred to me that the owncloud team has no (documented) support for such a situation - at least I don’t consider https://doc.owncloud.org/server/9.0/admin_manual/maintenance/package_upgrade.html#upgrading-across-skipped-releases helpful at all! I assume your next step was also to downgrade owncloud to the next minor version of your current release and by trying this, you jumped into the next fun situation that the owncloud-server package has a dependency on PHP5. IF SOMEONE CAN TELL ME WHY THIS IS CONSIDERED REASONABLE, I’M REALLY CURIOUS?

Anyway after reading through http://askubuntu.com/questions/762221/ubuntu-16-04-upgrade-killed-owncloud-and-many-other-things-but-right-now-i-n I finally came to this post http://askubuntu.com/questions/767158/ubuntu-16-04-cannot-install-owncloud-server which has a very interesting hint: The owncloud-files package has almost no depencies, is thus easy to downgrade and includes the updater! WOOHOO, JACKPOT!

So, I started by modifying my /etc/apt/sources.list.d/owncloud.list to Ubuntu 8.2

sh -c "echo 'deb http://download.owncloud.org/download/repositories/8.2/Ubuntu_16.10/ /' > /etc/apt/sources.list.d/owncloud.list"

and then forcing the downgrade and step-by-step upgrade by running

$ sudo apt-get update && sudo apt-get install owncloud-files=8.2.9-1.1
(...)
$ sudo -u www-data /var/www/owncloud/occ upgrade
(...)
$ sh -c "echo 'deb http://download.owncloud.org/download/repositories/9.0/Ubuntu_16.10/ /' > /etc/apt/sources.list.d/owncloud.list"
$ sudo apt-get update && sudo apt-get upgrade
(...)
$ sudo -u www-data /var/www/owncloud/occ upgrade
(...)
$ sh -c "echo 'deb http://download.owncloud.org/download/repositories/stable/Ubuntu_16.10/ /' > /etc/apt/sources.list.d/owncloud.list"
$ sudo apt-get update && sudo apt-get upgrade
(...)
$ sudo -u www-data /var/www/owncloud/occ upgrade
(...)

It took a while but worked pretty well. So in the end I’d like to thank John Orion for sharing his idea about owncloud-files and the owncloud team for providing this option - even though you don’t mention it on the pages where people are seeking for help. I hope this is helpful for others, struggling with upgrades.

Now I’m on to fix the remaining issues: Currently I’m facing a situation where all requests with index.php are returned with a HTTP302 redirect without index.php - which of course fails (e.g. /cloud/index.php/login is redirected to /cloud/login). NOTE: I’m running owncloud behind nginx with a contextroot other than /owncloud - as if I was asking for trouble, right? I’ll update this blog, in case I stumble upon something.

If you find anything to comment, optimize or fix about this, please leave me a github issue. Always think "If I have this issue, someone else might sooner or later have it, too" - and you can help that someone. Thanks a lot!

UPDATE: For those who want to follow along, I opened a Github issue under https://github.com/owncloud/core/issues/26869 and provided a pull-request for the remaining necessary changes here https://github.com/owncloud/core/pull/26882

Go to "Owncloud Updates between multiple Major Versions"

Migrate Owncloud from PostgreSQL to MySQL

I am running a self-hosted owncloud instance for more than a year now. Back in August I decided to setup a new Ubuntu 14.04 server at home and tried to run owncloud with a PostgreSQL backend - which turned out to be a bad decision.

Quite often I noticed the following error in my owncloud.log file:

{"app":"index","message":"Doctrine\\DBAL\\DBALException: An exception occurred while executing 'SELECT \"fileid\", \"storage\", \"path\", \"parent\", \"name\", \"mimetype\", \"mimepart\", \"size\", \"mtime\",\n\t\t\t\t\t   \"storage_mtime\", \"encrypted\", \"unencrypted_size\", \"etag\", \"permissions\"\n\t\t\t\tFROM \"oc_filecache\" WHERE \"storage\" = ? AND \"path_hash\" = ?':\n\nSQLSTATE[25P02]: In failed sql transaction: 7 ERROR:  current transaction is aborted, commands ignored until end of transaction block","level":4,"time":"2014-11-26T15:28:16+00:00"}
{"app":"PHP","message":"Cannot modify header information - headers already sent at \/var\/www\/owncloud\/lib\/private\/response.php#83","level":3,"time":"2014-11-26T15:28:16+00:00"}

Later on I found out that it occurred when owncloud tried to scan and index the files in the data folder. The way I got to know this is, I noticed that some of the files I uploaded were not displayed in owncloud but were present on the filesystem. So when I ran

sudo -u www-data /usr/bin/php5 /var/www/owncloud/occ files:scan ahe -v

the command also ended in this error message:

  [Doctrine\DBAL\DBALException]
  An exception occurred while executing 'SELECT "fileid", "storage", "path", "parent", "name", "mimetype", "mimepart", "size", "mtime",
  					   "storage_mtime", "encrypted", "unencrypted_size", "etag", "permissions"
  				FROM "oc_filecache" WHERE "storage" = ? AND "path_hash" = ?':

  SQLSTATE[25P02]: In failed sql transaction: 7 ERROR:  current transaction is aborted, commands ignored until end of transaction block



Exception trace:
 () at /var/www/owncloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:47
 Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /var/www/owncloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:140
 Doctrine\DBAL\Statement->execute() at /var/www/owncloud/lib/private/db/statementwrapper.php:63
 OC_DB_StatementWrapper->execute() at /var/www/owncloud/lib/private/db.php:235
 OC_DB::executeAudited() at /var/www/owncloud/lib/private/files/cache/cache.php:128
 OC\Files\Cache\Cache->get() at /var/www/owncloud/lib/private/files/cache/homecache.php:62
 OC\Files\Cache\HomeCache->get() at /var/www/owncloud/lib/private/files/cache/cache.php:367
 OC\Files\Cache\Cache->remove() at /var/www/owncloud/lib/private/files/cache/scanner.php:163
 OC\Files\Cache\Scanner->removeFromCache() at /var/www/owncloud/lib/private/files/cache/scanner.php:272
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:207
 OC\Files\Cache\Scanner->scan() at /var/www/owncloud/lib/private/files/utils/scanner.php:125
 OC\Files\Utils\Scanner->scan() at /var/www/owncloud/apps/files/command/scan.php:57
 OCA\Files\Command\Scan->scanFiles() at /var/www/owncloud/apps/files/command/scan.php:81
 OCA\Files\Command\Scan->execute() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Command/Command.php:244
 Symfony\Component\Console\Command\Command->run() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:897
 Symfony\Component\Console\Application->doRunCommand() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:191
 Symfony\Component\Console\Application->doRun() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:121
 Symfony\Component\Console\Application->run() at /var/www/owncloud/console.php:43
 require_once() at /var/www/owncloud/occ:11





  [PDOException]
  SQLSTATE[25P02]: In failed sql transaction: 7 ERROR:  current transaction is aborted, commands ignored until end of transaction block



Exception trace:
 () at /var/www/owncloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:138
 PDOStatement->execute() at /var/www/owncloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:138
 Doctrine\DBAL\Statement->execute() at /var/www/owncloud/lib/private/db/statementwrapper.php:63
 OC_DB_StatementWrapper->execute() at /var/www/owncloud/lib/private/db.php:235
 OC_DB::executeAudited() at /var/www/owncloud/lib/private/files/cache/cache.php:128
 OC\Files\Cache\Cache->get() at /var/www/owncloud/lib/private/files/cache/homecache.php:62
 OC\Files\Cache\HomeCache->get() at /var/www/owncloud/lib/private/files/cache/cache.php:367
 OC\Files\Cache\Cache->remove() at /var/www/owncloud/lib/private/files/cache/scanner.php:163
 OC\Files\Cache\Scanner->removeFromCache() at /var/www/owncloud/lib/private/files/cache/scanner.php:272
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:284
 OC\Files\Cache\Scanner->scanChildren() at /var/www/owncloud/lib/private/files/cache/scanner.php:207
 OC\Files\Cache\Scanner->scan() at /var/www/owncloud/lib/private/files/utils/scanner.php:125
 OC\Files\Utils\Scanner->scan() at /var/www/owncloud/apps/files/command/scan.php:57
 OCA\Files\Command\Scan->scanFiles() at /var/www/owncloud/apps/files/command/scan.php:81
 OCA\Files\Command\Scan->execute() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Command/Command.php:244
 Symfony\Component\Console\Command\Command->run() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:897
 Symfony\Component\Console\Application->doRunCommand() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:191
 Symfony\Component\Console\Application->doRun() at /var/www/owncloud/3rdparty/symfony/console/Symfony/Component/Console/Application.php:121
 Symfony\Component\Console\Application->run() at /var/www/owncloud/console.php:43
 require_once() at /var/www/owncloud/occ:11


files:scan [--all] [user_id1] ... [user_idN]

As this is a PostgreSQL error, I searched the owncloud forums for other people’s experience with owncloud + PostgreSQL. Unfortunately it seems that owncloud doesn’t play too nicely with PostgreSQL. :(

Looking back I had an owncloud setup with MySQL before and cannot remember having had any issues with it. So I decided to take the easy way and find out how to migrate my owncloud data from PostgreSQL to MySQL. This was when I stumbled upon this great little command in the owncloud console.php: db:convert-type

This little gem did the trick for me so that after executing the following command, my data had automatically been migrated over to MySQL and the owncloud configuration changed accordingly. As it’s been so nice to me I thought, I leave it here for others coming up with the same question:

sudo -u www-data /usr/bin/php5 /var/www/owncloud/console.php db:convert-type --all-apps mysql owncloud localhost owncloud

Go to "Migrate Owncloud from PostgreSQL to MySQL"

My next Conferences