Our OwnCloud server is:

folders.hpc.uidaho.edu - This is an OwnCloud 10.0 server, with some expanded sharing capabilities to other Owncloud servers. This server has 20TB of storage capacity.

You can transfer files to/from Lustre/Ceph and OwnCloud using the WebDAV protocol. The software that manages the mount is Rclone.

First, log into one of our standalone servers. Then, configure rclone

rclone config

Choose Webdav, owncloud, type in your username and password, the URL is: https://folders.hpc.uidaho.edu/remote.php/webdav


benji@seraph ~ * rclone config
2020/06/18 09:49:51 NOTICE: Config file "/mnt/ceph/benji/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> IBEST_owncloud
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / A stackable unification remote, which can appear to merge the contents of several remotes
   \ "union"
 2 / Alias for a existing remote
   \ "alias"
 3 / Amazon Drive
   \ "amazon cloud drive"
 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
   \ "s3"
 5 / Backblaze B2
   \ "b2"
 6 / Box
   \ "box"
 7 / Cache a remote
   \ "cache"
 8 / Dropbox
   \ "dropbox"
 9 / Encrypt/Decrypt a remote
   \ "crypt"
10 / FTP Connection
   \ "ftp"
11 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
12 / Google Drive
   \ "drive"
13 / Hubic
   \ "hubic"
14 / JottaCloud
   \ "jottacloud"
15 / Koofr
   \ "koofr"
16 / Local Disk
   \ "local"
17 / Mega
   \ "mega"
18 / Microsoft Azure Blob Storage
   \ "azureblob"
19 / Microsoft OneDrive
   \ "onedrive"
20 / OpenDrive
   \ "opendrive"
21 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
22 / Pcloud
   \ "pcloud"
23 / QingCloud Object Storage
   \ "qingstor"
24 / SSH/SFTP Connection
   \ "sftp"
25 / Webdav
   \ "webdav"
26 / Yandex Disk
   \ "yandex"
27 / http Connection
   \ "http"
Storage> 25
** See help for webdav backend at: https://rclone.org/webdav/ **

URL of http host to connect to
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Connect to example.com
   \ "https://example.com"
url> https://folders.hpc.uidaho.edu/remote.php/webdav
Name of the Webdav site/service/software you are using
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Nextcloud
   \ "nextcloud"
 2 / Owncloud
   \ "owncloud"
 3 / Sharepoint
   \ "sharepoint"
 4 / Other site/service or software
   \ "other"
vendor> 2
User name
Enter a string value. Press Enter for the default ("").
user> benji
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank
y/g/n> y
Enter the password:
Confirm the password:
Bearer token instead of user/pass (eg a Macaroon)
Enter a string value. Press Enter for the default ("").
Remote config
type = webdav
url = https://folders.hpc.uidaho.edu/remote.php/webdav
vendor = owncloud
user = benji
pass = *** ENCRYPTED ***
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name                 Type
====                 ====
IBEST_owncloud       webdav
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
benji@seraph ~ *

Next, mount the owncloud webdav, using a directory in /tmp/ for file caching (performance will be better if files are cached locally versus in Ceph or Lustre)

benji@seraph ~ * mkdir owncloud
benji@seraph ~ * mkdir /tmp/rclone_cache_benji
benji@seraph ~ * rclone -vv mount --vfs-cache-mode minimal --cache-dir /tmp/rclone_cache_benji  --daemon IBEST_owncloud:/ owncloud
2020/06/18 10:23:38 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "-vv" "mount" "--vfs-cache-mode" "minimal" "--cache-dir" "/tmp/rclone_cache_benji" "--daemon" "IBEST_owncloud:/" "owncloud"]
2020/06/18 10:23:38 DEBUG : Using config file from "/mnt/ceph/benji/.config/rclone/rclone.conf"
2020/06/18 10:23:38 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "-vv" "mount" "--vfs-cache-mode" "minimal" "--cache-dir" "/tmp/rclone_cache_benji" "--daemon" "IBEST_owncloud:/" "owncloud"]

Test the mount

benji@seraph ~ * ls owncloud/
adir-sync  Documents  grc  lfs3-sync  newshare  ownCloud Manual.pdf  Photos  RPMS
benji@seraph ~ *

Important When you’re done - unmount the owncloud directory

benji@seraph ~ * fusermount -u owncloud

You should only mount owncloud on one server at a time since there is some local file caching being done by Rclone.

Cadaver (Depreciated)

Cadever is a CLI client that uses the WebDAV protocol

First, log in and load the grc/3.0 module

module load grc/3.0

Then, connect to files.ibest.uidaho.edu or folders.hpc.uidaho.edu (accept ssl, and log in with normal login/password)

cadaver https://folders.hpc.uidaho.edu/remote.php/webdav/

You can change remote directories using

cd test

To upload file(s), use

mput *.gz

Other useful commands:

  • ls - remote ls on WebDav server
  • cd - change folder on WebDav server
  • mput - multiple put for uploading (can be used with wildcards)
  • mget - multiple get for downloading (can be used with wildcards)

  • lls - local ls on central-storage file system

  • lcd - local change directory
  • lpwd - show current local directory

further information