Using symlinks as user_library_import_dir with network drives not working

, ,
1

Hi all,

we are running our own local galaxy service in a multi-user environment at version 21.09 currently.

We want to use the following setup to allow user import of data:

a mounted network drive at:

/galaxy-clients/user1/<data1..X>

our galaxy is installed at /.../galaxy/ and config/galaxy.yml has set:

user_library_import_dir: /.../galaxy/database/files/ 
user_library_import_symlink_allowlist: /galaxy-clients/user1/
user_library_import_dir_auto_creation: false

and inside /.../galaxy/database/files/ we created a symlink to the mounted network share:

user1@abc.com -> /galaxy-clients/user1

now if we try to upload data to a data library from user directory for user1 we receive the error:

The specified directory does not exist [/.../galaxy/database/files//user1@abc.com.].

The mounted drive is accessible by the user thats running the galaxy process, following the symlink to the mount is working fine.

The exact same setup with only change of a symlink email-address pointing to a directory simply on the machine instead of a mounted share is working fine.

The same setup using an older galaxy instance of version 19.01 is working fine.

We are quite at a loss why its not working, maybe someone of you has encountered a similar problem?

2

Hi @galaxy_user

I’ve asked our administrative group for help at their chat. They may reply here or there, and feel free to join the chat in either of these:

You might have seen these resources already, but for reference:

Let’s start there :slight_smile:

3

I’ve recreated pretty much your exact setup locally and I’m unable to reproduce the error. Is /.../ just a collapsing of a real path? If so, can you verify that there are no typos in there? If not, I actually tested with that as a path and could not reproduce the issue:

$ tree /...
/...
└── galaxy
    └── database
        └── files
            └── nate+test@bx.psu.edu -> /galaxy-clients/nate-test

4 directories, 0 files

$ tree /galaxy-clients
/galaxy-clients
└── nate-test
    └── 1.fastqsanger

1 directory, 1 file

galaxy.yml options:

user_library_import_dir: /.../galaxy/database/files/
user_library_import_symlink_allowlist: /galaxy-clients/nate-test/
user_library_import_dir_auto_creation: false

Importing as my nate+test@bx.psu.edu user works in this scenario using the latest 21.09 (72bfa2b76db1aae57e5814e8f10d9755fa81f795).

Unrelated to the problem, but if /.../galaxy/database/files is really the database/files directory in your Galaxy server clone, this is not a good path to use for this purpose. Although newer Galaxy servers store datasets in database/objects, the files dir is the default for older installations. The user_library_import_dir can be anywhere (especially since you’re just symlinking from it), preferably outside the Galaxy clone entirely.

1 Like
4

Hi, thank you for looking into it!

Yes the /…/ is just a collapsed normal path.

I have moved the user import dir to just /data/galaxy_upload and the galaxy installation at /srv/galaxy and unfortunately also with a symlink in /data/galaxy_upload/user1@abc.com -> /galaxy-clients/user1 i get the same error.

I tried mounting something else to make sure its not just this one specific mount and if I point the symlink to /mnt/home/tux without having it in the allow_list i get the apropriate error directory (/data/galaxy_upload/user1@abc.com/.) is a symlink to a location not on the allowlist

After adding it to the allow_list i no longer receive an error and can import data from the symlink-ed mountpoint.

So in conclusion I think it must have something to do with how the mountpoint at /galaxy-clients is set up on our system, if other mountpoints work just fine.