files_2_FTP not working after changing ftp dir to new partition

, ,
#1

Hello,

We used files_2_FTP regularly but when changing the FTP directory to a new partition we can not export files anymore.
The FTP server is working correctly as I can upload files to the server and import them into galaxy.
The error that the tool outputs is [User dir on FTP could not be created: </path/to/user/dir>] and this happens because the script does not recognize that the dir actually exists and tries to create it again (os.path.exists(path) returns False for the existing path).
If I take the tool_script.sh from the job_working_dir and run it on the command line within the galaxy virtual environment, the script runs without a problem.

Any idea how to solve this problem?
And also what is the difference between running the tool from the interface or from the command line as the galaxy user?

Thanks!
Cristian

1 Like
#2

Welcome, @Cristian!

The FTP directory in Galaxy is for importing (uploading) data, not for outputting/downloading data (although, using it for downloading data has been discussed).

It sounds like this is a custom tool you developed or are developing. Planemo can help with developing tools that will work, and that can be tested, within the Galaxy architecture. If designed correctly, there should be no difference between running a tool as the Galaxy user line-command versus through the GUI: https://planemo.readthedocs.io/en/latest/writing_standalone.html.

#3

Update: Forgot that this was a community tool from the ToolShed: Galaxy | Tool Shed

Anyone from the community can create/publish tools. But the level of support they offer can vary. This functionality is not built into Galaxy natively (yet, see the link in my original post).

The wrapper was written in 2014 but could probably be adjusted to run in your new configuration (appears to use the default FTP directory location now). Fixing this might involve changes to the scripts and/or configuration files. You’ll need to review the tool. The tool author might be able to help or update the script to be more flexible – try contacting them through the ToolShed (log in and look under “Repository Actions”). Warning: their Github account doesn’t appear to be active and no new contributions have been made to the Galaxy Project for over 2 years.

#4

Hello @jennaj,

We use exporting to the FTP dir so that users can recuperate big files without worrying about downloads getting interrupted due to internet stability.
In any case, I solved my problem! The export is executed on a node that had the NFS mount on an instable state so it could not stat the dir nor copy the files. Unfortunately the only message was that it could not create an existing dir.

The script as is, is working for us, but I will try to improve the error message that gets printed so that I won’t get lost in the future.

Thanks anyway!
Cheers,
Cristian

1 Like
#5

Super, glad it was only a configuration problem, and that it is resolved now.

On our side, I’m going to link in this script to the other development ticket as an example. I’m really hoping we can get this into Galaxy as a built-in function. Very useful for the exact reasons you state :slight_smile: