Problems using the scratch 1TB

[Antoni_Garcia-Molina]

Hi all,

since yesterday I have problems to store my data at the “scratch” 1TB. I clicked in my user preferences to the 1TB options, but it seems that the new outputs are still being stored at my “solid” 250Gb space and therefore all my (large) processes get blocked. Any thought how I could alternatively switch to the 1TB space?

Thanks in advance for your help,

Toni

[jennaj]

Welcome, @Antoni_Garcia-Molina

We have a guide here just as a reference for anyone else reading later on (it sounds like you have seen it!) → What should I do if my data exceeds the given 250GB of storage?

@Antoni_Garcia-Molina

therefore all my (large) processes get blocked

If your jobs are getting paused (light blue in color), that probably means that your are still technically “over quota” in your permanent storage area.

This part of the guide above explains how to check.

@jennaj

Remember: your permanent 250 GB space must be under quota for new jobs to process.

• Rearrange data as needed, and consider processing in batches.
• Not sure where your data is? Review your User → Preferences → Storage Dashboard.
• More about data management strategies .
• More about how jobs process fairly at the public clusters.
• Working at a different server? Create an account at this one too!
• How to move data between servers .

You can move datasets out of the permanent storage and into the temporary storage (“scratch space”) individually. This part of the guide explains.

@jennaj

You can decide where to write and store results (everything in your account, or per history or per workflow) and can move individual datasets between the two spaces directly at any time. View which storage space a dataset is currently stored in, and modify that instantly, on the Job Details page (using the i-info icon ).

Please give that a review and see if it actually is your situation.

Then, if you think something more is going on, I can take a look at your account directly. You do not need to post back your email address or password! Instead, I am going to ask you for your user name (sometimes labeled your public name) for your account at UseGalaxy.org. You can even modify that first to be something generic!

This is how to find (and modify) your public name → FAQ: Changing acount email or password

Please let us know how this works out, and we can certainly follow up more. Thanks!

[jennaj]

Hi @Antoni_Garcia-Molina

It looks like you sorted this out and the new read extraction is going into the 1 TB storage area now. Glad this worked out but please let us know if you need more help!

[Antoni_Garcia-Molina]

Hi Jenny.

I wanted to write to you this morning. My user is angarmo3. I really I have problems to set the default of my story as “scratch” and I have to change the location of the outputs manually file by file (which is a bit annoying). Also when I launch each tool I select the 1TB space, but it does not work. Any thoughts on how could I fix this issue?

Thank you very much for your help.

Best,

Toni

El dv, 4 oct. 2024 a les 18:39 Jennifer Hillman-Jackson via Galaxy Community Help <notifications@galaxy.discoursemail.com> va escriure:

[jennaj]

Hi @Antoni_Garcia-Molina

I just reviewed, and it looks like the new extraction is going into the 1 TB temporary storage space in the history you were running the NCBI SRA extraction in. Am I looking at the wrong history?

@Antoni_Garcia-Molina

I really I have problems to set the default of my story as “scratch” and I have to change the location of the outputs manually file by file (which is a bit annoying).

So, how this works is that you are setting the default for newly created histories and the outputs that write into there. It looked like you were working in an older history this morning.

Try this

1. Set your default storage location for new histories under User → Preferences
2. Create a new history and give it a unique name
3. Check the storage indicator near the top of the history – it should be set for the option you set under preferences.

• If not, you can stop and write back here. Please let me know the history name, and don’t make any other changes to preferences. This is something I would want to to review.
• If yes, run a few more quick tests…

4. Load some new data (Upload tool) into that history, then run a simple tool
5. Click on the i-icon for those test data – the storage location for these should also be correct.

Another small note: if you adjust the storage location at the top of the history to be something different, any new data written into that specific history (and only that history) will write into that location you set. The account User → Preferences setting doesn’t impact this custom choice since it is not a “newly created history”.

Let us know if that works or not, we definitely want to get this working for you. When I try all of this in test accounts, these seems to work but maybe I am missing something you are doing.

[Antoni_Garcia-Molina]

Hi Jennifer,

thanks for your valuable time! It makes using Galaxy worth!

Now it works! I just had to create a new history and adjust the setting to Scratch before starting loading files and executing tools.

All the best,

Toni

Missatge de Jennifer Hillman-Jackson via Galaxy Community Help <notifications@galaxy.discoursemail.com> del dia dv., 4 d’oct. 2024 a les 19:10:

[jennaj]

Great, glad that worked out!