What is scratch storage?

usegalaxy.org support
1

I was running out of the storage (250GB) and clicked on the storage option.
I found that I have 1.2TB of scratch storage. What is scratch storage? How can I use for usegalaxy?
Thank you.

1 Like
2

See → What should I do if my data exceeds the given 250GB of storage? - #2 by jennaj

Welcome, @Josh

The function is brand new. We would love feedback about it. Let’s use this topic to gather that from you and others :rocket:

The UseGalaxy.org server (only for now) has a brand new feature that will help when processing larger amounts of data → :sparkles: Scratch Space :sparkles:

Go into your User > Preferences, and click into the Preferred Storage Location for one way to set this up. All new datasets will use this global storage setting.

Scratch space grants 1 TB of temporary short-term storage space to everyone, and you can move data between the original long term and to this new short term space.

The temporary short-term storage is cleared after 30 days. Your original long-term storage of the 250 GB default remains unchanged.

Brand new so no tutorial yet – instead, just notice the little database icons and see the tool tips directly in the app. You’ll figure out how it works :slight_smile:

1 Like
3

Thank you so much for your prompt reply. Even after choosing the preferred storage, the job says that it is paused to the insufficient storage. It looks like the preferred storages doesn’t save the result in the short-term storage. Do you know how to troubleshoot it? Thank you.

4

just in case you need to take a look at the account

5

Hi @Josh

The new rerun looks like it is writing to the short terms space, and is Ok now.

Was the prior done using the “rerun” function? That would be interesting and I might be able to dig into the logs… I wonder if we can change that behavior or not (instead of documenting it). Did you also use anything special in that – rewrite the outputs back into the original collection, etc? I can’t quite see how that was done or missed it so far.

The way this works is: after the preference toggle is made, any new datasets are written to the new space. That means a history can contain datasets in different storage areas. Moving data between the long/short storage will be possible but not totally perfect yet. Probably doesn’t matter for exploration … so just be aware of that.

More tuning for buttons to control where data lives will be coming soon. Right now you’ll see this in two other places that I know of: top of the history panel + on workflow run forms (in the gear icon, same place where “send to new history” is located)

I’ll also be running some more functionality tests. :slight_smile:

Let me know if this specific issue isn’t solved and I misunderstood. And, sharing the history was super helpful, thank you!