Navigating your data storage: existing data, new data, and different storage space options with screenshots!

Welcome @1112

Do you mean that you purged data but your usage quota did not immediately update? That is normal. The spaces is freed up as the data is actually removed from the server storage spaces. For large amounts of data this can take several hours, possibly up to a day.


:scientist: User β†’ Preferences β†’ Storage Dashboard



The Storage Dashboard where to start any account data review. On your page, you can try using the Refresh button as one way to speed the processing up.

You may also be interested in accessing your self-serve extended quota at the public UseGalaxy servers. Your permanent storage will need to be under 250 GB before you can run new jobs, but using your extended scratch space can help to get things organized and avoid problems later on. Guide for how to find and use it. β†’ What should I do if my data exceeds the given 250GB of storage?

How to set your Account level default Storage at UseGalaxy.org. Other servers will look similar but each offers different default and server-hosted resources. Check under the :white_question_mark: at the top to read about what a server offers.




Troubleshooting

For cases where the storage seems to refuse to clear up, one of these is usually going on:

  1. A workflow is running.

As storage space is freed up, it is immediately replaced by the output of jobs that were at first paused, and are now running. This can pause jobs all over again!

Killing the workflow run is how to recover now.

Then, next time you want to run the workflow, configure either your Account level, History level, or Workflow level output destination to write to a different storage space with more room for the outputs.

This is one example of the Workflow level storage choices with screenshots. β†’ Data Storage choices when using Workflows

  1. Multiple copies of datasets

Each copy of a dataset can exist in different places in your account. This is assigned by the Account, History, or Workflow choice, or directly changed on the Details view using the i-icon.



Exact copies of datasets do not consume any extra quota space, but if you still have one copy left, then that data is still in your account! Purging a copy will not change the overall account quota at all.

  • History A contains a copy of sample-3.fastq
  • History B contains a copy of sample-3.fastq
  • Permanently deleting sample-3.fastq from History A still leaves a copy in History B, and the account’s overall quota does not reduce.
  1. Dataset stored in an unexpected location.

Where a dataset is stored is set when it is originally written into your account. Moving it between histories does not change the storage location.

You can review the size of each History and the datasets inside of it on the Activity bar β†’ Histories view.




You can also review the datasets using the database icon at the top of the history. From there, you can review the size of all the data in your account on the History Storage Overview page.

If you need to set a preference for a particular storage location after a dataset is created (or uploaded) you can directly reassign where to keep it!



This per-dataset assignment can be tedious to do if you have many! So, instead, try to get set up at the start of a project. Choose the default storage for your account to a location plenty of room.

Then at the lower levels – for a History or for a Workflow run – make a choice about where that specific output data will write if you want some of your data to be handled differently.



This was a good question and we just updated how the interface views look for some of these choices, so I took the opportunity to post some newer screenshots.

Please let us know if your data quota calculation has resolved or not by now. And, if you need more help please ask. Let’s start there, thanks! :slight_smile: