Welcome, @swagat23
Large Scale Data Analysis at Public Galaxy servers
The UseGalaxy public Galaxy servers offer accessible, scaled, computational resources for everyone!
When working at UseGalaxy.org
To set where you are storing data, go to the User > Preferences view and review the selection options for 250 GB + 1 TB under Storage Location.
Some ideas about how to use the extra 1 TB scratch-space.
- Load raw sequence files into the temporary 1 TB.
- Write intermediate files to the temporary 1 TB. … experiment with parameters… polish your workflow… then rerun it with new data.
- Write final results and associated graphics into the permanent 250 GB.
- Export your history to an archive file and store that anywhere, or transfer it to a different Galaxy server, then load it up again later for full context preservation. You can even uncompress that archived bundle and access your files directly on your own computer.
- Create a workflow for processing, or extract a workflow from your direct work to create a pipeline → raw data + workflow == publication quality reproducibility. Share and remember what you did with the summary Workflow Reports.
- You can decide where to write and store results (everything in your account, or per history or per workflow) and can move individual datasets between the two spaces directly at any time. View which storage space a dataset is currently stored in, and modify that instantly, on the Job Details page (using the i-info icon).
Remember: your permanent 250 GB space must be under quota for new jobs to process.
- Rearrange data as needed, and consider processing in batches.
- Not sure where your data is? Review your User → Preferences → Storage Dashboard.
- More about data management strategies.
- More about how jobs process fairly at the public clusters.
- Working at a different server? Create an account at this one too!
- How to move data between servers.
When working at UseGalaxy.eu or UseGalaxy.org.au
Please find the temporary extended quota request form linked on the server homepages in a box labeled Our Data Policy.
Not sure how to find the server homepage? Click on the Galaxy icon in the very top left. Once that loads, scroll down a bit to find →
When working at other Galaxy servers
The contact for the administrators is usually in your original account activation email, or the server homepage, or in their directory listing here Public Galaxy Servers.
If you can’t find the contact, please ask a question at this forum. Include the server URL with your question and we can try to help more.
Production pipelines
The public resources are considerable yet some researchers will still need to consider a Private Galaxy Servers option. And many will use both, together, for publication or other data sharing reasons.
.
Hope this helps!