When a job fails for memory reasons, this is because it was either actually too large to execute on the cluster node or you have an input problem.
Try rerunning at least once. Sometimes two large jobs end up on the same node.
Then check for input problems.
FAQ that explains this exact issue and your options. It links to most other troubleshooting FAQs:
Importantly, the quota space in your account is unrelated to the memory tools use during execution. Meaning: more quota will not resolve issues with jobs failing for resource reasons (memory or runtime aka “walltime”). Running that many samples thorough analysis together will almost certainly fail at any public Galaxy server. If these are distinct analysis that will not be merged in a downstream step, run the data in batches (if a cluster node or input problem was found/fixed and you can get successful results).
Alternatives for large data analysis include setting up your own Galaxy server and allocating enough resources. Galaxy will use about the same resources as the underlying 3rd party wrapped tool(s) do – so those manuals/tool guides, even though command-line usage-based, usually include details about what resources a tool requires (some are linked at the bottom of the tool form, others you’ll need to do an online search for). All of this is somewhat dependent on the input content – and some tools are known to be very memory intensive (hold large portions of input data in memory –
RNA Star is that kind of tool –
HISAT2 can be an alternative).
Some options are straightforward to set up; scientist and teachers (and other non-technical end-users) often choose to use a cloud option for various reasons. Cloudman/AWS, in particular, is utilized by many since the account set up/server administration is mostly web-based. There are other options in the links above –
Jetstream is an academic cloud cluster that is linked to a local Galaxy (free for US-based researchers), but that will involve more administrative work to get set up.
Your dedicated server translates to faster and larger analysis executions, scalable/on-demand resources, and the ability to customize tools/data (though much is pre-configured). AWS offers “Grants in Education” (includes teaching + research work). These are simple to apply for, and often renewable, via their web application process. Grants can help to cover the 3rd party costs. Galaxy itself is always free, but the resources for data storage and computational work are fee-based when using a commercial provider.
Related: Troubleshooting resources for errors or unexpected results