Hi @cventophd
Interactive Notebooks in Galaxy
How to use GIEs at public Galaxy servers: We recommend running a Galaxy workflow in the main Galaxy application (GUI or command line), then exploring the results for visualization and other custom parsing inside of a dedicated GIE like JupyterLab. You’ll be able to save these to your Galaxy history for your own reuse and for sharing/publication reasons.
Why?
The interactive notebooks in Galaxy are individual container instances dedicated to your account. These are great for the interactive parts of your downstream data exploration steps, and data can be moved to/from your Galaxy history. However, as these are limited to 4 GB of RAM on the public servers, these would be insufficient for all steps, including mapping steps!
Even if these had more memory allocated, any custom ad-hoc workflows would be unlikely to make the most of the UseGalaxy public cluster’s full computational power (as this would bypass the tools, configurations, and supports from Galaxy itself) and potentially cause problems for other users of the systems. The fully independent work then becomes outside of the scope of Galaxy, for at least the processing part. Later, you can always upload your results of course! As an example: groups like New England Biolabs publish data this way for their customers.
Galaxy Workflows on the Command-line
Running Galaxy workflows on the command line is also possible! We fully support command-line interactions on the public servers through the API. We have a utility named Planemo that can test, develop, and run Galaxy workflows for you! Remember that a custom Galaxy workflow could include just a single tool.
This tutorial is a good place to start!
Direct links to related resources
For RNA-seq, we have many resources!
Training workflows
Start with the
Galaxy Training Network (GTN). The first few tutorials include in-depth analysis pathway that includes feature discovery. The next section is a walk-through for a known feature analysis pathway from reads through to visualization.
HTP workflows
For HTP analysis, please see the
IWC Workflow Library! These are polished workflow templates that can crunch through very large sample groups, maximising your use of the computational resources on the public clusters. The usual strategy is to process reads through to a count matrix then to perform DE across different conditions/groups. Once completed, explore the data at a closer, custom level using visualization tools and in notebook environments.
Discover a workflow
All of these workflows in the Galaxy Ecosystem can also be discovered under the Workflows → Public workflows tab at any UseGalaxy server, but finding them through the entry points above provides much more context. You can import any shared workflow and make adjustments to customize them further.
Extract a workflow
You can also extract existing ad-hoc analysis from a history and edit it further. However, to really optimize the throughput potential, consider using one of our templates or at least learning why some of the choices in those were applied. For a quick walk-through, this is a quick and simple example of how this publishing from a history to a workflow can be used.
Questions and support
Questions about any of these workflow or resources are welcome here at the forum!
Your original question came in during a month when most of the team was away and the community was helping out with questions. We are back for the Fall semester now and ready for your questions! Hopefully what I am sharing is enough to get your oriented and we can definitely follow up more. 