CSVLogger (callback) with Create deep learning model

usegalaxy.eu support
,
1

Hi,

I am using the Create deep learning model tool (with an optimizer, loss function and fit parameters) and I would like to get an indication about the convergence of the loss function as a function of the epoch.

I thought that information would come in the CSVLogger but that does not seem to be the case, is this the correct option?

Can someone point me to where I can find the training loss history (and not simply the final value)?

J34ni

2

Welcome, @j34ni

You found a bug! We’ll be reviewing more and fixing this. Please follow the ticket for updates, and we will also get back to you here once done. Thanks for reporting the problem :rocket:

3

Hello @j34ni, thanks for reporting an issue of Galaxy-ML. Your question is a little vague to me. Would you possibly share some example code for us to reproduce the issue, or a galaxy history if the web tool suite is used?

4

Hi @qiagu,

My initial contact was not to report a bug but to ask about how to get an indication about the convergence of the loss function as a function of the epoch.

I thought that this would come with the CSVLogger but did not manage to make it work.

If this is the function I should be using, could you provide an example?

If not, is there a way to do it in Galaxy, and again where can I find an example?

Sorry for the confusion

5

You mean you want to add a custom function to show how fast the loss converges? TTBOMK, the CSVLogger does nothing but saves the training logs to a file. There is little room for customization. Can TensorBoard work for your purposes?
The Galaxy-ML implements a light wrapper on top of Keras APIs, good for some simple DL tasks. If better customization is needed, try tensorflow or maybe Ludwig. FYI, we have Galaxy-Ludwig quite ready.

1 Like
6

@qiagu I only want to get the loss history (i.e., the value of the loss function after each epoch) and I think that this tool does what I need, however I am not sure that it writes this output into a CSV file in the end: whether this callback is selected or not the exact same output files are created.

7

Hi @ j34ni,

I would like to share this workaround to see the loss/accuracy values for each deep learning model training epoch in Galaxy.

  1. Please set verbose=2 in the Create deep learning model with an optimizer, loss function and fit parameters (Galaxy Version 1.0.10.0) tool. This will output accuracy and loss values after each epoch of training (see the following image). The default value of verbosity is 1. Make sure to set it as 2.

create_dl_model
create_dl_model794×166 13.5 KB

  1. For training, one should use Deep learning training and evaluation conduct deep training and evaluation either implicitly or explicitly (Galaxy Version 1.0.10.0) tool in which the dataset created by the tool mentioned in step 1 is used. Once the training is finished successfully, please look into the dataset details of the datasets produced by the training tool. In the tool standard output, you will find the accuracy, loss etc. metrics logged while the model was trained (see the following image).

tool_output_dl_training_eval_tool
tool_output_dl_training_eval_tool1222×693 110 KB

Currently, the CSVLogger feature does not output the logs available in the tool standard output of the deep learning training and evaluation tool as an output file in Galaxy history because no output file is specified in the internal script. We will work on this issue. Thank you for pointing out.

Best,
Anup

1 Like
8

Thanks @anuprulez, I will try that

1 Like