Create separate folder when only running 1 case in parallel

[lisa.kvalbein@ife.no]

Hi,

I like the feature that creates a separate folder for each case when the "Max Runs"-option in Run Manager is set to 2 or more, but I experience that when this setting is set to 1 max run, the creation of separate folders is not active. Is there any way to activate this feature also for 1 max run? This would help a lot with regards data management and keeping every file related to each case run together, and not being overwritten.

Regards Lisa

[Antti-L]

I am not sure in which situation files would be overwritten where they should not be.  Different cases are distinguished by the different sets of scenario files included, and the run files and the result GDX files have different names anyway for each case (the run name). So, I have not been having any problem with files being overwritten where they shouldn't be, but maybe you can give an example?

Therefore, I find it convenient to use the same model-specific work folder for running different cases of one model, and so I think this option should remain available. However, of course a new option could be added for always creating a new folder, if some users want it that way.

[lisa.kvalbein@ife.no]

The RUN-file and the dd-files for the scenario-files would be overwritten. Different case runs could include different versions of a scenario-file as the scenario-files sometimes evolve during model creation, and so overwriting of dd-files deletes some of the case run-information.

But another related question, in a data management plan, which files for each run is necessary to keep so that the case can be run again without veda and the excel-files?

[AKanudia]

what to do you lose if you keep maxruns=2?

[Antti-L]

But another related question, in a data management plan, which files for each run is necessary to keep so that the case can be run again without veda and the excel-files?

The files for each run that are necessary to keep, so that the case can be run again without VEDA consist of the following:
  • The RUN file
  • The datagdx file (or, alternatively the DD files)
  • Any GDX file used for fixing first periods (if FIXBOH or REG_BDNCAP is used with LPOINT)
  • The GDX containing the Base Prices (if using elastic demands)

I don't see how the RUN file would ever get overwritten by different cases, as each case has a unique run name. But sure, the contents of DD files could evolve over time, and therefore you should use the DATAGDX option to be able to reproduce on old run at a later time. You don't need to save any old versions of DD files.

[lisa.kvalbein@ife.no]

what to do you lose if you keep maxruns=2?

Now I have had several (~100) case runs using a lot of memory (spines-stochastic). As I am monitoring the solution time for each run i wanted to make sure each run get the same amount of processor and memory. In this case i wanted the runs to be done in series and no one in parallel. I am not an expert in processor-usage and do not know how this is handled when having parallel runs, so I did not take any chance.

[lisa.kvalbein@ife.no]

But another related question, in a data management plan, which files for each run is necessary to keep so that the case can be run again without veda and the excel-files?

The files for each run that are necessary to keep, so that the case can be run again without VEDA consist of the following:
  • The RUN file
  • The datagdx file (or, alternatively the DD files)
  • Any GDX file used for fixing first periods (if FIXBOH or REG_BDNCAP is used with LPOINT)
  • The GDX containing the Base Prices (if using elastic demands)

I don't see how the RUN file would ever get overwritten by different cases, as each case has a unique run name. But sure, the contents of DD files could evolve over time, and therefore you should use the DATAGDX option to be able to reproduce on old run at a later time. You don't need to save any old versions of DD files.

Thank you, this is clarifying . The GDX-files is being saved in GamsSave with an unique name, the same with one of the RUN-files. There is another Run-file, named RunFile.run, but I see from the date stamp of the file that it is a static file created with the first run.

I then guess the files being overwritten, like the QA_CHECK.LOG, VTRUN.CMD, END_GAMS is also static files that is the same for all runs and there is not actually any problem with them being overwritten.