Veda2.0 Released!


Tradeoff for user-defined objective function
#1
Dear VEDA community

I am trying to use the tradeoff-function (based on this documentation) to implement a user-defined objective function. I adopted the example provided in this forum thread for my needs.

While my model produces results and the tradeoff seems generally to work, not everything is 100% clear to me and some things work differently than I expected. I cannot attach my testmodel as it is too large (2.6 MB), thus I uploaded it here: https://we.tl/t-5WLxcXrYzx

What do I intend to do?
I would like to produce 3 simulations (see SuppXLS file Scen_05_MultiCriteria_Optimization)
  1. First, a scenario that optimizes the costs. 

  2. Second, a scenario that optimizes the time consumption with a certain maximum deviation of the costs (10%) compared to scenario 1.

  3. Third, a scenario that optimizes the costs without changing the time consumption from scenario 2 (deviation = 0%) in order to produce meaningful energy prices and physical results.



Questions:
  1. In the .lst file, I see that 6 simulations are calculated. Also in VFE, I get after the simulation 6 lines in the pop-up window that reports the simulation result and objective function. I understand that the simulations 1 to 3 optimize costs, time, and again costs without taking into account the maximum deviation bound that I have defined. Subsequently, the simulations 4 to 6 re-optimizes the results from 1-3, respectively (4 based on 1, 5 based on 2, and 6 based on 3), with taking into account the maximum deviation bound. Also, VBE reports only the results from simulations 4 to 6. Do I understand this correctly?

  2. In the .lst file, I see that...
    ...in the 4th simulation, the lower and upper bound for OBJ1 are enforced according to the bound that OBJZ can deviate by 10% up and down (I believe, in relation to simulation 1)
    ...in the 5th simulation, the lower and upper bound for OBJ1 are enforced according to the bound that OBJ_TIME must remain constant (I believe, in relation to simulation 2)
    ...in the 6th simulation, I intend the same value for OBJ_TIME like in simulation 5 (due to S_UC_RHS(OBJ1,N,2,2)=0) and OBJZ would be optimizated under that condition (to produce meaningful energy prices and physical results). However, the model optimizes OBJZ and there are no bounds for OBJ_TIME. How can I adjust the model, so it optimizes in the end OBJZ but keeps the OBJ_TIME constant from the simulation 5?

  3. I don't fully understand why the maximum deviations defined via S_UC_RHS  have to be defined for Stage 2 (adopted from the example)...? From the figures in the documentation, I understand that the Stages distinguish the time periods in which different SOWs are present. Why is it in my example not sufficient to use 1 stage across the entire time horizon with 3 different SOWs (as I understand, each SOW reflects on optimization "scenario", as outlined above).



Thank you in advance for your help.

Kind regards,
Sandro
Reply
#2
I had a look at your Scen_05_MultiCriteria_Optimization.xlsx, and it seem very similar to the example you linked:  A multiphase set-up with three steps (SOWs). Your description (What do I intend to do) also suggests that a multiphase set-up is intended. I just tested again the earlier multiphase example, and verified the three steps being solved: 

  1) first use the standard objective function for a Baseline
  2) then minimize GHG emissions subject to a limit OBJ increase compared to Baseline
  3) finally, minimize the costs again, by imposing the emission level obtained in step 2

So, I don't see why you would be seeing the model being solved 6 times (6 "simulations"), if that is what you mean. That would imply that the set-up is for some reason not being recognized as a multiphase tradeoff run, even though you seem to be defining S_UCOBJ('OBJ1','1')=1;

Anyway, as far as I can see, the earlier example (described above) works as expected, and the solution is reported for SOW=1,2,3 (as verified from the vd file).

As said, I am not able to spot any explanation to your findings from the Excel file, but if you like, I would be willing to look at this model case if you can provide me with the input files (the *.DD and *.RUN files plus the listing file *.LST), so that I can reproduce your findings, e.g. about six solves being executed.
Reply
#3
Thanks for sharing your thoughts and confirming how I understood my setup.

Indeed, I find in VBE that the model regonizes 3 SOWs.
Attached you find a screenshot of the 6 solutions shown in the "GAMS Status" pop-up windows of VFE:

Also, you find attached the .dd, .RUN, and .lst Files.

I appreciate your thoughts and look forward to hearing from you.


Attached Files Thumbnail(s)
   

.zip   Gams_Wrk_MultiCriteria_Forum.zip (Size: 82.25 KB / Downloads: 3)
Reply
#4
Ok, thanks.  I was able to reproduce the six solves that you reported to be executed.

The explanation for that is the following: The RUN file revealed that you are not using a multiphase set-up after all, because it had the following controls:

$ SET SENSIS YES
$ SET STAGES YES

As described in the documentation, multiphase requires using the stochastic mode (i.e. $ SET STAGES YES), but the SENSIS setting overrides it, and so multiphase cannot be used. Your set-up therefore corresponds to the Case C in the documentation.  The documentation does clearly state this prerequisite of using the multiphase set-up; quote: "The set-ups D and E described in the introduction are only possible with the setting STAGES YES, i.e. without the setting $SET SENSIS YES ".

Anyway, I tested your model also with multiphase (by removing the SENSIS setting), and only three solves were executed, as expected. I would think that such a multiphase set-up would suit with what you intended, and so maybe you could try using that?
Reply
#5
Sorry about failing to address explicitly your questions in the above. Please find some attempts below:

> 1. Do I understand this correctly?

Yes, for the set-up C that you were using, I'd say your understanding is correct.

> 2. How can I adjust the model, so it optimizes in the end OBJZ but keeps the OBJ_TIME constant from the simulation 5?

That is not possible. In the set-up C you were using, in phase 2 you can define deviation bounds for the OBJ or any UC constraints only in relation to the same SOW in phase 1. Your "simulation 5" must be referring to the second SOW in phase 2, and so you cannot refer to SOW=2 in the end (where SOW=3). However, in the multiphase set-up that would be possible.

> 3. I don't fully understand why the maximum deviations defined via S_UC_RHS  have to be defined for Stage 2 (adopted from the example)...?

It is simply because by definition, there is only a single SOW=1 in stage 1. For stage 2, you are defining 3 SOWs by specifying SW_SUBS.  Therefore, you need to specify '2' as the stage when referring to any of these SOW<>1. The stage 2 covers the full model horizon under sensitivity / tradeoff analysis. Sure, I understand that the stage index may seem superfluous here, but it has been designed this way to be able to make use of the same attributes and code routines both for multistage stochastics and sensitivity / tradeoff analysis.
Reply
#6
Dear Antti

Thanks for investigating the situation and providing such a clear problem analysis, solution, and explanations.
I tested the model with removed SENSIS setting, and now it seems to work as intended.

Thanks again for your help and kind regards,
Sandro
Reply
#7
Dear Antti

May I ask you one follow-up question?

In the results, I find for the different SOWs the Objective Function values as shown in this screenshot: https://we.tl/t-kIuHzzAQ9v

In SOW=1, the model optimizes for costs, i.e. OBJZ is optimized. OBJZ is 5720. However, the OBJ1 value in SOW=1 is 6292. I would have expected that OBJ1 should then also be 5720, because in SOW=1 OBJ1 is defined as OBJZ
Could you please explain this deviation between OBJ1 and OBJZ in SOW=1?
Reply
#8
In the multiphase set-up, for every SOW, VAS_UC(OBJ1,SOW) represents the objective function value in that phase.
And therefore, in the first phase (SOW=1), the value of VAS_UC(OBJ1,1) will indeed be the value of the standard objective function, which is 5720.44285348534.

However, in the second phase you define a deviation bound for the objective function used in the first phase. Therefore, the  value of VAS_UC(OBJ1,1) will have the upper bound of 5720.44285348534 × 1.1 = 6292.48713883.  And, it turns out that minimizing the OBJ_TIME in the second phase causes VAS_UC(OBJ1,1) to be at its upper bound.

In the third phase, your model minimizes the value of the standard objective function again, but the upper bound is still in force (as described in the doc), while the deviation bound for OBJ_TIME is now set to zero.  These constraints cause the value of VAS_UC(OBJ1,1) to remain at its upper bound.

Consequently, after solving phase 3, VAS_UC(OBJ1,1) has the value 6292.48713883 (the upper bound).

Nonetheless, the value of the standard OBJZ in each phase is not at all lost, but you can get it at least from the following reporting attributes:
  • Reg_obj(reg,sow)  –  objective by region
  • User_con(OBJZ,sow) –  total objective
Reply
#9
(25-07-2022, 11:08 PM)Antti-L Wrote: In the multiphase set-up, for every SOW, VAS_UC(OBJ1,SOW) represents the objective function value in that phase.
And therefore, in the first phase (SOW=1), the value of VAS_UC(OBJ1,1) will indeed be the value of the standard objective function, which is 5720.44285348534.

Dear Antti

Thanks for outlining this. I agree and can follow your statements. However, I feel my concrete question is not answered - or I just can't identify the answer to this specific issue in your previous reply. Let me rephrase:

In the quoted statement, you say that in phase 1 the VAS_UC(OBJ1,1) is equal to the standard objective (OBJZ): 5720. In principle, that is also my understanding. However, what remains unclear to me is why I see in VEDA-BE that OBJ1 in SOW=1 is 6292 instead of 5720 (see screenshot: https://we.tl/t-kIuHzzAQ9v).

If your previous answer contained an (implicit) answer to this, please forgive that it does not appear obvious to me.

Thanks in advance for your efforts,
Sandro
Reply
#10
Hmm...  I think there was an explicit answer:

AL> Consequently, after solving phase 3, VAS_UC(OBJ1,1) has the value 6292.48713883.

Because the value of VAS_UC(OBJ1,1) is 6292.48713883 after all three phases are completed, it is also reported to have that value. That is the value for VAS_UC(OBJ1,sow) with sow=1 (the second index of VAS_UC(OBJ1,1) is the SOW index). And the User_con reporting attribute shown in your screenshot reports exactly those values: VAS_UC(OBJ1,sow), for sow=1,2,3.

Your screenshot does NOT show any OBJ1 in SOW=1 to be 6292. OBJ1 is a member of the set UC_N, and it has no value.  The value 6292.48713883 is shown for the attribute User_con(sow,uc_n,r,t,ts), which reports the levels of the variables VAS_UC(uc_n,sow) (among others). And because the variable VAS_UC(OBJ1,1) has that value as its level, it is being reported so.

Does that make it any clearer?
Reply
#11
Antti, thanks for providing such additional insights. It's making the background more clear.
Thanks for always being very helpful!
Reply
#12
> It's making the background more clear.

Well thanks, but I hoped that the "foreground" would also have been more clear. The levels of the these variables reported by TIMES are obtained directly from the solution provided by the solver, and correspond to the optimal solution. The value you expected to see would thus be deviating considerably from that solution. Blush

TIMES does calculate some variable levels also by itself, but basically only for the so-called reduced variables, according to the variable substitutions TIMES has made. I think it is good practice that the reported levels are consistent with the optimal solution.
Reply
#13
(22-07-2022, 05:22 PM)Antti-L Wrote: Ok, thanks.  I was able to reproduce the six solves that you reported to be executed.

The explanation for that is the following: The RUN file revealed that you are not using a multiphase set-up after all, because it had the following controls:

$ SET SENSIS YES
$ SET STAGES YES

As described in the documentation, multiphase requires using the stochastic mode (i.e. $ SET STAGES YES), but the SENSIS setting overrides it, and so multiphase cannot be used. Your set-up therefore corresponds to the Case C in the documentation.  The documentation does clearly state this prerequisite of using the multiphase set-up; quote: "The set-ups D and E described in the introduction are only possible with the setting STAGES YES, i.e. without the setting $SET SENSIS YES ".

Anyway, I tested your model also with multiphase (by removing the SENSIS setting), and only three solves were executed, as expected. I would think that such a multiphase set-up would suit with what you intended, and so maybe you could try using that?

Dear Antti

I want to follow up on our previous discussion about activating the tradeoff function. Thanks to your valuable remarks, executing the tradeoff function in my simple test model worked well by deactivating $SET SENSIS YES  and activating $SET STAGES YES .

Now, I want to apply the tradeoff function in a more complex model. However, once I activate $SET STAGES YES , I receive an "Unexpected Termination of Run"  and several error messages in the .lst file. This is even though I use the same TIMES Source code as in the test model. I am unsure how to fix the errors outlined in the .lst file. Attached you will find the .RUN and .lst file of the simulation that leads to the error messages.

Do you know what is going wrong and how I can fix this issue? I would highly appreciate your insights!

Kind regards,
Sandro


Attached Files
.zip   TradeOff_Error_Messages.zip (Size: 13.69 KB / Downloads: 1)
Reply
#14
Looks like you are using an old TIMES v4.4.1, which is also extended with some additional PSI module(s), which are probably in some way not compatible with stochastic TIMES. Therefore, the code is perhaps not quite compatible with the ETSAP version of TIMES.

Anyway, first of all, the TIMES version is basically already too old for technical support (it is from February 2019). But if you would like me to tell you more clearly what is going wrong, please provide me with:

   ● The version of TIMES code you are using (the full folder GAMS_SRCTIMESV441_MODALSHIFT)
   ● The full set of model input files *.DD and *.RUN for the model case
Reply
#15
Actually, I found in my archives some PSI GAMS code *.GRD, which I have received earlier from Evangelos Panos.  Indeed, it looks like that code is not fully compatible with ETSAP TIMES.

If using such GAMS code of your own development as a TIMES extension, I think you should be able to provide your own technical support for it. Maybe you could contact the developers of that code at PSI?
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)