Veda2.0 Released!

Tradeoff for user-defined objective function
Dear VEDA community

I am trying to use the tradeoff-function (based on this documentation) to implement a user-defined objective function. I adopted the example provided in this forum thread for my needs.

While my model produces results and the tradeoff seems generally to work, not everything is 100% clear to me and some things work differently than I expected. I cannot attach my testmodel as it is too large (2.6 MB), thus I uploaded it here:

What do I intend to do?
I would like to produce 3 simulations (see SuppXLS file Scen_05_MultiCriteria_Optimization)
  1. First, a scenario that optimizes the costs. 

  2. Second, a scenario that optimizes the time consumption with a certain maximum deviation of the costs (10%) compared to scenario 1.

  3. Third, a scenario that optimizes the costs without changing the time consumption from scenario 2 (deviation = 0%) in order to produce meaningful energy prices and physical results.

  1. In the .lst file, I see that 6 simulations are calculated. Also in VFE, I get after the simulation 6 lines in the pop-up window that reports the simulation result and objective function. I understand that the simulations 1 to 3 optimize costs, time, and again costs without taking into account the maximum deviation bound that I have defined. Subsequently, the simulations 4 to 6 re-optimizes the results from 1-3, respectively (4 based on 1, 5 based on 2, and 6 based on 3), with taking into account the maximum deviation bound. Also, VBE reports only the results from simulations 4 to 6. Do I understand this correctly?

  2. In the .lst file, I see that... the 4th simulation, the lower and upper bound for OBJ1 are enforced according to the bound that OBJZ can deviate by 10% up and down (I believe, in relation to simulation 1) the 5th simulation, the lower and upper bound for OBJ1 are enforced according to the bound that OBJ_TIME must remain constant (I believe, in relation to simulation 2) the 6th simulation, I intend the same value for OBJ_TIME like in simulation 5 (due to S_UC_RHS(OBJ1,N,2,2)=0) and OBJZ would be optimizated under that condition (to produce meaningful energy prices and physical results). However, the model optimizes OBJZ and there are no bounds for OBJ_TIME. How can I adjust the model, so it optimizes in the end OBJZ but keeps the OBJ_TIME constant from the simulation 5?

  3. I don't fully understand why the maximum deviations defined via S_UC_RHS  have to be defined for Stage 2 (adopted from the example)...? From the figures in the documentation, I understand that the Stages distinguish the time periods in which different SOWs are present. Why is it in my example not sufficient to use 1 stage across the entire time horizon with 3 different SOWs (as I understand, each SOW reflects on optimization "scenario", as outlined above).

Thank you in advance for your help.

Kind regards,
I had a look at your Scen_05_MultiCriteria_Optimization.xlsx, and it seem very similar to the example you linked:  A multiphase set-up with three steps (SOWs). Your description (What do I intend to do) also suggests that a multiphase set-up is intended. I just tested again the earlier multiphase example, and verified the three steps being solved: 

  1) first use the standard objective function for a Baseline
  2) then minimize GHG emissions subject to a limit OBJ increase compared to Baseline
  3) finally, minimize the costs again, by imposing the emission level obtained in step 2

So, I don't see why you would be seeing the model being solved 6 times (6 "simulations"), if that is what you mean. That would imply that the set-up is for some reason not being recognized as a multiphase tradeoff run, even though you seem to be defining S_UCOBJ('OBJ1','1')=1;

Anyway, as far as I can see, the earlier example (described above) works as expected, and the solution is reported for SOW=1,2,3 (as verified from the vd file).

As said, I am not able to spot any explanation to your findings from the Excel file, but if you like, I would be willing to look at this model case if you can provide me with the input files (the *.DD and *.RUN files plus the listing file *.LST), so that I can reproduce your findings, e.g. about six solves being executed.
Thanks for sharing your thoughts and confirming how I understood my setup.

Indeed, I find in VBE that the model regonizes 3 SOWs.
Attached you find a screenshot of the 6 solutions shown in the "GAMS Status" pop-up windows of VFE:

Also, you find attached the .dd, .RUN, and .lst Files.

I appreciate your thoughts and look forward to hearing from you.

Attached Files Thumbnail(s)

.zip (Size: 82.25 KB / Downloads: 2)
Ok, thanks.  I was able to reproduce the six solves that you reported to be executed.

The explanation for that is the following: The RUN file revealed that you are not using a multiphase set-up after all, because it had the following controls:


As described in the documentation, multiphase requires using the stochastic mode (i.e. $ SET STAGES YES), but the SENSIS setting overrides it, and so multiphase cannot be used. Your set-up therefore corresponds to the Case C in the documentation.  The documentation does clearly state this prerequisite of using the multiphase set-up; quote: "The set-ups D and E described in the introduction are only possible with the setting STAGES YES, i.e. without the setting $SET SENSIS YES ".

Anyway, I tested your model also with multiphase (by removing the SENSIS setting), and only three solves were executed, as expected. I would think that such a multiphase set-up would suit with what you intended, and so maybe you could try using that?
Sorry about failing to address explicitly your questions in the above. Please find some attempts below:

> 1. Do I understand this correctly?

Yes, for the set-up C that you were using, I'd say your understanding is correct.

> 2. How can I adjust the model, so it optimizes in the end OBJZ but keeps the OBJ_TIME constant from the simulation 5?

That is not possible. In the set-up C you were using, in phase 2 you can define deviation bounds for the OBJ or any UC constraints only in relation to the same SOW in phase 1. Your "simulation 5" must be referring to the second SOW in phase 2, and so you cannot refer to SOW=2 in the end (where SOW=3). However, in the multiphase set-up that would be possible.

> 3. I don't fully understand why the maximum deviations defined via S_UC_RHS  have to be defined for Stage 2 (adopted from the example)...?

It is simply because by definition, there is only a single SOW=1 in stage 1. For stage 2, you are defining 3 SOWs by specifying SW_SUBS.  Therefore, you need to specify '2' as the stage when referring to any of these SOW<>1. The stage 2 covers the full model horizon under sensitivity / tradeoff analysis. Sure, I understand that the stage index may seem superfluous here, but it has been designed this way to be able to make use of the same attributes and code routines both for multistage stochastics and sensitivity / tradeoff analysis.
Dear Antti

Thanks for investigating the situation and providing such a clear problem analysis, solution, and explanations.
I tested the model with removed SENSIS setting, and now it seems to work as intended.

Thanks again for your help and kind regards,
Dear Antti

May I ask you one follow-up question?

In the results, I find for the different SOWs the Objective Function values as shown in this screenshot:

In SOW=1, the model optimizes for costs, i.e. OBJZ is optimized. OBJZ is 5720. However, the OBJ1 value in SOW=1 is 6292. I would have expected that OBJ1 should then also be 5720, because in SOW=1 OBJ1 is defined as OBJZ
Could you please explain this deviation between OBJ1 and OBJZ in SOW=1?
In the multiphase set-up, for every SOW, VAS_UC(OBJ1,SOW) represents the objective function value in that phase.
And therefore, in the first phase (SOW=1), the value of VAS_UC(OBJ1,1) will indeed be the value of the standard objective function, which is 5720.44285348534.

However, in the second phase you define a deviation bound for the objective function used in the first phase. Therefore, the  value of VAS_UC(OBJ1,1) will have the upper bound of 5720.44285348534 × 1.1 = 6292.48713883.  And, it turns out that minimizing the OBJ_TIME in the second phase causes VAS_UC(OBJ1,1) to be at its upper bound.

In the third phase, your model minimizes the value of the standard objective function again, but the upper bound is still in force (as described in the doc), while the deviation bound for OBJ_TIME is now set to zero.  These constraints cause the value of VAS_UC(OBJ1,1) to remain at its upper bound.

Consequently, after solving phase 3, VAS_UC(OBJ1,1) has the value 6292.48713883 (the upper bound).

Nonetheless, the value of the standard OBJZ in each phase is not at all lost, but you can get it at least from the following reporting attributes:
  • Reg_obj(reg,sow)  –  objective by region
  • User_con(OBJZ,sow) –  total objective
(25-07-2022, 11:08 PM)Antti-L Wrote: In the multiphase set-up, for every SOW, VAS_UC(OBJ1,SOW) represents the objective function value in that phase.
And therefore, in the first phase (SOW=1), the value of VAS_UC(OBJ1,1) will indeed be the value of the standard objective function, which is 5720.44285348534.

Dear Antti

Thanks for outlining this. I agree and can follow your statements. However, I feel my concrete question is not answered - or I just can't identify the answer to this specific issue in your previous reply. Let me rephrase:

In the quoted statement, you say that in phase 1 the VAS_UC(OBJ1,1) is equal to the standard objective (OBJZ): 5720. In principle, that is also my understanding. However, what remains unclear to me is why I see in VEDA-BE that OBJ1 in SOW=1 is 6292 instead of 5720 (see screenshot:

If your previous answer contained an (implicit) answer to this, please forgive that it does not appear obvious to me.

Thanks in advance for your efforts,
Hmm...  I think there was an explicit answer:

AL> Consequently, after solving phase 3, VAS_UC(OBJ1,1) has the value 6292.48713883.

Because the value of VAS_UC(OBJ1,1) is 6292.48713883 after all three phases are completed, it is also reported to have that value. That is the value for VAS_UC(OBJ1,sow) with sow=1 (the second index of VAS_UC(OBJ1,1) is the SOW index). And the User_con reporting attribute shown in your screenshot reports exactly those values: VAS_UC(OBJ1,sow), for sow=1,2,3.

Your screenshot does NOT show any OBJ1 in SOW=1 to be 6292. OBJ1 is a member of the set UC_N, and it has no value.  The value 6292.48713883 is shown for the attribute User_con(sow,uc_n,r,t,ts), which reports the levels of the variables VAS_UC(uc_n,sow) (among others). And because the variable VAS_UC(OBJ1,1) has that value as its level, it is being reported so.

Does that make it any clearer?
Antti, thanks for providing such additional insights. It's making the background more clear.
Thanks for always being very helpful!
> It's making the background more clear.

Well thanks, but I hoped that the "foreground" would also have been more clear. The levels of the these variables reported by TIMES are obtained directly from the solution provided by the solver, and correspond to the optimal solution. The value you expected to see would thus be deviating considerably from that solution. Blush

TIMES does calculate some variable levels also by itself, but basically only for the so-called reduced variables, according to the variable substitutions TIMES has made. I think it is good practice that the reported levels are consistent with the optimal solution.

Forum Jump:

Users browsing this thread: 1 Guest(s)