About the stochastic version of TIMES

[Mahmoud]

Dear all, 

I have been trying to use the stochastic version of TIMES. In order for me to be sure about the paradigm behind this version I have some questions: 

Let's say I am defining a problem where the installation cost of process A is uncertain and the year corresponding to the resolution of uncertainty about this parameter is 2040. Once the uncertainty resolved, the model discovers two SoWs. So far, I have defined a model with two stages such that the second stage has 2 SoWs. If my understanding of the documentation in this regard is good (http://iea-etsap.org/docs/TIMES-Stochastic.pdf): 
1- their are two ways of implementing the stochastic version of TIMES (first way is the set of equations (1), (2), (3) of the aforementioned documentation (page 7) and the second way is the set of equations (1'), (2'), (3'))
2- the second formulation is less parsimonious in terms of additional variables and constraints, but many of these extra variables and constraints are in fact eliminated by the pre-processor of most optimizers (cplex included)
3- the two formulations are equivalent and would have the same results
4- the first formulation is used in the current formulation of TIMES. 
5- if the answer to 4 is yes, then are the levels of variables (installed capacities, retired ones, activity levels, etc.) going to be found once to satisfy all SoWs or are we supposed to have a level per SoW for each variable of the problem (which seems to be the case because X has to two params X(t, s))? In other words, is TIMES going to find a solution that satisfy the two subproblems SoW1 and SoW2 of the aforementioned problem or is it going to find a solution (variables levels) per SoW?

Thanks in advance for your answers
Mahmoud

[Antti-L]

4- the first formulation is used in the current formulation of TIMES.
5- if the answer to 4 is yes, then are the levels of variables (installed capacities, retired ones, activity levels, etc.) going to be found once to satisfy all SoWs or are we supposed to have a level per SoW for each variable of the problem (which seems to be the case because X has to two params X(t, s))? In other words, is TIMES going to find a solution that satisfy the two subproblems SoW1 and SoW2 of the aforementioned problem or is it going to find a solution (variables levels) per SoW?

4: Yes, the answer to 4 is yes, as stated in the documentation.
5: I don't understand this question.  As stated, the two formulations are equivalent.  As described in the documentation, in the second case we would force X(2010,1)=X(2010,2)=X(2010,3)=X(2010,4).  The equivalent formulation is obtained by just replacing X(2010,2), X(2010,3), and X(2010,4) everywhere by X(2010,1).  And by doing that we get the first formulation. Neither of these two formulations has any sub-problems. Instead, the whole model is solved as a single optimization problem in both cases.  In both cases we thus have, implicitly or explicitly, a level per SoW for each variable of the problem.

A decomposition algorithm would have sub-problems, but such has not been implemented.

[Mahmoud]

Thank you Antti for the clear, simple and precise answer.
Best regards
Mahmoud

[Antti-L]

Thanks, but I still wonder about the reason behind presenting the question in the first place.

You mention you want to "be sure about the paradigm behind this version".  But your question seemed to be all about implementation. However, implementation is basically a technical matter not much related to the paradigm behind the generic mathematical design. So what is there to wish to be sure about?

To put this another way: I think the paradigm here is multi-stage stochastic programming.  But the implementation, be it in technical details formulated in any way, as a single big optimization problem or with a decomposition algorithm, should in my view not matter much to users, because the choice of the implementation approach would not affect the model results (apart from numerical quality).  A decomposition approach would probably be better in time-performance but worse in numerical performance and more complex to implement.

[Mahmoud]

Sorry Antti for the late reply. Indeed, my question was to certain extent related to the paradigm behind the stochastic version of TIMES. I am not an expert in Multi-stage stochastic programming but when I first attempted to use this version, i was expecting the following: If I have n states of the world at the last stage of the model, TIMES would find one robust solution (the level of variables would be the same for all SoWs) that is available for al the SoWs. But as you mentioned, It is not the case. 

To make it simple, say we have uncertainty on the level of demand. the demand's values are revealed in 2040 and it turns out that there are 3 levels of demand. The unique solution for the 3 SoWs before 2040 is then more robust than a determinstic run with one level of demand. But the 3 solutions after 2040 are purely determinstic because we don't have any other uncertainty. This is completely realistic and normal but I had a hard time making sense of this 

In this respect, among the possible uses of the stochastic version of TIMES could be to calculate the cost of having perfect information by substracting the solution of a deterministic scenario from the solution of a stochastic one.

[Antti-L]

Ok thanks, now I understand you better.

Indeed, the TIMES multi-stage stochastic option is for analysing optimal hedging strategies against some long-term uncertainties, as documented. Hedging is here related to uncertainties longer in the future, not in the same model period. In the periods of your two branches, according to this approach the key uncertainty modelled has already been resolved, and so the demands are then already known, and therefore the solutions for these periods may be considered robust with respect to the resolved uncertainty.

The TIMES stochastic programming paradigm is quite similar to that in Markal, except that TIMES supports truly multiple stages, while Markal supported only two.

Quote from the Markal documentation:
"The Stochastic Programming paradigm consists of representing multiple scenarios (usually called states-
of-the-world, or sow), each having a possibility of occurring, within a single coherent formulation. The key to success is simply to define, for each particular decision to be considered, as many decision variables as there are states-of-the-world in that time period. For instance, in the example evoked above, there should be a single copy of investment and other variables for the 15 years prior to uncertainty resolution (the point in time when we know which outcome materializes), precisely because a single strategy must be followed during that period of time. On the contrary, for later periods, the outcome of the event is unknown, and therefore, there should be two sets of variables, each representing a particular decision contingent on the state-of-the-world that might prevail."

Nonetheless, with TIMES, you could also try using multiple stages and/or the short term "recurring uncertainties" option.  And, if you can provide a design for implementing exactly what you want (one robust solution even for the periods after the uncertainty resolution date), please share it or submit a proposal for implementing it.