Dear community,
I have been trying to implement a stochastic run in TIAM, where the availability of carbon dioxide removal is uncertain.
I have explored the Advanced Demos on stochastic runs, and have tried to implement a stochastic scenario file in the same manner. It's a simple two-stage set-up, where in the second stage CDR is available with a probability of 50%. Please see the attached file for details of the scenario file I am using.
However, when I run this, I get a range of errors, as seen in the attached .lst file, and the run is terminated.
These errors include '141 - Symbol declared but no values have been assigned.' - which makes me think I might not be defining all the variables required for a stochastic run - although I am declaring SW_START/SUBS and SPROB in my scenario file. There's a whole bunch of other errors too...
At a bit of a loss as to where to start with this, so if anyone has any ideas of what might be going wrong / can see any errors in my scenario file that would be hugely appreciated.
I'm using VEDA_FE: 4.5.828 TIMES Source Code: V4.27 Microsoft Office 365
I just tested stochastic run with the old TIMES v4.2.7 code, and did not get any GAMS errors like you are getting. Seeing the run file might reveal something. However, as with many other strange issues, providing the full model input files (*.DD, *.RUN) might be the only easy way to reproduce the issue.
Hi Antti,
Thanks for this reply - good to know the issue isn't with the scenario file.
This folder has all the relevant files for the issue (.DD and .RUN etc) - hopefully something in this will be of use to you? I was just running a fairly generic 2 degree run in TIAM, with the scenario file I attached previously added in, and the stochastic switch in the Case Manager turned on.
29-06-2020, 06:39 PM (This post was last modified: 29-06-2020, 06:41 PM by Antti-L.)
Ok, thank you for the model input files. I tested running the model instance you provided, and immediately saw that there were problems:
1) When running with the old TIMES v4.2.7, I can see two GAMS compile errors in the equation EQL_COMCES (which was introduced in v4.2.3). These errors appear only when using TIMESED with STAGES (i.e. a stochastic run with elastic demands), and the errors were caused by not having made that new equation fully stochastic-aware. This small bug in the code was subsequently fixed in TIMES v4.3.1 (in February 2019).
2) However, your listing file shows a lot of more errors, which I cannot reproduce with TIMES v4.2.7 (which is the version number your listing file indicates you are using)! Therefore, I suspect that you, or someone else, has modified your TIMES code in such a way that you get many more GAMS errors.
Consequently, if you are willing to use the ETSAP TIMES code, the solution to the problem is simple: Just upgrade to the latest version, v4.4.1. However, if you want to continue using some modified version of TIMES, I am afraid I am not able to suggest any solution; ETSAP cannot really provide support for any modified versions of the code. But in that case I would also strongly encourage you to share the modified code with the ETSAP community, such that if you have implemented any useful new features or improvements, those can be integrated into the common code.
Dear Antti,
Many thanks for your rapid reply. I am not aware of any modifications to the TIMES code that we're using... But I will check with my colleagues and clarify that immediately. But sounds like the issue will be solved by moving to the latest ETSAP TIMES code.
Hi Antti,
My stochastic runs are working as expected now - so thanks for your support on this. Just a couple of quick questions:
1. I can't get my stochastic runs to work with price elasticity at the moment. When I use a base price from a deterministic run, e.g. a Current Policies reference scenario, the model builds fine, but the GAMS solver fails to find a solution. It completes both the primal and the dual, but then gets stuck in a set of iterations that it doesn't break out from. I'm attaching a couple of screen shots from the terminal that might give some information on this issue, as well as my LST file, which doesn't seem to have any issues.
I'm wondering if you have any thoughts about why this might be occuring?
We've enabled elastic demand, but not for all demands, so some demands have COM_BPRICE/ELAST/VOC defined for them, and others don't. Could this be an issue when running in stochastic mode?
2. I've been using SW_LAMBDA to represent risk aversion. In the documentation, it says that this can range from 0 to inf. Does anyone have any literature to justify a range of lambda to explore within this? Even setting lambda to 1 makes a significant difference to my results, and I've seen some papers in the literature that suggest certain decision-makers have lambda <<1. If you have any suggestions around justifiable values of lambda to use, that would be really appreciated.
I just tested a stochastic run with elastic demands, and it ran well to completion.
I am afraid it is not possible for me to know why your runs have such numerical difficulties. But I suspect that your model has "bad scaling" or other issues, which cause numerical instability. I assume that you have disabled any dummy imports, which tend to cause such problems? If you have not done that, I think that would be the first thing to try. Thus, try setting e.g. NCAP_START(r,IMP*Z)=2200. Of course, your model must be feasible before doing that...
13-07-2020, 09:55 PM (This post was last modified: 13-07-2020, 10:01 PM by Antti-L.)
To be more precise, you should disable the dummy imports already in the Base run (the deterministic Current Policies reference scenario), in order to make sure your Base prices are not polluted by the impact of the dummy import marginals. And then of course keep them disabled in the stochastic run(s).
And further, it is also advisable to disable elastic demands in the first periods (e.g. up to 2020), by setting either COM_VOC or COM_ELAST to zero in those first periods. That will give additional assurance that any bad prices that may often appear in the first periods will not cause numerical problems. After all, you cannot expect historical demands to be changed by your policies, can you?
13-07-2020, 11:34 PM (This post was last modified: 13-07-2020, 11:38 PM by Antti-L.)
I recalled now that you actually provided a model instance for me earlier in this thread.
I ran that model now again without stochastics, and saw from the results that there were many dummy import flows, and even in large amounts, see attached picture below. As I am sure you know, dummy import flows signify infeasibilities in your model. If you follow good practices, you should of course eliminate them. And when that has been done, the model is truly feasible and the dummy import processes can then be disabled.
If your model still has such infeasibilities, I am not surprised that your model does not solve well under elastic demands. So, I would suggest to follow good practices and eliminate those infeasibilities, and then disable the dummy import processes to make sure your prices remain stable throughout the solution process.
14-07-2020, 02:10 PM (This post was last modified: 14-07-2020, 02:21 PM by NeilGrant.)
Hi Antti,
Many thanks for your posts, which are very helpful. I will have a look at these dummy import flows and try and eliminate them (certainly want to follow good practices!)
The picture you attached didn't seem to work unfortunately - any chance that you could reattach that? And if you have any thoughts about a sensible or justified range of lambda in the context of long-term climate and energy scenarios, I'd be interested to know! (Though appreciate this is less a TIMES-specific modelling question).
Hi Antti,
One last question (I hope!) on these stochastic runs.
I've managed to advance my runs pretty well with your help, but I have a remaining issue. When I try and run binary stochastic runs with just two future states of the world, the model solves fine in ~2hrs.
However, when I attempt to run stochastic scenarios in which there are >2 future states of the world (I'm currently trying with just 3 future SOWs), the model fails to solve - it gets stuck in iterations after the primal/dual phase, and then times out on the resource limit of the solver. I've tried pushing the RESLIM up to 10,000, and the model still hits this limit, giving the 'Resource Interrupt/Intermediate Infeasible' error.
I'm running with
VEDA_FE: 4.5.828 (need to move to VEDA2.0 soon!)
TIMES Source Code: V4.4.1
GAMS: 33.1
Cplex settings: scaind 1
rerun yes
iis yes
lpmethod 4
baralg 2
barcrossalg 1
barorder 2
eprhs 1e-9
I've had a look at potential scaling issues in our version of TIAM with JacAnalysis, and there are no constraints with a spread of >10^6, so I don't think this is the issue.
I was wondering whether you could just confirm that my scenario file is correct? It intends to represent 3 different worlds, each with equal probability, in which the CDR potential is low/medium/high (all dummy numbers, so don't focus on the exact level of the constraints). Can you see any obvious errors in this file which could be preventing model convergence? And if not, do you have any thoughts about what could be leading to this behaviour?
30-11-2020, 05:41 PM (This post was last modified: 30-11-2020, 05:48 PM by Antti-L.)
Hmm... at a quick glance I cannot see anything wrong in your scenario.
But apparently the solver is having numerical difficulties again. Can you tell whether you have disabled all the dummy import processes now? E.g. by setting NCAP_START(r,IMP*Z)=2200?
Maybe you could post the listing file and the QA_Check.log to see if they reveal anything suspicious?
