Veda2.0 Released!


TIMES-Macro-MLF infeasible baseline recuperation run
#1
Hi again,

I have been testing the new MLF method in a full sized model now, and am running into some infeasibility: *** Error at line 233331: rPower: FUNC DOMAIN: x**y, x < 0
The baseline run worked, but when attempting the baseline recuperation run it doesn't solve. (see files attached)

What I tested so far:
- I don't think it is due to some constraint that is not allowing the Macro demand variation, because I tested a version with all constraints turned off, and I still get the same infeasibility.
- I tested increasing or decreasing the base year GDP (as I think this parameter could influence the calibration of the production function and ultimately lead to the rPower error?) But this also doesn't seem to solve the infeasibility. I also think I defined the value correctly? (provided in the MACRO monetary unit, so the TIMES unit but scaled with the scale_cst?)



Does anyone have any idea what else could cause this infeasability?

Thanks in advance!


Attached Files
.zip   mlf_tests.zip (Size: 81.75 KB / Downloads: 7)
Reply
#2
Yes, interestingly your MLF run shows that the calibration is not working at all: All the small calibration runs that are needed for the MLF run are shown as ending with the "Locally infeasible" status in your listing file. To clarify a bit that, when using MLF,  the MACRO calibration is done "on-the-fly", by using the Base prices for all the demands and the annual cost data from the Baseline run.  This has been working fine in my tests with some larger models as well, but apparently it is not succeeding with your model.

Once again, it is very hard to tell why that is happening, I am very sorry about that. Unfortunately, the MACRO calibration is indeed  a rather delicate process (both with MSA and MLF).  But make sure the com_bprice.GDX is, in fact, the correct GDX file produced from the Baseline run.  Note that under VEDA, you can link that GDX file into the run also under GDX references.
Reply
#3
I see you are using IPOPT for solving the NLPs. Is that intended?
Could you try using CONOPT instead? The NLPs are so small that it should work even without having a license for CONOPT.
Reply
#4
Still infeasible using CONOPT it seems to me.
This is the last part of the run_log:

--- Generating NLP model MCE
--- pc_10_0314-mlf-baserecup.RUN(233290) 488 Mb
--- LOOPS NITER = 1
--- MREG = BE
--- 87 rows 108 columns 240 non-zeroes
--- 316 nl-code 59 nl-non-zeroes
--- Range statistics (absolute non-zero finite values)
--- RHS [min, max] : [ 1.000E+00, 1.371E+03] - Zero values observed as well
--- Bound [min, max] : [ 3.202E-01, 1.491E+03] - Zero values observed as well
--- Matrix [min, max] : [ 5.734E-05, 1.043E+03] - Zero values observed as well
--- pc_10_0314-mlf-baserecup.RUN(233290) 485 Mb
--- Executing CONOPT (Solvelink=1): elapsed 0:00:12.328

CONOPT 4 46.2.0 ac4adda6 Mar 5, 2024 WEI x86 64bit/MS Window

--- *** This solver runs with a demo license. No commercial use.

*** Error Cannot open parameter file "D:\VEDA2\Gams_Wrk_TIMES_MACRO\conopt.opt"
*** Error Error code = 2; No such file or directory



C O N O P T version 4.33
Copyright © ARKI Consulting and Development A/S
Bagsvaerdvej 246 A
DK-2880 Bagsvaerd, Denmark

Will use up to 16 threads.


The user model has 87 constraints and 108 variables
with 240 Jacobian elements, 59 of which are nonlinear.
The Hessian of the Lagrangian has 59 elements on the diagonal,
20 elements below the diagonal, and 59 nonlinear variables.

Iter Phase Ninf Infeasibility RGmax NSB Step InItr MX OK
0 0 4.3211180878E+04 (Input point)

** An equation in the pre-triangular part of the model cannot
be solved because the infeasibility is positive after all
variables in the constraint have been moved to best bound.
Some bounds may have been derived from other constraints.


The pre-triangular part of the model has 22 constraints and 35 variables.
The post-triangular part of the model has 2 constraints and variables.

** Infeasible solution. The Preprocessor has determined that the model
is infeasible.

--- Reading solution for model MCE
--- Executing after solve: elapsed 0:00:12.460
--- pc_10_0314-mlf-baserecup.RUN(233240) 485 Mb
*** Error at line 233296: division by eps
--- pc_10_0314-mlf-baserecup.RUN(233240) 485 Mb 1 Error
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
--- pc_10_0314-mlf-baserecup.RUN(233319) 485 Mb
*** SOLVE not executed
*** Error at line 233326: division by eps
--- pc_10_0314-mlf-baserecup.RUN(233319) 485 Mb 2 Errors
*** Error at line 233333: division by eps
--- pc_10_0314-mlf-baserecup.RUN(233319) 485 Mb 3 Errors
*** Error at line 233341: division by zero (0)
--- pc_10_0314-mlf-baserecup.RUN(233319) 485 Mb 4 Errors
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
*** SOLVE not executed
--- pc_10_0314-mlf-baserecup.RUN(233462) 485 Mb
*** Error at line 233462: division by eps
--- pc_10_0314-mlf-baserecup.RUN(236161) 494 Mb 5 Errors
--- pc_10_0314-mlf-baserecup.RUN(236824) 494 Mb 5 Errors
*** Error at line 236824: Execution halted: abort$5 '*** ERRORS IN GAMS EXECUTION ***'
--- pc_10_0314-mlf-baserecup.RUN(236824) 494 Mb 6 Errors
--- Putfile END_GAMS D:\VEDA2\Gams_Wrk_TIMES_MACRO\END_GAMS
--- Putfile QLOG D:\VEDA2\Gams_Wrk_TIMES_MACRO\QA_CHECK.LOG
--- GDX File D:\VEDA2\Gams_Wrk_TIMES_MACRO\GAMSSAVE\pc_10_0314-mlf-baserecup.gdx
--- Profile Summary (2413 records processed)
2.360 0.469GB 215677 Assignment ACT_FLO (820080)
1.859 0.494GB 236824 GAMS Fini
0.907 0.086GB 212290 Loop
0.625 0.287GB 214937 Assignment RTPCS_VARF (3113631)
0.625 0.317GB 215501 Assignment COEF_AF (11308)
0.531 0.396GB 215506 Assignment COEF_AF (1448101)
0.469 0.055GB 207694 IF-ELSE
0.313 0.485GB 218978 Loop
0.313 0.055GB 205115 IF-ELSE
0.266 0.201GB 214542 Assignment UC_FLO (1320280)
--- pc_10_0314-mlf-baserecup.RUN(236824) 494 Mb 6 Errors
*** Status: Execution error(s)
--- Job pc_10_0314-mlf-baserecup.RUN Stop 07/08/25 17:12:34 elapsed 0:00:14.654
Reply
#5
Ok, thanks anyway for trying it.  Blush
How about the MSA algorithm, does it work for you now?
I think you said earlier that you managed to get it working?
Reply
#6
Out of curiosity, I tried MLF now with a UK TIMES test model instance that I happened to use for testing MLF also a few years ago, and I was happy to see it still working well (this test model is much larger than the MLF demo). Therefore, I am at least relieved that the issue you are seeing does not seem to be any side-effect related to the recent small improvements that have been implemented into MLF.

Concerning TM_GDP0, if your currency unit is in million EUR, TM_GDP0 should by default be in 1000 million EUR, which I think is the case, and so that should ok.
Reply
#7
Thanks for the responses!

The MSA formulation does seem to be working, but not if I include a net zero constraint in 2050.
I assume this is again because I am not allowing a demand variation in 2050 due to this constraint, blocking the MACRO functionality.

I wanted to test the impact of including a CO2 tax on the system:
1) CSA run: NO net zero constraint in 2050 + low carbon tax in 2050 -> MSADDF file looks ok
2) MSA recuperation run: GDP loss is near zero -> successful calibration
3) MSA run: NO net zero constraint in 2050 + high carbon tax in 2050 -> GDP loss is very large (>10%) so I think something is going wrong again


Looking at the LST file, I see some 'infeasible' statements. (see attachement)
How can I check if the MSA policy run was successful or not? (Is an MSA_ERR of 0.12 too large?)

Thanks in advance!


Attached Files
.zip   policy_msa.zip (Size: 83.05 KB / Downloads: 7)
Reply
#8
Sorry to hear about those persisting problems.
Looking at the listing file, the model appears to become infeasible again in the 4th master iteration. And this is happening just because the demand levels (COM_PROJ) are updated for the master LP model.

When the algorithm was designed, such issues, where the model would become infeasible just because the demand levels are adjusted (in general they are mostly adjusted downwards but can somewhat oscillate during the algorithm), were not anticipated to be happening. But it is obvious now that one should implement an automatic termination of the algorithm when such happens. It cannot produce any reliable results if the model becomes infeasible.

Indeed, looking at the macro results, the GDP is reported to increase (not decrease) notably in this MSA run (starting from 2030), which does not even make sense.

Returning back to your MLF experiment, could you perhaps post the com_bprice.gdx from the Baseline, just to see in case it happens to reveal some imminent problem.
Reply
#9
Thanks for the input!
Here the com_bprice file of my latest MLF baserun in attachment.


Attached Files
.zip   com_bprice.zip (Size: 27.48 KB / Downloads: 3)
Reply
#10
Getting back to this topic after some time again:

In the meantime, I managed to get the MLF implementation running by running until 2050 instead of 2070. (workable for now)

I did notice the value of the demand coefficient TM_B has reduced to a single value (looking in the gdx file), while I would expect a different value for each demand type as it was in the previous implementations (original MACRO and MACRO-MSA).
I don't immediately find an explanation in the MLF note, so I am wondering what the reason for this is.
It seems to me that if this is the case, the MLF implementation considers substitution between all demands as equal? Is this correct?

All insights are welcome and thanks in advance!
Reply
#11
Thanks for the follow-up.

In the original TIMES-Macro and TIMES Marco-MSA formulations, TM_B(r,c) is a calibration coefficient for the individual demands in the (KL)E CES aggregate (and TM_AKL is the corresponding coefficient for the KL component). So, TM_B is not an input parameter, but derived by TIMES during the calibration.  In the MLF formulation, the individual demands have been moved to a second CES function, producing a demand aggregate, which is then replacing the E component in the (KL)E CES function. Therefore, only a single TM_B coefficient is needed for the E component. But that second CES function is now calibrated with share coefficients for the individual demands. In my understanding, the resulting nested CES function should be equivalent to the original unnested CES function. However, as I am not at all a CGE modeling expert, it may be possible that I could have understood it wrong.  Let me know if you are convinced of that being the case.

Concerning elasticity of substitution, the original TIMES-Macro and TIMES Marco-MSA formulations have only a single CES elasticity TM_ESUB, which is the same for all individual demands and the KL aggregate (as these are all under a single CES function).  In the MLF formulation, one can define an substitution elasticity for the (KL)E CES function (TM_ESUB), and another substitution elasticity for the demand CES aggregate (TM_DESUB).  So, in that respect the MLF formulation gives some more flexibility.  If not defined, by default TM_DESUB is set equal to TM_ESUB.

> It seems to me that if this is the case, the MLF implementation considers substitution between all demands as equal?

In a standard CES function, the elasticity of substitution is indeed equal between the components of each CES. This is so in the original TIMES-Macro, TIMES Marco-MSA and the MLF formulation. But in the MLF formulation, the original single CES has been disaggregated into two nested CES functions, which thereby can have elasticities differing from each other.
Reply
#12
> In the meantime, I managed to get the MLF implementation running by running until 2050 instead of 2070. (workable for now)

I am very pleased to hear that, given the many difficulties you had with it (and MSA) before. Although my earlier impression was that your model most probably had some problems with the stability of the marginal prices, if there is now any useful information that you could share about how you managed to solve the problems with using MSA/MLF, I think that could be helpful. And, similarly, if you have any suggestions for improving the implementation.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  TIMES-Macro-MSA and gdx behaviour Enya 16 2,622 06-06-2025, 08:39 PM
Last Post: Enya
  Running TIMES model on linux HPC cluster LucasRM 7 9,352 21-01-2025, 11:13 PM
Last Post: AKanudia
  TIMES-Macro-MSA MSADDF.DD file UNDF Enya 7 2,142 16-12-2024, 06:06 PM
Last Post: Enya
  A question about EU-TIMES xiao.li8@mcgill.ca 1 816 19-08-2024, 12:07 AM
Last Post: Antti-L
  One question about EU-TIMES seanli12354 0 825 09-06-2024, 06:54 PM
Last Post: seanli12354
  An error when I read JRC-EU-TIMES Lee 7 4,070 03-06-2024, 05:28 PM
Last Post: Lee
  One question of EU-TIMES: CO2 emissions for gas/oil production/transmission process xiao.li8@mcgill.ca 1 1,109 30-05-2024, 02:58 PM
Last Post: Antti-L
  The potential mistake exists in the EU-TIMES xiao.li8@mcgill.ca 11 5,624 09-05-2024, 03:42 PM
Last Post: wnijs
  Error at line 904570: Equation infeasible due to rhs value frangb99 12 5,338 03-04-2024, 12:19 AM
Last Post: frangb99
  Model Infeasible Question VanessaDing 4 2,846 14-03-2024, 07:47 PM
Last Post: Antti-L

Forum Jump:


Users browsing this thread: 2 Guest(s)