Veda2.0 Released!


Absolute values via UPD Tables
#1

Although the original purpose of the UPD tag was to transform existing data values (in a rule-based manner), they can be very useful for introducing absolute values as well. The only difference with INS table in this case is that the scope of indexes is determined by what already exists. So you typically use this without specifying each and every index. In other words, absolute values in UPD tables work like a search and replace facility.

For example, consider a generic process used to model "other" demand or something, which has several inputs and one output. There are upper and lower shares declared for various input, in different years and different regions. Now, I want to run a scenario with all the lower shares relaxed (=0). I can simply make an update table with Attribute, Process and Limtype. No need to specify any commodity or commodity group.
But it is easy to forget that you are actually providing transformation instructions if positive and negative values are mixed in your specification. Here is an example of what entries in a UPD table would do:
  • +x will add x to the existing data values
  • x will replace existing data values with x
  • -x will subtract x from existing data values
Reply
#2
AKanudia Wrote:
  • +x will add x to the existing data values
  • x will replace existing data values with x
  • -x will subtract x from existing data values
So, what should the user do if she wants to replace existing data values with negative values? As you know, negative values are quite meaningful for many TIMES attributes.
Is it at all possible?  Perhaps by using several UPD tables, the first of which updates to the absolute value, and the second multiplies by -1?  Does that work (it did work in earlier VEDA versions)?

Thanks
Antti
Reply
#3

I don't see any easy workaround; INS table is the only option. One way to make the task easier would be to browse the data you are looking to replace and use its exported form to construct the INS table.

You are right, earlier versions of VEDA did allow updated values to serve as seed values for subsequent updates within the same scenario. But this was stopped as it was spoiling a very basic overwriting facility that VEDA offers across all data specification tables - to specify a value with a coarse filter and then overwrite with different values with filters that are subsets of the first one. Examples:
  • One wants a scenario with one UPDate rule for all cars but a different one for ELC cars. I would first specify the rules for all cars and then overwrite it for ELC cars.
  • One wants to disable all LO ACTivity bounds on a particular set of processes after the base year. I would update with 0 without a year specification first and then do a *1 for year=<base year>.

To make this work, the basic principle we applied was that all UPDates should see the same seed value in one scenario.

Further, a related fact that deserves to be mentioned here - UPD tables do not take seed values from scenarios that are alphabetically superior to the current scenario. Even INS tables in the same scenario can provide seed values. Of course, UPD in TRADES scenarios will see BASE, all SubRES and DEMAND scenarios regardless of the name and all regular scenarios will see all TRADES scenarios in addition.
Reply
#4
Similar to this example, but we're trying a different application. We have 5 process with multiple comm-in and only one comm-out. We're trying to use a ~tfm_upd table to replace only one input, INDELC, with a different commodity. However, our attempts have failed so far. We've tried altering both the comm-in and VDA_FLOP attributes. Both attempts got sync errors saying no table rows had been generated. Is there a different way to use UPD tables to replace comm-in commodities? Additionally, how can we specify which comm-in we want to replace for processes that have multiple comm-in commodities?

Thanks
Reply
#5
please post a screenshot of the process/attribute in browse, and be more specific on what you are trying to do.
Reply
#6
I've attached two screenshots, one of the base scenario and the other of the tfm_upd table. We are trying to create a scenario that will replace the INDELC listed for a number of technologies as comm-in commodities with another, ELCH2. Since we're doing this for a number of technologies and it's only part of one analysis, we were trying to refrain from adding ELCH2 as an input commodity for all of these processes and then using the tfm_upd table to change the input shares. Being able to switch out one comm-in for another is much more versatile and lets us apply the change to any process that we name rather than having to actually add in that commodity as an input for all of the processes. However, we have not successfully been able to use the tfm_upd table to change the specific comm-in.


Attached Files Thumbnail(s)
       
Reply
#7
I am not a VEDA expert, but I don't think what you are trying to do can be supported by TFM_UPD.  The purpose of TFM_UPD is to update values of parameters, not the domain indexes of the tuples for which the parameters have been defined. Each parameter instance has a unique value, which can therefore be updated unambiguously, by overwriting the earlier value. But I don't see any unambiguous way of updating parameter domain indexes, at least not within the TFM_UPD paradigm.

I think you could achieve what you want by using TFM_TOPINS (adding Input commodities), TFM_MIG (adding VDA_FLOP to new input flows), and using FLO_OFF or time-series filtering (for removing the unwanted flows replaced by new ones), in a well-defined way. But of course there could be also some other well-defined ways to do it in the current VEDA, which I may not know/recall.
Reply
#8
I think you're right, I'll work on implementing the multi-table approach and see if I can get the switch working
Reply
#9
Is there documentation on the TFM_MIG and TOPINS tables anywhere? I've found a couple of posts that give more information but there's no documentation in the TIMES and VEDA official documentation.
Reply
#10
Yes, they are not TIMES features, and so are not included in the TIMES documentation.

They are VEDA features, but I don't know whether VEDA has full documentation for them somewhere. However, TFM_TOPINS looks quite straightforward (I guess you have seen this):
https://forum.kanors-emr.org/showthread.php?tid=144

TFM_MIG is somewhat more complex, but as far as I know, the basic idea is that you can use (almost) any column header postfixed by 2, to indicate the desired index for the target parameter. So, to copy the VDA_FLOP parameters of certain processes from one input commodity to another, I think you can put the source commodity under CommGrp and the target under Cset_CN2 (and the process filter under Pset_PN, and "*1" under the columns with Region names).  I just tested that with a demo model, and I was happy to see it worked.
Reply
#11
Thank you for that suggestion. I've encountered some issues here and am hoping you'll be able to help me troubleshoot.


I've attached images of the scenario file, commodity definition in system settings, and the res diagram. For context I'll give an explanation of what I'm trying to achieve overall and what I've tried so far. Our goal is to require that electricity from renewable energy be used for hydrogen production. In order to do this, we've created a new commodity, RNWELC, and defined it in the syssettings file. It's not used anywhere in our basetrans files. The only place we're introducing it is in a scenario file, which is attached. The first topins table is meant to add rnwelc as an output for all solar and wind electricity production processes. The intention here is to have it as an alternate output, either solar and wind techs can produce ELC, as is the case in the basetrans files, or they can produce RNWELC. 

The second topins table is meant to add RNWELC as an input commodity for hydrogen production techs. The idea here is that we can replace INDELC, the current electricity input with RNWELC, guaranteeing that all electricity inputs are renewable electricity inputs. The tfm_mig table below that is the table I created based on your suggestion above. Each process has a different input flow value. I first used "Input" as the attribute and specifically gave the value for each process, but that did not work. Now I'm using your method of having INDELC listed under CommGrp and RNWELC listed under Cset_CN2 to copy the VDA_FLOP values from INDELC to RNWELC, using "*1" as the value for AllRegions. However, as you can see from the attached res diagram, those vda_flop values are not present for RNWELC, they're only present for INDELC. Additionally, the last table, the tfm_upd table, is supposed to change the INDELC input values to 0, essentially to *turn it off* and turn RNWELC on. However, the RNWELC values arent showing up and the INDELC values are not going to 0. Clearly I'm not building this correctly.

question 1) are there any additional characterizations that you think we need to make for RNWELC here, for example specify flow values? or can we assume, as we have done, that the topins table sticks RNWELC as an output for solar and wind techs with a default output of 1. I attached an image of the res for a solar tech and I can see that RNWELC is defined as an output because it's appearing in the res diagram, but I dont actually see any flows for it. There are flo_emis values for the emissions, but the only flow I see for ELC, the commodity we're trying to replicate with RNWELC, is defined with UC_FLO in two constraints that we have. Where can I *check* that RNWELC is properly flowing out of the solar techs?

question 2) do I need to have these tables in separate scenario files, and if so, does it matter the order that they're in? I'm particularly interested in this with regards to the _mig table that copies the INDELC vda_flop values and the _upd table that changes the INDELC input value to 0. Are these somehow mutually exclusive the way I've built them?

question 3) you mentioned in an earlier comment using FLO_OFF to *turn off* the INDELC flow, however I've read through the TIMES documentation and could not discern how to implement that

Any assistance with this would be greatly appreciated. Additionally, if theres a different way to achieve our goals that would be simpler, I'd be happy to explore it. 

Thanks!
Morgan


Attached Files Thumbnail(s)
               
Reply
#12
I am very sorry to hear that you have still problems with the issue and apparently have wasted maybe even two weeks on it!  Sad  However, please note that I am not a VEDA developer, and therefore if you have VEDA-related problems or questions, I am not able to be of much help.

Concerning TIMES, I am indeed sorry for the FLO_OFF name issue. As you can see from the documentation and from the VEDA-FE Attributes Master, there are several "OFF" attributes in TIMES: new capacity OFF, activity OFF, flow OFF, and commodity OFF.  It was my bad mistake to remember incorrectly that the flow OFF attribute would be named "FLO_OFF", but of course it is not. Looking at the Attributes Master, filtering by "OFF". you can immediately see that new capacity OFF = PRC_NOFF, activity OFF = PRC_AOFF, flow OFF = PRC_FOFF, and commodity OFF = COM_OFF.  Again, very sorry for my bad mistake; I have not used PRC_FOFF very often.

PRC_FOFF description:
PRC_FOFF: Set of sextuples specifying that the flow of commodity c at process p and timeslice s is not available between the years y1 and y2 in region r [this is a switch; >0 => YES]

Concerning TFM_MIG and TFM_TOPINS, I can only repeat what I said before: In my tests TFM_MIG works well for copying VDA_FLOP from one input commodity to another.  I now tested also TFM_TOPINS, and it also worked as expected.  See picture below showing the TFM_MIG, TFM_TOPINS and the resulting VDA_FLOPs.

   

Explanation: For the TCAREELC process in the Demo12 model, I first added a new input commodity TRALPG with VDA_FLOP=0.7 into the BASE template . And then I used TFM_MIG and TFM_TOPINS in the CO2_TAX scenario for adding a TRAGSL input.  As a result, VDA_FLOP was copied from TRALPG to TRAGSL, as expected, and TRAGSL was also shown added as an input to the topology. What was left untested is adding PRC_FOFF('REG2','TCAREELC','TRALPG','ANNUAL','BOH','EOH') for the process, to disable the original TRALPG flow and thus complete replacing it with TRAGSL.

I also repeat once again my disclaimer: "But of course there could be also some other well-defined ways to do it in the current VEDA, which I may not know/recall."
Reply
#13
thanks, Antti, for this expert and lucid guidance. I confirm that this is the best approach for this task.
Reply
#14
The issue here was that CSET_CN2 was supported in version 115 of Veda2.0 and the user was on version 110. Please make sure you get updated versions frequently when using Veda2.0.

For the information of other readers, I have been waiting for results handling to become more functional before a formal launch of the new version of Veda. It will happen in the next couple of days.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)