VEDA2 Question

[Antti-L]

I have been getting some more SyncLog issues with VEDA2 v3.0.7, compared to earlier versions, e.g.:

  o TechDesc column in FI_T tables triggers a warning about column not recognized or invalid
  o  NCAP_AFC~ANNUAL in FI_T tables triggers an error about column not recognized or invalid

The warnings of the first kind are of course cosmetic of nature, but because TechDesc has been used in numerous model templates in the past (e.g. ETSAP-TIAM), I think it should not trigger any warning. Could this cosmetic clutter be removed from the SyncLog? Earlier VEDA2 versions did not produce such.

The warnings of the second kind signify a bug in VEDA2. The NCAP_AFC(ANNUAL) parameters are fully valid just like NCAP_AFC(DAYNITE), and fortunately NCAP_AFC~DAYNITE does not trigger such a warning. Could this be fixed, please?  NCAP_AFC(ANNUAL) parameters are now left missing from the model input data.
   

[rohit]

Dear Antii,

TechDesc column is not supported in FI_T tables (can be verified via the menu item Information -> Veda Tags -> Tag name: fi_t).
If you prefer, we can create a new user option to ignore/hide such messages related to unrecognized or invalid columns.

We attempted to reproduce the NCAP_AFC~ANNUAL error in Model_Demo_Adv_Veda model in SubRES_Storage.xlsx file, but encountered no issues.

Could you please provide a reproducible test case?

[Antti-L]

Thanks. 

I am confused if you say that TechDesc would be an invalid column in FI_T, because it has been heavily used in TIMES models developed by KanOrs itself. For example, the well-known ETSAP-TIAM model. And it has never before caused any warning about "unrecognized or invalid column".  Not in VEDA-FE, and not in VEDA2 v2.x  So, can you please explain how could it now suddenly, after all these years of being valid under VEDA, would now be considered "unrecognized or invalid column"? 

I think it is a break in backwards compatibility although only in terms of warnings. But I cannot believe anyone would want those warning messages about TechDesc columns. 

Therefore, thanks but no, I don't want a new user option to ignore/hide such messages related to unrecognized or invalid columns in general. I would rather wish that you would reinstate TechDesc as being considered a valid ("comment") column in FI_T tables. Could you not revert this change about TechDesc, because I think the warnings about it are not at all desirable?

Concerning a reproducible test case, please find one attached.  Note that ANNUAL is both a predefined timeslice level, and a predefined timeslice, and for NCAP_AFC it is a timeslice level.  However, please note that ANNUAL should always be valid also as a timeslice.  There is no TIMES model that would not have the ANNUAL timeslice and the ANNUAL timeslice level.

  DEMOBUG.ZIP (Size: 291.26 KB / Downloads: 1)

[AKanudia]

Well, the entire idea of reporting column headers that are not recognized (and processed) is new. This was (NOT) being done silently so far. So, there is no question of backwards compatibility as this is a NEW feature.

Would you agree with this?

[Antti-L]

Hmm...  but assuming that TechDesc is a valid column (it always was so by convention), I would say the new feature is not consistent with that long-standing convention, by writing out numerous irrelevant warnings about TechDesc, which are just making it harder to focus on relevant warnings and errors.

What is the motivation behind now labeling it invalid / unrecognized?

[AKanudia]

On further reflection, I do see how this could be construed as a backward compatibility issue. There *was* a time when the TechDesc col actually worked in FI_T - before the ~FI_Process tag came along. Units were a part of FI_T too... we are talking more than two decades ago. But that col stayed on even after it stopped being read. It does serve a useful purpose when I am not very familiar with Tech names, but I simply use *TechDesc now.

I am sure you agree that reporting col headers that are not recognized is a very useful enhacement. The point is that we only know what we CAN PROCESS... everything esle is flagged as "not recognized". We cannot add "TechDesc" to any list that we use currently. We can certianly add an exception, but we have resisted doing so in Veda2. Veda_FE was replete with such exceptions, and we know what that lead to, eventually.

We have an ETSAP project wtih VITO on having Veda read "documentation". We will certainly keep TechDesc in mind in that endeavor.

[Antti-L]

Ok, thanks for the additional elaboration, and considering the possibility of adding TechDesc as a recognized column.

Yes, I agree that reporting col headers that are not recognized can be a very useful enhacement.  In fact, it was because of that I discovered the bug in VEDA2 I reported  above.  Have you been able to reproduce it now, with the test case I provided?

[AKanudia]

Yes, it has been fixed for the next release.

[Antti-L]

Thanks Amit. 

[AKanudia]

For handling the *new* sync log warnings, how about adding a Veda parameter that lets users identify column headers that should NOT trigger the warning?

Please suggest a name for this parameter - for example, UI_VALCOLHED (user instruction - valid col headers).

[Antti-L]

  Thanks Amit. Letting users identify custom column headers NOT to trigger warnings sounds fine to me.

Using a VEDA parameter (e.g. UI_VALCOLHED) looks slightly odd to me, as it gives me the impression it would be scenario-specific, for which I am seeing no need. Earlier in my posts above, I was thinking that e.g. the legacy "TechDesc" header could just be added to the list of supported columns of FI_T tables, with the processing function set to NOP (no operation), and so neither any exceptions nor new parameters would be needed. But if you see such a new parameter useful for more flexible documentation, I am fine with it, and with the name you suggest.

[AKanudia]

Sorry for the confusion. The list of col headers declared via this attribute would be valid across all files in a model. I would use BY_Trans to declare this attribute.


 But perhaps there is a better option:

A new SysSetting table - ~NOPCOL. Veda would ignore the columns with headers listed under this tag. In the attribute approach, we would need to clean up the sync log warnings at the end of processing.

[Antti-L]

Thanks, that SysSettings option does indeed sound very good to me 

[AKanudia]

Great, we will do it in the next update.

[AKanudia]

This feature has been introduced in version 3.1.2 (13 Jul 24). You can list the col headers to be ignored in a ~NOPCOL table (col header = name).

This will ignore the cols provided it is not a valid col header for a particular tag. For example, if you list "Sets" in this table, then these columns will be ignored in FI_T tables but NOT in FI_Process/Comm tables.