#21 Payload Structure -> Metric Structure -> ...

999 Backlog
closed
nobody
None
2020-09-14
2020-04-04
i-campbell
No

So looking at the attached .txt, you can see that each metric can have:

recursive properties

So in theory you could receive a payload: Payload.metric[0].properties.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].propertyset_value.values[0].string_value 'ad finitum'

recursive metrics

Same for: Payload.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0].template_value.metrics[0]

a large 'static' address space for decoding Payload Blobs

Take for example "Payload.metrics[].dataset_value.rows[].elements[].string_value"
This is fine for encoding, because you can build only the array elements you need.
If we limit the decoder statically (even with values set in the user application), to allow any of 64 incoming metrics to have a dataset with 64 rows and 10 columns, which any could be allowed to be string(80)s, would require 81 x 10 x 64 x 64 = 3MB just for that address. repeat for the others, allow for a few more rows of data, and you've quickly filled your memory.

So this ticket is to discuss the structure of the memory management. Some options I can currently think of:
1. Convert to JSON or XML using one of those libraries, and then use the same JSON or XML library to access the data
2. Each Payload has a metric pool array, properties pool array, dataset pool array, template pool array. Rather than a metric having his own arrays, he just has index values to the pools.
3. Use one of the FB_Factories to allow dynamic creation and distruction of FB instances.

I will put this ticket in the 999.0 backlog, and for milestone 1.0 FB_Payload will only support those fields listed in the ticket [#11]

1 Attachments
Payload.txt

Related

Tickets: #11

Discussion

  • aliazzz

    aliazzz - 2020-04-05
     

  • aliazzz

    aliazzz - 2020-04-05

    Ticket [#11]: Large 'static' address space for decoding Payload Blobs

    Take for example "Payload.metrics[].datasetvalue.rows[].elements[].stringvalue"
    This is fine for encoding, because you can build only the array elements you need.
    If we limit the decoder statically (even with values set in the user application), to allow any of 64 incoming metrics to have a dataset with 64 rows and 10 columns, which any could be allowed to be string(80)s, would require 81 x 10 x 64 x 64 = 3MB just for that address. repeat for the others, allow for a few more rows of data, and you've quickly filled your memory.

    I'd opt for static limit of 32 devices per address.
    We could opt for a user dependent max setting (library parameter) and default it to 32.
    Reason is memory usage. Plus 32 devices behind a single Node is quite a few devices if you ask me...
    If needed, the setting can be changed by the user. Then their mileage may vary.

    Ticket 999 options

    1. Convert to JSON or XML using one of those libraries, and then use the same JSON or XML library to access the data
    2. Each Payload has a metric pool array, properties pool array, dataset pool array, template pool array. Rather than a metric having his own arrays, he just has index values to the pools.
    3. Use one of the FB_Factories to allow dynamic creation and distruction of FB instances.

    Option 3 seems the nicest too me. I am open to this option but I am hesitant towards long term usage in an industrial environment stability wise. Using it in a lab environment should be no issue at all.

     

    Related

    Tickets: #11

  • i-campbell

    i-campbell - 2020-04-19

    Payload has LinkedList of PayloadMetric
    PayloadMetric has propertyset, which is a linked list of keys + a linked list of values. Maybe key+propertysetvalue pair?
    propertysetvalue has propertyset, which is a linked list of keys + a linked list of values. Maybe key+propertysetvalue pair?
    propertysetvalue has linked list of propertyset
    PayloadMetric has dataset, which is a linked list of cloumns + linked list of types + linked list of Rows.
    Rows are a linked list of elements.
    Metrics has a template, which has a linked list of Metrics + linked list of parameters

    FB_Payload has pool of metrics + pool of keys+pool of propertysetvalues + pool of propertysets + pool of columns + pool of datasettypes + pool of rows + pool of elements + pool of parameters

    Extends LinkedListElementBase
    METHOD FB_reinit : BOOL VAR_INPUT END_VAR

    UpdateElement(itfElement := THIS^);

     

  • aliazzz

    aliazzz - 2020-04-20

    A very good solution to implement a linked list!

     

  • i-campbell

    i-campbell - 2020-05-18
    • status: open --> closed
     

Log in to post a comment.