"Out of global data memory". Is there a way to increase the memory?

leandroct
2022-03-25
2022-03-28
  • leandroct - 2022-03-25

    I use Codesys Control Win MC 3.5 SP16 and I created in my existing application two new arrays of [600,3000,650] OF BOOL. When I try to download the application, it shows up an error message "Out of global data memory. I tried clean all and retried the download with no sucess and I deleted persistent varibales too. Is there a way to increase the memory? My PC has 16Gb of memory and it has memory available. Where I can access the configuration to chance this? Thanks!

     

    Last edit: leandroct 2022-03-25
  • fajean - 2022-03-26

    That is a pretty big array. It has 1170000000 entries. If you use a 32-bit runtime, I assume each BOOL eats up a whole 4-byte word, so this works out to more than 4Gb of memory, which would typically be the limit a 32-bit process can address under the best circumstances.

    If you are using a 64-bit runtime, this doubles the memory used because memory is addressed in 8-byte words. I am not sure a 9GB+ contiguous array should be expected to fit in a machine with 16GB of memory, but if it does there sure is little room to grow.

    Maybe you should pack your BOOL values as single bits, which will obviously require a bit of fun with bitwise operators and translating "bit addresses" to byte+bit parts. You would increase your density by a factor of 32 or 64, which should easily fit in your available memory

     
  • sgronchi - 2022-03-28

    A BOOL is stored as 1 byte, 8 BOOLs are stored in 1 64-bit word or 2 32-bit word.
    But even if you pack 8 BOOLs/byte you have 1,4 GB of data for each array. And this is a giant code smell for PLC code. That's why the default size is 4 kB (almost always you need less than 1 kB), not including visualization which gets a separate area.

    If I guessed correctly your target application, at least 99.99% of these BOOLs will always be FALSE. Perhaps you should look into sparse matrices representation?
    Anyway, this post is a clear example of the XY Problem: https://xyproblem.info/ so please detail what is the original problem you are trying to solve.

    For reference:
    You can override runtime memory size by right clicking on Application -> Properties -> Target memory settings -> Override memory settings, but the max you can go is 2 GB (you'd need at least 3 GB with those arrays).

     
    • fajean - 2022-03-28

      A BOOL is a 1-byte value, but the CPU reads and writes memory as 4-byte values (32-bit) or 8-byte values (64-bit). The actual space occupied by a BOOL in a structure will depend on the packing mode. If BOOL arrays use single byte alignment, this means CODESYS performs byte-manipulation under the hood, which is certainly possible. I would like to know if this is documented somewhere, and a reliable behavior irrespective of platform.

       
  • sgronchi - 2022-03-28

    https://help.codesys.com/webapp/_cds_pragma_attribute_pack_mode;product=codesys;version=3.5.16.0

    1-byte variables are always packed at byte addresses irrespective of the pack mode.
    You can verify experimentally for yourself (I did it on x86, x86_64, ARM Cortex-A and PowerPC e200z7).

     
    • fajean - 2022-03-28

      You are right, my comment was based on an incorrect recollection of packing mode behavior.

       
  • leandroct - 2022-03-28

    Thanks to all by the answers.
    - I use 64 bits Codesys.
    - I saw the Target memory settings that you said, sgronchi. Thanks! But really it can’t reach my needed memory even if I increase my PC memory, if I understood.
    - To explain about why I want this big array, my application does a 3D reading of an area with size x(600cm), y(3000cm) and z(650cm). To create a logic that navigate in the space to analyze the objects and surfaces is very hard to me unless I could put the read points in a 3D array. In this case, each BOOL representing 1 square cm is enough for me. The BOOL array TRUE values would vary, but I think 5 to 10% is a default we can imagine.
    - Interesting the Sparce Matrix idea. I can think about it.
    If I have no solution by this way, then I will need to think in a new way. But my main question here was answered by sgronchi. Thanks to all!

     
  • sgronchi - 2022-03-28

    So I guessed correctly that you were trying to build a point cloud (and that's where the "at least 99.99% will always be FALSE" came from ;-) ) and that's exactly the reason why they're always stored in a sparse matrix (that could range from a simple linear array storing coordinates to more complex data structures).

     
  • leandroct - 2022-03-28

    I couldn't imagine a logic that would analyze the readings next to the current point, in all 3D directions, without having to convert the sparse matrix into a 3D matrix before processing (only in the region analyzed at the moment). That would solve it, but I'm guessing I'll have processing speed issues. But I will analyze. Thank you again.

     

Log in to post a comment.