read big csv file

hki75
2017-04-05
2017-05-15
  • hki75 - 2017-04-05

    in order to read a csv file, I usually use the FILE.Read FB from File from CAA_File.Library

    The read data is stored into the the ADR(buff) area: how get big amount of data? (buffer size >xxxxxxxxxxxxx): Get multiple file-read?how to split reading operations?

    IMG: 05

     
  • tonverra - 2017-05-11

    Hi!

    If I may hook into your thread..

    Do you have any luck with this?
    Are you able to do this with smaller csv files?
    Where do you store your csv?

     
  • josepmariarams - 2017-05-12

    Hi.

    For reading csv, first i ask about csv size using sysfile.getsize, after that i make a __new(byte,size+1) over a byte pointer.

    To read values, you can create an array of arrays of pointer to string (first array are rows). You can go over the big array pointing the strings after field separators and subst the field separator by an 0. (End line changes first array).

    You can put all inside a fb and acces fields via get(x,y).

    Don't forget to destroy dynamic memory previous a new read and destroy it in fb_exit.

    To write csv, the dynamic array is a little bit mor complicated.

     
  • tonverra - 2017-05-14

    Thanks for the explanation!

    Any chance you could post some example code/project?

    I got it to work, but am looking at individual bytes (ASCII) and combining/converting those to INTEGER values. I am sure there are more elegant ways of doing this..

     
  • josepmariarams - 2017-05-15

    Hi.

    this is a simplified example:

    I recomend to you that a buffer be in a separate FB which manages the construction and destruction of it,

    The same for the double array, an FB with the double array and get and set methods to control array bounds.

    In array creation add 1 byte more to put the last string 0.

    IMG: Captura de 2017

     
  • tonverra - 2017-05-15

    Thank you so much!

     
  • josepmariarams - 2017-05-15

    Hi.

    I made some mistakes on last code (in _new):

    That is a resume of my code. You have to manage errors on case of reading fail. And at the end parse code over strings is not filled.

    I have not test it, it could contain some mistakes,

    IMG: Captura de 2017

     

Log in to post a comment.