I want to write a string to a file from a specific position. I use SysFileSetPos() from SysFile library to set the position. The thing is it works only when you open the file with write mode, and with write mode, It meanwhile overwrites the original content. I tried to use Append mode, but it always write to the end of the file.
If you need to modify your file data, access_mode should always be write.
Then, your problem is about inserting some bytes in your filestream at specified position.
You may then create a new file, write the part of your file before the position where you will insert. Then write your new data, then write the lasting data from previous file.
Delete old file and rename new.
That's no point elegant but may solve it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have also found this to be an issue. I am creating a function block to create a .CSV file using the SysFile.Lib but also want to implement code so that it is possible to log data in a ring buffer format.
The principle is that once a maximum entry limit is reached the last entry will be removed by truncating the file. This works no problem, however, because appending to files in a specific position is not possible the oldest data stays and the latest entry in the file changes each time the code is executed rather than buffering. If there was a way to append to a file at a set position then the code would work nicely.
My only other option is to use the method described by dFx. Whilst I'm sure it will work, it will be very intensive on processing time and will likely decrease the rate at which the code can log data. At the moment the code can be reliably processed in under 25ms giving a possible sample rate of 40Hz, which would be a major advantage over current solutions I have if I could append at a specific location.
Does anyone know of a way to do this? Perhaps a newer version of SysFile.lib or a different library will allow this?
Many Thanks
John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I believe that the Datalog manager functions like a ring buffer if you set the limit as a total number of entries when logging to an sqlite file. However I'm not sure if it does that when using the csv functionality.
You could instead use the sysfile library to move the log based on some condition and maybe append a datestamp to it. Then have it delete files that are older than a certain time. Kind of like a conveyor line of files that feed into a pit of deletion.
The datalog manager will create a new log file even if the old one is no longer present. Which contrasts with the Trend's logging functionality which will fail if the correct file can't be found or is corrupted.
The datalog manager uses attributes to figure out what variables to log and how to log them though. I'm not sure how well that works with your function block approach but I think that there would be a way.
Last edit: Morberis 2020-08-26
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After having a look at the Datalog manager it seems as though it is a new feature of SP16, so perhaps I could download the new version and try this. I'm not sure if my target device will work with it but it's worth a look for sure. I'm fairly familiar with using attributes so if this method is possible for my target then maybe this is the way to go with my FB.
I have also noticed that the CAA File library doesn't seem to use the functions of SysFile, so perhaps I will be able to set a position for appending to a file using this library. I'll post here if I have success.
I have considered the method you've suggested as I'm sure it would work. My problem with the method is that there would be two log files for one 'log session' as such. If there have been many different logs generated it could get a little messy! definitely a good back up plan though.
Thank you for your help so far!
John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The datalog manager isn't new to SP16 so it might be worth checking to see with the version you're using. I just linked to the SP16 help file because it's the most recent SP.
Good catch on the CAA File vs SysFile libraries.
Last edit: Morberis 2020-08-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes after a bit more investigation I can see that it is a module that must be added, whereas I was trying to add it to my application. I will have a go!
I've had a little bit of success with the SysFile library, but unfortunately it still doesn't help my problem! If I use the mode 'AM_READ_PLUS' this allows me to write data to a specific location after using the SysFileSetPos function as I had hoped. The problem now is although I can now write in a specific location, any data from that point equal to the write size is overwritten.
I've tried to code around this issue but nothing I've tried has worked. All seems to point to either buffering the data locally and writing it to a file in one go, or writing to two files alternately.
I am still yet to try the CAA File library, although I have a feeling I will get the same result - hopefully not!
John
👍
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I want to write a string to a file from a specific position. I use SysFileSetPos() from SysFile library to set the position. The thing is it works only when you open the file with write mode, and with write mode, It meanwhile overwrites the original content. I tried to use Append mode, but it always write to the end of the file.
So Is there any better solution for this.
more posts ...
If you need to modify your file data, access_mode should always be write.
Then, your problem is about inserting some bytes in your filestream at specified position.
You may then create a new file, write the part of your file before the position where you will insert. Then write your new data, then write the lasting data from previous file.
Delete old file and rename new.
That's no point elegant but may solve it.
I have also found this to be an issue. I am creating a function block to create a .CSV file using the SysFile.Lib but also want to implement code so that it is possible to log data in a ring buffer format.
The principle is that once a maximum entry limit is reached the last entry will be removed by truncating the file. This works no problem, however, because appending to files in a specific position is not possible the oldest data stays and the latest entry in the file changes each time the code is executed rather than buffering. If there was a way to append to a file at a set position then the code would work nicely.
My only other option is to use the method described by dFx. Whilst I'm sure it will work, it will be very intensive on processing time and will likely decrease the rate at which the code can log data. At the moment the code can be reliably processed in under 25ms giving a possible sample rate of 40Hz, which would be a major advantage over current solutions I have if I could append at a specific location.
Does anyone know of a way to do this? Perhaps a newer version of SysFile.lib or a different library will allow this?
Many Thanks
John
I believe that the Datalog manager functions like a ring buffer if you set the limit as a total number of entries when logging to an sqlite file. However I'm not sure if it does that when using the csv functionality.
You could instead use the sysfile library to move the log based on some condition and maybe append a datestamp to it. Then have it delete files that are older than a certain time. Kind of like a conveyor line of files that feed into a pit of deletion.
The datalog manager will create a new log file even if the old one is no longer present. Which contrasts with the Trend's logging functionality which will fail if the correct file can't be found or is corrupted.
The datalog manager uses attributes to figure out what variables to log and how to log them though. I'm not sure how well that works with your function block approach but I think that there would be a way.
Last edit: Morberis 2020-08-26
Hi Morberis,
Thank you very much for your quick reply!
After having a look at the Datalog manager it seems as though it is a new feature of SP16, so perhaps I could download the new version and try this. I'm not sure if my target device will work with it but it's worth a look for sure. I'm fairly familiar with using attributes so if this method is possible for my target then maybe this is the way to go with my FB.
I have also noticed that the CAA File library doesn't seem to use the functions of SysFile, so perhaps I will be able to set a position for appending to a file using this library. I'll post here if I have success.
I have considered the method you've suggested as I'm sure it would work. My problem with the method is that there would be two log files for one 'log session' as such. If there have been many different logs generated it could get a little messy! definitely a good back up plan though.
Thank you for your help so far!
John
No prob.
The datalog manager isn't new to SP16 so it might be worth checking to see with the version you're using. I just linked to the SP16 help file because it's the most recent SP.
Good catch on the CAA File vs SysFile libraries.
Last edit: Morberis 2020-08-27
Yes after a bit more investigation I can see that it is a module that must be added, whereas I was trying to add it to my application. I will have a go!
I've had a little bit of success with the SysFile library, but unfortunately it still doesn't help my problem! If I use the mode 'AM_READ_PLUS' this allows me to write data to a specific location after using the SysFileSetPos function as I had hoped. The problem now is although I can now write in a specific location, any data from that point equal to the write size is overwritten.
I've tried to code around this issue but nothing I've tried has worked. All seems to point to either buffering the data locally and writing it to a file in one go, or writing to two files alternately.
I am still yet to try the CAA File library, although I have a feeling I will get the same result - hopefully not!
John