Web Visualization and Trending Data from CSV file

tvm
2017-03-23
2019-03-05
  • tvm - 2017-03-23

    I have a question about web visualization in Schneider SoMachine, which is basically Codesys and Schneider's HMI software wrapped up together. I am storing .csv files to the flash memory using the CAA file library, which I would then like to plot on a graph in a web visualization.

    The problems with Web Visualization:
    1. SoMachine doesn't implement the Codesys Trend Manager
    2. I could use a histogram or a trace, but they have their own issues--the histogram doesn't seem to allow changing options at run time (scale, data set, etc), and the trace seems more for short term graphing, not historical data sets.
    3. Some of my .csv files are 1440 rows (once per minute, one file per day) and 20 columns. That's a lot of data for the web visualization to handle.

    Partial Solution:
    1. I had some old code that took a .csv file and displayed it in a trend on a webpage using javascript, using a library called dygraphs. It works if I upload it to the /usr/visu directory and reference it directly. I even created a sort of wrapper to load my trend webpage, as well as the visualization page, and even other external pages. Basically, I get the file through a javascript AJAX request, process it, and display it using dygraphs. This all works.
    2. The problem with this solution is that the compact flash card is mounted at /sd0, but the webvisu root directory is /usr/visu, therefore, the web visualization can't see the compact flash card because it can't see any files above it's own root directory. This leaves me with only the internal flash memory to store files, with a total size 128MB on the Schneider M241/M251 PLC. I want to be able to store, and then access the files on the compact flash card from the web server.

    Questions:
    1. Is there any way to change the root directory of the web visualization? That would be the simplest solution--just put it all on the compact flash card.
    OR
    2. Is there any way to access the compact flash card from the web visualization? I can actually load a file from the flash card into the visualization text editor, but it looks like it's using an internal protocol to get the file through the PLC, and I don't know how that protocol works.
    OR
    3. Is there a way to get the Codesys trend manager into SoMachine 4.2?

    thanks for your input

    Tim

     
  • Anton Bolander - 2017-06-18

    Hi!

    I would also like to see a solution for this. However I am really interested in your Partial Solution since I have no solution at all. Is it possible that you would like to share the code for that? Do you embed a code into a visualization page, what kind of wrapper do you use?

    I have talked to Schneider about having the trend option in SoMachine but they did not sound that enthusiastic..

     
  • tjarcoboerkoel - 2018-11-07

    Hello Readers,

    I'm bumping into the same issue.
    I need long term tracing of PV systems, log them and plot them into a graph/history of which I can scale the X and Y as (_ etc).
    Is a solution found for this?

    Could also do the trick by 'loading' the data on a WebVisu event from the csv file to a databuffer, running some ST code with the desired interval and timewindow.
    I've seen that the WebVisu has a file copy task for events but I don't know what its permissions are (won't be realtime).

    Another option would be a third party script/webpage. I mean some HTML implementation of, for example, HighCharts. You could just insert a custom HTML page on you PAC controller and make an EVENT to load the (local) URL (as popup), and pass the CSV to the HighChart page. HighCharts has _ capabilities and different chart presentations.

    What you could also do is a array buffer as FIFO, just use the ARRAY ROTATE and insert the new values at the end.
    For SoMachine it's the function FC_RolArrReal(i_iShiftPosNumber: INT, i_prStartAddr: POINTER TO REAL, i_bySize : BYTE ).

     
  • tvm - 2018-11-08

    What I ended up doing was paying a developer to write a Java applet that actually gets the file from the SD card through the ftp server on the PLC, then embedded custom web pages in the visualization on the internal memory. It's a total hack, and I hate everything about it, but it works. For now. As long as I don't upgrade Java, or the browser, or...

    The only reason I need the SD card is storage space. If all your files fit within the 128MB of internal memory on the M241/251 controllers, then it's pretty easy. I used dygraphs.com and wrote a bunch of javascript to make it work--basically get the file through the http server, convert it to an array, and display it. The problem for me is 128 MB isn't enough.
    I've been thinking that it should be possible to copy a file from the SD card to the internal memory, and then get that file and display it. Copying the file is easy, I can do that with the CAA file library. Displaying the graph is easy, I have all the javascript code for that written. The connecting link is figuring out a way to triggers a variable in the PLC, and then read back a bit to say that the file is loaded. And I haven't figured out how to do that through a javascript/custom html page yet.

     
  • tvm - 2019-03-05

    This post seems to have generated a lot of views, so I thought I'd post my latest solution:

    1. Create a separate HTTP server on the PLC . Schneider has a pretty good example program for a TCP server, and I built a limited HTTP server on top of that. There's also some stuff in the OSCAT library. This HTTP server operates on a different port than the visualization, so it will have to respond with Access-Control-Allow-Origin: * as a header in order to allow Cross Origin Resource Sharing.
    2. create a custom HTML file which includes an iframe that points to a web visualization. A web visualization will have to be set up in the PLC, and it will create a default webvisu.htm file (or whatever you want to call it), but the custom HTML file will need to be a different name--for example x.x.x.x:8080/index.htm
    3. the web visualization contains a file selector, and a get button, created with the normal visualization tools. When the get button is pressed, files are copied from the compact flash card at /sd0 over to /usr/visu/temp/ (from the web perspective this will be /temp/, from the PLC's perspective it's /usr/visu/temp/). This is all done on the PLC with the CAA File library, and it's actually a lot faster than I thought it would be--about 1 second for 4 files of ~56K
    4. Once the file is copied, in the PLC create a JSON string which contains information about the files copied, and a timestamp. This is then made available to the HTTP server as a file (graphconfig.json)
    5. The custom HTML file GETs the graphconfig.json from the HTTP server every 1 second using Javascript/AJAX. Convert that string to a JSON object.
    6. When the timestamp in the JSON object changes, check the rest of the object because it will contain information about which files are in the /usr/visu/temp directory.
    7. Get those files, in my case they're CSV format--parse them into arrays or objects usable to the graphing library, in my case dygraphs.

    This may seem like an overly complex solution. In my case I had created the HTTP server and the JSON library (which I have released publicly elsewhere) already, and I had done some work with the CAA File library already, so it was a matter of putting them all together. The javascript/HTML was quite a bit of work, mostly because I'm not really a web programmer, but I had some of it written from a previous project on a different PLC platform.
    I won't be releasing the code, because it represents a significant amount of work. But maybe the general direction is useful to someone. Sometimes knowing that it's possible is a good start.

     

Log in to post a comment.