#21 Future of cforge package

1.0
open
nobody
None
2020-04-16
2020-01-18
Ingo
No

What do you think about the current structure and handling of the package?

Shouldn't we put the SVN specific commands, and later the Git commands into hooks of Tortoise SVN or Tortoise Git?

Could improve the handling of repositories and the setup enormously in my opinion.

Hope to here some feedback and ideas from you.

Discussion

  • aliazzz

    aliazzz - 2020-01-18

    Hi, just some thoughts:

    • GIT integration for cForge (and a plugin for GIT in CODESYS IDE?) would be very very favourable!
    • Extend cForge with Webhooks to let any builder (arbitrairly which or where) start a job very cool!
    • Generating .CHM and or .PDF documentation (with choice of a few cool prefab documentation frames) from the reSructured text in a project or Library
    • Letting cForge Tool run a custom user made script from windows explorer with a right click (an "execute custom" command). What the script itself does is up to the user, as long as it combines cForge and CODESYS in python.
     
    • i-campbell

      i-campbell - 2020-02-03
      • "Git integration for cForge (and a plugin for GIT in CODESYS IDE?)" @aliazzz
        So the plugin comes with SP16. Do you think any prelim work is needed, ie adding some CODESYS independent GIT things? Sure, this feature with cForge is a must after SP16. For me, I think GIT is fundamental for the collaborative nature of open source, but until SP16, a wiki article is enough so we don't have to start again in May.

      • "Generating .CHM and or .PDF documentation" @aliazzz
        Yes use of the libdoc tools gets my vote too! I know the built in libdoc with CODESYS does not have a pdf as standard (any more). Perhaps there is some external tools that can be added to achieve this. Perhaps in the long future there is also scope for an open source libdoc like tool.

      • "run a custom user made script" @aliazzz
        Are you meaning a call to this command line switch run CODESYS script from command line? Or do you intend a broader scope than just this? The first one definitely has my vote.

       
      • aliazzz

        aliazzz - 2020-02-03

        Hi,
        On point1:
        Git integration ETA SP16, nice!

        On point 2:
        Currently, generating a .pdf output is cumbersome, to say the least, due to frames, latex and python dependencies, etc, etc.
        This, while generating a .chm file can be generated via a single command. But only if you read the .libdoc documentation carefully, then you can spot that specific command. Maybe documentation has improved over time, I have not checked it since over a year. Anyway, both a themed .chm and a .PDF are high on my bucketlist, but one must start small ;-)

        On point 3:
        I mean your first option: just calling any arbitrarily written python script from and run it against the CODESYS IDE through a right mouse click over while selecting it and choose "cforge -> run script" or something similar like this...

         
  • Ingo

    Ingo - 2020-01-31

    Thanks for your input Aliazzz!
    Thats very valuable.

    Regarding the webhook:
    Did you already try them? Allura supports webhooks for the code repositories:
    https://forge-allura.apache.org/p/allura/wiki/Webhooks/

    Or are you missing s.th.?

     
  • i-campbell

    i-campbell - 2020-02-01

    I have kind of been thinking on this on and off, so haven't until now commented.
    My view was to look at:
    1. what types of cForge users exist,
    2. what skills they may have / may lack,
    3. and then what tasks each type of cForge users should want to perform.

    It is my opinion that open source development and its associated workflows are new to many automation engineers. I think the short term development of the cForge tool should focus on guiding the user to complete tasks.

    1. Types of users:
      1.a admin role for forge.
      1.b Contribute new projects intending others to contribute.
      1.c Contribute new projects with no intent to seek contributions.
      1.d Contribute code to existing projects.
      1.e Contribute non-code to existing projects, such as documentation or issue management.
      1.f Take some existing code and modify it but don't contribute.
      1.g Wanting to spend as little time in forge as possible, just wanting to grab the driver or lib they want.
      1.h Just wanting to talk on the forum.

    2. Skill/Experience
      I think that some people will have varying levels of each of these skills:
      2.a Computer programming
      2.b Version control systems
      2.c Testing
      2.d CODESYS
      2.e PLCs
      2.f open source communities and issues
      2.g Electronics
      2.h Physical applications

    3. Tasks and workflows
      So for me the cForge tool should be aimed at users 1.b through 1.e (hereafter referred to as contributors). The main goal is to help contributors to make their projects as accessible as they want it to be for users 1.f through 1.g (hereafter referred to as audience). A secondary goal is to help the contributors make their projects easier to contribute to.

    So we have the documentation [forge:wiki:Home] which describes some workflows.
    Should CODESYS be a prerequisite? Well, what if the project doesn't have any CODESYS code? for example an open source python script to test modbus comms or something?
    So I think the very first getting started step is install cforge.exe.
    Then I would like cforge to have a developer readiness checklist:
    - Install CODESYS IDE (link to https://store.codesys.com/codesys.html, or maybe link to a forge page with some installation tips and tricks? both?)
    - Install SVN (link to store or forge)
    - Install Tortise SVN
    - Install cforge.package (will be seperate to cforge.exe)
    - Set path variables
    - Link to OAuth credentials
    - etc.
    The checkboxes can be marked by the contributor as TO DO, WON'T DO, NOT PLANNED, DONE etc. Maybe a bonus if we can autodetect some of them like isCODESYSInstalled.

    Then there will be some workflows, once the developer is satisfied with their PC's readiness:
    * Start new forge project
    - Driver project, lib project, hack?
    - Just under my user page for now? or on the main directory?
    - do you have files already, or do you want a blank folder to start with?

    • Link existing forge project

    Then there will be a project readiness checklist, similar to the development machine readiness checklist.
    * SVN or Git repository created
    * If you're at work, do you have permission from your boss to open source this?
    * used external libs correctly (link to info on maintaining copyright etc)
    * extended someone else's work correctly (link to info)
    * License selected and added (link to help on selecting a license)
    * Is there a readme.md (button to create one and open in text editor)
    * Is there contrib guidelines
    * Package Manifest exist (button to create one)
    * Package exist (button to create one)
    * package uploaded to forge.codesys.com
    * Wiki exists on forge.codesys.com
    * Link exists on the Project Home page of project to download package
    * Link exists on the Project Home page to discuss project in forum (or maybe search forum for discussions related to the project)
    * etc.

    Perhaps a button to initiate a check for some of these things. For instance it could check if the manifest is up to date, or if the package installs successfully.

    Each task could have a "search for topic in forge docu / forge forum / help.codesys.com / google". A "Discuss topic in forum" button could create a Nicely formatted Discussion topic, with some key info already filled in, eg. cForge version, codesys version, cForge workflow item, Developer readiness %?. It may even have a template such as some subheadings [Background] [Question]

    Anyway thanks for listening to my rambling. If this direction for cForge is of interest I am happy to contribute further.

     
    • Ingo

      Ingo - 2020-02-02

      Hi Ian,

      You crazy guy! ;)
      Very nice analyzation of the user groups and tasks they want to achieve. I used your lists to try to form use cases and defined even three groups of actors, who want to complete those use cases:

      Beside the contributer and the consumer, I defined the biggest group of users. Those who only search for informations and tipps to solve their own problem in their proprietary software.

      My thought is, that we can then sort those use cases into the different user interfaces:

      • web page
      • app
      • cforge tool

      Because I believe that you assigned some features to the cforge tool, which are better solved in other places. Especially because also other user groups want those features, who don't have the cforge tool.

      Please check it out. I am looking forward for your feedback.

       
      ๐Ÿ‘
      1

      Last edit: Ingo 2020-02-02
  • i-campbell

    i-campbell - 2020-02-02

    I love the direction!
    So I think from this:
    1. web page can generate an item from a series of templates, maybe call the item type "Worflow instance" or some other well thought out name. "Workpiece" might fit in well with the forge metaphor
    2. contributors can make new templates and add them to some central repository (eventually a template for the template workflow ;))

    3. Maybe a "Do" button to start a new workpiece or work on an existing one

    4. cforge.exe should not be a .package but a standalone executable. It should ideally be able to help with the installation of the IDE, but also eventually we will want it to run on a linux device for troubleshooting CODESYSControl.cfg problems etc
    5. the templates can work with cforge:// links somehow to query data about the machine
    6. possible to create a workpiece without logging in. It won't save it though. Might be handy for "find something and download and install" workpieces.
    7. Maybe an option for showing only CODESYS Group approved work items...
    8. Can create individual workpieces under /u/username, or collaborative workpieces attached to a project (for example /drv/mcp3008).
    9. android app, I have no experience for. Sure it will give a subset of the web page features. Possibly subscribe to a notification when a task of a workpiece is complete.

     

    Last edit: i-campbell 2020-02-02
  • aliazzz

    aliazzz - 2020-02-02

    Another possibility could be to let the cforge executable act as a remote-builder on the system it is installed. Maybe even a platform independent technology suitable for at least Windows and Linux (Mac is debatable for lack of any CODESYS runtimes at this moment).
    This way the new cForge executable (could still be installed via a .package on windows) bridges the gap between a machine anywhere on the Internet and this web based platform in a secure manner.
    The executable can interact with any software installed on the local machine (via scripts etc) but also with the remote host i.e. cForge platform via Git/SVN through webhooks to trigger remote builds on the machine the remote builder runs on. For extra clarity, interaction with the builder is triggered from this platform and scripts are pushed down.

    "Remote" Builder for CI/CD scenario's;

    Hosted GIT/SVN repo <-> Trusted executable on Win/Lin <-> CODESYS

    Something like this is allready done by Jenkins, GitLab and Drone and is very very scalable and proven technology!

     

    Last edit: aliazzz 2020-02-02
  • i-campbell

    i-campbell - 2020-04-09

    Hi Team!
    So since February, a lot has happened!
    1. forum integration.
    2. I made my first forge.codesys.com/prj/
    3. I collaborated with aliazzz on my second forge.codesys.com/prj/
    4. lots of people are working from home.
    5. SP16 is sooo close!
    6. So many more public and private things for everyone.

    Some points I have come up with:
    1. It took me 4 hours to get my first project from "I have a .project" to "I have a /prj/ people can access". I think I missed a few steps, but it is there. I would like to get that down to half an hour for first time forgers, with ten minutes possible for repeat forgers. While I think it is important that our forge works with the existing users tools / workflows, we should have some super simple "UI" which asks the minimal questions for: 1. converting a .project and .library to a forge project (filling in the appropriate descriptions, licences etc., adding a Ticket tool...) 2. Starting work on a change (branch), either as project member or non project member (have to clone into forge/u/). 3. merge request 4. merge. This UI should then be documented in the "minimal steps with minimal choices" section.
    2. Collaboration tools are needed. Ideally we would recommend (or host!) a standard communication toolset.
    a. As a test, I setup a slack for collaboration, but I don't think that is optimal for open source use. Looking at other OSS projects, I see irc (specifically irc.freenode.net) as a common "chat". One of the tools for a forge/prj/, could be adding the CODESYS forge bot to their irc, and thus archiving the chat logs. I think it should be made clear that chat is best for "we need to "sit in the same room and discuss this"" whereas [talk] or [tickets] is best for "Please take time to consider and respond when you have time". There is an interesting discussion on the use of the two tools here. Having tried MS Teams and Slack, I think irc is the least intrusive to people's workflow, as with irc you are free to choose your own client. It also has the best licencing model.
    b. Additionally, some sort of webinar like software is needed in the toolset. Again, I would prefer an OSS solution, such as the self hosted openmeetings.apache.org. Again, the recordings could be archived with the forge/prj. Projects could book the CODESYS Forge OpenMeetingRoom (or, as the forge expands, MeetingRoom2 ;)), and interested parties could look at the calendar and see "oh, I see the cforge project is having a scrum meeting this weekend, I'll watch along". Obviously, projects are free to pick their own toolsets.

     

    Related

    Tickets: tickets

  • aliazzz

    aliazzz - 2020-04-09

    a. As a test, I setup a slack for collaboration, but I don't think that is optimal for open source use. Looking at other OSS projects, I see irc (specifically irc.freenode.net) as a common "chat". One of the tools for a forge/prj/, could be adding the CODESYS forge bot to their irc, and thus archiving the chat logs. I think it should be made clear that chat is best for "we need to "sit in the same room and discuss this"" whereas [talk] or [tickets] is best for "Please take time to consider and respond when you have time". There is an interesting discussion on the use of the two tools here. Having tried MS Teams and Slack, I think irc is the least intrusive to people's workflow, as with irc you are free to choose your own client. It also has the best licencing model.

    In general my mantra is: Keep It Simple Stupid. So, for this moment, a cForge built-in web chat client would be my personal ideal.

    Let me paint my vision for you;
    A permanent chat would be accessible only if you are logged into cForge (a chat button next to search).
    Your cforge username acts as the your personal chat nick automatically.
    The chat will have some "prefab" channels which you can join;
    projects,
    libraries,
    drivers,
    chat,
    * etc.
    If a user wishes, he/she can open a private channel with one or more others. No need for installed software,
    plus the log can be saved as a text file if needed...

    The chat can add an extra dimension in support question handling, if CODESYS is open for this....
    Many commercial companies do this type of communication allready.

     

    Related

    Tickets: tickets

  • Ingo

    Ingo - 2020-04-09

    Proposal:
    We plan a jitsi meeting to discuss and priorize this stuff.
    What do you think?

     
    • i-campbell

      i-campbell - 2020-04-09

      Great Idea, I could make any time this long weekend, but after 8pm is easiest.

       
      • Ingo

        Ingo - 2020-04-12

        Hi Ian,
        if it's just us, lets discuss this via teams next week during worktime.

         
        • i-campbell

          i-campbell - 2020-04-16

          Minutes of short meeting:
          Present: Ingo, i-campbell
          From the diagram in the above comment we should focus most on the role of the "contributer". And that we should make the start for the potential contributer, who is new to "open source", "version control" and "continous integration", as easy as possible. This indirectly helps the other user types.

          • Make contributing easier

            • cforge tool should provide the "Wizards" for, for example committing your first project. This should build on the "notes" Ian took committing his first project Publicise notes (TODO: TICKET)
            • Installation of cforge should be smoother. (TODO: TICKET)
            • Integrate SVN commands seemlessly into tortoise svn (TODO: TICKET)
            • To assist those new to open source, a more rigid (recommended) workflow should be documented and encouraged. This will read more like a tutorial or how to. The current documentation documents the platform, and was written when the forge was younger. (TODO: TICKET)
            • A Tube-tutor video should be made, of the typical workflow for collaborating on a Forge project. (TODO: TICKET)
          • Get the community more involved in the governance of forge.

            • Solicit ideas and thoughts on the recommended workflow on forge/talk/ (TODO: FORUM POST)
            • As a lower priority (i.e. for the future), document the governance model, ie, how are decisions about the platform made and documented. how are ideas presented, etc. how are individual projects governed. (TODO: BACKLOG TICKET)
            • Consider monthly(?) "Town Hall" meetings. Think up a coller name though, maybe to do with forges. (TODO: TICKET)
          • Make collaboration on projects easier:

            • Investigate integrating a user chat possibility. Eg private chat. Discussions like this one (the meeting, not the ticket). project meetings. Could include chat / screensharing/video/audio. eg. XMPP server or Jitsi server. Ideally uses same login, rather than having to sign up for yet another account somewhere. Ideally open source. Decide on neccessity of chat log archival. (TODO: TICKET)
           

          Last edit: i-campbell 2020-04-16

Log in to post a comment.