#84 Sparkplug: conform to Technology Compatibility Kit (TCK)

pending
None
2022-07-01
2020-10-22
i-campbell
No

According to these slides from 2020.10 Eclipse IoT, Edge and Sparkplug Update - Community Meeting, Eclipse are "planning for the Technology Compatibility Kit (TCK) and compatibility logo program"

This is a test suite, which confirms if your device is compatible to the standard, and if so grants you the ability to use the compatibility logo.

When this TCK is available, we should test our software against it, and aim to get the logo.

The repository for the Sparkplug spec, https://github.com/eclipse/sparkplug will at a minimum point you to the TCK.

Discussion

  • i-campbell

    i-campbell - 2020-10-22
    • summary: Sparkplug: --> Sparkplug: conform to Technology Compatibility Kit (TCK)
     
  • i-campbell

    i-campbell - 2020-11-03

    Extract of Eclipse IoT Working Group Slack 2020-11-1
    Ian Campbell 20:24
    Hello, me again. I am trying to work out the administrative requirements for using an eclipse TCK for my open source sparkplug library.
    using the TCK to verify the library during development.
    Saying "We conform to the Eclipse SparkplugTM TCK" and displaying the TCK results on a webpage
    displaying the conformance logo on our homepage
    Do any of those three require a) written permission or b) sparkplug workgroup membership ?

    Mike Milinkovich 15:48
    Let me start by saying that the Sparkplug conformance program is not yet fully established. But assuming that the Steering Committee decides to largely emulate the approach taken by Jakarta EE:
    1 and 2 do not require any permissions or participation beyond what is in the Eclipse Foundation's TCK License.
    3 would require both signing the Sparkplug trademark license agreement and membership in the Sparkplug working group

     
  • i-campbell

    i-campbell - 2020-11-03

    So from this:
    - We must confirm details closer to the release of the TCK.
    - If it is the same as the Eclipse Jakarta EE TCK, displaying the conformance logo would require an investment of 2000USD/year, to join the Eclipse Foundation and the Sparkplug Working Group.
    - If we don't use the Eclipse TCK Logo, we can still prove our product and use the text "We conform to the Eclipse SparkplugTM TCK"

     
  • i-campbell

    i-campbell - 2020-11-03
    • status: open --> pending
     
  • i-campbell

    i-campbell - 2021-02-23

    The very first "test" in the TCK is available for testing:
    https://github.com/eclipse/sparkplug/tree/develop/tck
    We should document what is needed to setup a system to run our library against this one test.

     
    • hermsen

      hermsen - 2022-02-17

      First TCK test run by i-campbell 👌;
      First results;

      payloads-nbirth-timestamp: ;
      topics-dbirth-timestamp: PASS;
      topics-dbirth-mqtt: PASS;
      topics-nbirth-timestamp: ;
      payloads-dbirth-timestamp: PASS;
      edge-subscribe-dcmd: FAIL(Edge node should subscribe to DCMD level topics to ensure device targeted message from theprimary host application are delivered);
      message-flow-edge-node-birth-publish-subscribe: ;
      payloads-nbirth-bdseq: ;
      payloads-dbirth-qos: PASS;
      payloads-dbirth-seq: PASS;
      payloads-nbirth-rebirth-req: ;
      payloads-sequence-num-zero-nbirth: ;
      topics-nbirth-bdseq-included: ;
      topics-dbirth-seq: ;
      payloads-dbirth-order: ;
      payloads-nbirth-retain: ;
      topics-ndeath-seq: ;
      operational-behavior-data-commands-rebirth-value: ;
      payloads-ndeath-seq: ;
      payloads-ndeath-bdseq: ;
      principles-birth-certificates-order: ;
      payloads-nbirth-qos: ;
      principles-persistence-clean-session: FAIL (Clean session should be set to true.);
      topics-nbirth-mqtt: ;
      topics-nbirth-seq-num: ;
      payloads-ndeath-will-message: ;
      operational-behavior-data-commands-rebirth-datatype: ;
      payloads-nbirth-seq: ;
      edge-subscribe-ncmd: FAIL(Edge node should subscribe to NCMD level topics to ensure Edge node targeted message fromthe primary host application are delivered);
      payloads-dbirth-seq-inc: ;
      topics-nbirth-rebirth-metric: ;
      topics-ndeath-payload: ;
      payloads-ndeath-will-message-qos: FAIL (NDEATH message must have Qos set to 1);
      payloads-ndeath-will-message-retain: ;
      topics-nbirth-bdseq-matching: ;
      operational-behavior-data-commands-rebirth-name: ;
      payloads-dbirth-retain: PASS;
      OVERALL: FAIL;
      
      
      topics-ddata-timestamp: PASS;
      payloads-ddata-seq: PASS;
      payloads-ndata-seq: PASS;
      payloads-ddata-retain: PASS;
      topics-ddata-payload: PASS;
      payloads-ndata-timestamp: PASS;
      topics-ddata-mqtt: PASS;
      topics-ddata-seq-num: PASS;
      payloads-ndata-retain: PASS;
      payloads-ddata-qos: PASS;
      topics-ndata-seq-num: PASS;
      topics-ndata-timestamp: PASS;
      topics-ndata-payload: PASS;
      payloads-ddata-timestamp: PASS;
      payloads-ndata-qos: PASS;
      topics-ndata-mqtt: PASS;
      OVERALL: PASS;
      
      
      operational-behavior-data-commands-rebirth-action-1: FAIL;
      operational-behavior-data-commands-rebirth-action-3: ;
      operational-behavior-data-commands-rebirth-action-2: FAIL;
      OVERALL: FAIL;
      
       

      Last edit: hermsen 2022-02-17
  • hermsen

    hermsen - 2022-02-17
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1,7 +1,85 @@
    -According to these slides from [2020.10 Eclipse IoT, Edge and Sparkplug Update - Community Meeting](https://docs.google.com/presentation/d/1iDAN7j8O-X8_lVMz0TGSy9J81C1Y_eqArWV4irI2HhE/edit#slide=id.g7192d53e92_0_145), Eclipse are "planning for the Technology Compatibility Kit (TCK) and compatibility logo program"
    +# First TCK Test RUN Results
    +### Sparkplug TCK
    
    -This is a test suite, which confirms if your device is compatible to the standard, and if so grants you the ability to use the compatibility logo.
    +This is the Sparkplug Test Compatibility Kit (TCK) or test suite.
    +It is an independent third party provided compatibility kit to test our Edge Of Network and Device library against. Further details are found at https://github.com/eclipse/sparkplug/tree/develop/tck
    
    -When this TCK is available, we should test our software against it, and aim to get the logo.
    
    -The repository for the Sparkplug spec, https://github.com/eclipse/sparkplug will at a minimum point you to the TCK.
    +### Conclusion of our first TCK Test RUN
    +
    +At least 2 FAILS seem on first glance QUICK fixes. 
    +So, overall we think that we are on the right path but we also see room for improvements. 
    +
    +
    +### Test results
    +
    +1) We assume that when no outcome has been printed, this test is not yet implemented.
    +2) The TCK is still a work in progress. So some of the results will be incorrect, some not.
    +
    +test run by i-campbell 👌;
    +
    +-----
    +
    +payloads-nbirth-timestamp: ;
    +topics-dbirth-timestamp: PASS;
    +topics-dbirth-mqtt: PASS;
    +topics-nbirth-timestamp: ;
    +payloads-dbirth-timestamp: PASS;
    +edge-subscribe-dcmd: FAIL(Edge node should subscribe to DCMD level topics to ensure device targeted message from theprimary host application are delivered);
    +message-flow-edge-node-birth-publish-subscribe: ;
    +payloads-nbirth-bdseq: ;
    +payloads-dbirth-qos: PASS;
    +payloads-dbirth-seq: PASS;
    +payloads-nbirth-rebirth-req: ;
    +payloads-sequence-num-zero-nbirth: ;
    +topics-nbirth-bdseq-included: ;
    +topics-dbirth-seq: ;
    +payloads-dbirth-order: ;
    +payloads-nbirth-retain: ;
    +topics-ndeath-seq: ;
    +operational-behavior-data-commands-rebirth-value: ;
    +payloads-ndeath-seq: ;
    +payloads-ndeath-bdseq: ;
    +principles-birth-certificates-order: ;
    +payloads-nbirth-qos: ;
    +principles-persistence-clean-session: FAIL (Clean session should be set to true.);
    +topics-nbirth-mqtt: ;
    +topics-nbirth-seq-num: ;
    +payloads-ndeath-will-message: ;
    +operational-behavior-data-commands-rebirth-datatype: ;
    +payloads-nbirth-seq: ;
    +edge-subscribe-ncmd: FAIL(Edge node should subscribe to NCMD level topics to ensure Edge node targeted message fromthe primary host application are delivered);
    +payloads-dbirth-seq-inc: ;
    +topics-nbirth-rebirth-metric: ;
    +topics-ndeath-payload: ;
    +payloads-ndeath-will-message-qos: FAIL (NDEATH message must have Qos set to 1);
    +payloads-ndeath-will-message-retain: ;
    +topics-nbirth-bdseq-matching: ;
    +operational-behavior-data-commands-rebirth-name: ;
    +payloads-dbirth-retain: PASS;
    +OVERALL: FAIL;
    +
    +
    +topics-ddata-timestamp: PASS;
    +payloads-ddata-seq: PASS;
    +payloads-ndata-seq: PASS;
    +payloads-ddata-retain: PASS;
    +topics-ddata-payload: PASS;
    +payloads-ndata-timestamp: PASS;
    +topics-ddata-mqtt: PASS;
    +topics-ddata-seq-num: PASS;
    +payloads-ndata-retain: PASS;
    +payloads-ddata-qos: PASS;
    +topics-ndata-seq-num: PASS;
    +topics-ndata-timestamp: PASS;
    +topics-ndata-payload: PASS;
    +payloads-ddata-timestamp: PASS;
    +payloads-ndata-qos: PASS;
    +topics-ndata-mqtt: PASS;
    +OVERALL: PASS;
    +
    +
    +operational-behavior-data-commands-rebirth-action-1: FAIL;
    +operational-behavior-data-commands-rebirth-action-3: ;
    +operational-behavior-data-commands-rebirth-action-2: FAIL;
    +OVERALL: FAIL;
    
     
  • hermsen

    hermsen - 2022-02-17
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -18,7 +18,7 @@
    
     test run by i-campbell 👌;
    
    ------
    +~~~
    
     payloads-nbirth-timestamp: ;
     topics-dbirth-timestamp: PASS;
    @@ -83,3 +83,4 @@
     operational-behavior-data-commands-rebirth-action-3: ;
     operational-behavior-data-commands-rebirth-action-2: FAIL;
     OVERALL: FAIL;
    +~~~
    
     
  • hermsen

    hermsen - 2022-02-18
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1,86 +1,2 @@
    -# First TCK Test RUN Results
    -### Sparkplug TCK
    -
     This is the Sparkplug Test Compatibility Kit (TCK) or test suite.
     It is an independent third party provided compatibility kit to test our Edge Of Network and Device library against. Further details are found at https://github.com/eclipse/sparkplug/tree/develop/tck
    -
    -
    -### Conclusion of our first TCK Test RUN
    -
    -At least 2 FAILS seem on first glance QUICK fixes. 
    -So, overall we think that we are on the right path but we also see room for improvements. 
    -
    -
    -### Test results
    -
    -1) We assume that when no outcome has been printed, this test is not yet implemented.
    -2) The TCK is still a work in progress. So some of the results will be incorrect, some not.
    -
    -test run by i-campbell 👌;
    -
    -~~~
    -
    -payloads-nbirth-timestamp: ;
    -topics-dbirth-timestamp: PASS;
    -topics-dbirth-mqtt: PASS;
    -topics-nbirth-timestamp: ;
    -payloads-dbirth-timestamp: PASS;
    -edge-subscribe-dcmd: FAIL(Edge node should subscribe to DCMD level topics to ensure device targeted message from theprimary host application are delivered);
    -message-flow-edge-node-birth-publish-subscribe: ;
    -payloads-nbirth-bdseq: ;
    -payloads-dbirth-qos: PASS;
    -payloads-dbirth-seq: PASS;
    -payloads-nbirth-rebirth-req: ;
    -payloads-sequence-num-zero-nbirth: ;
    -topics-nbirth-bdseq-included: ;
    -topics-dbirth-seq: ;
    -payloads-dbirth-order: ;
    -payloads-nbirth-retain: ;
    -topics-ndeath-seq: ;
    -operational-behavior-data-commands-rebirth-value: ;
    -payloads-ndeath-seq: ;
    -payloads-ndeath-bdseq: ;
    -principles-birth-certificates-order: ;
    -payloads-nbirth-qos: ;
    -principles-persistence-clean-session: FAIL (Clean session should be set to true.);
    -topics-nbirth-mqtt: ;
    -topics-nbirth-seq-num: ;
    -payloads-ndeath-will-message: ;
    -operational-behavior-data-commands-rebirth-datatype: ;
    -payloads-nbirth-seq: ;
    -edge-subscribe-ncmd: FAIL(Edge node should subscribe to NCMD level topics to ensure Edge node targeted message fromthe primary host application are delivered);
    -payloads-dbirth-seq-inc: ;
    -topics-nbirth-rebirth-metric: ;
    -topics-ndeath-payload: ;
    -payloads-ndeath-will-message-qos: FAIL (NDEATH message must have Qos set to 1);
    -payloads-ndeath-will-message-retain: ;
    -topics-nbirth-bdseq-matching: ;
    -operational-behavior-data-commands-rebirth-name: ;
    -payloads-dbirth-retain: PASS;
    -OVERALL: FAIL;
    -
    -
    -topics-ddata-timestamp: PASS;
    -payloads-ddata-seq: PASS;
    -payloads-ndata-seq: PASS;
    -payloads-ddata-retain: PASS;
    -topics-ddata-payload: PASS;
    -payloads-ndata-timestamp: PASS;
    -topics-ddata-mqtt: PASS;
    -topics-ddata-seq-num: PASS;
    -payloads-ndata-retain: PASS;
    -payloads-ddata-qos: PASS;
    -topics-ndata-seq-num: PASS;
    -topics-ndata-timestamp: PASS;
    -topics-ndata-payload: PASS;
    -payloads-ddata-timestamp: PASS;
    -payloads-ndata-qos: PASS;
    -topics-ndata-mqtt: PASS;
    -OVERALL: PASS;
    -
    -
    -operational-behavior-data-commands-rebirth-action-1: FAIL;
    -operational-behavior-data-commands-rebirth-action-3: ;
    -operational-behavior-data-commands-rebirth-action-2: FAIL;
    -OVERALL: FAIL;
    -~~~
    
     
  • hermsen

    hermsen - 2022-02-18
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1,2 +1,7 @@
    -This is the Sparkplug Test Compatibility Kit (TCK) or test suite.
    -It is an independent third party provided compatibility kit to test our Edge Of Network and Device library against. Further details are found at https://github.com/eclipse/sparkplug/tree/develop/tck
    +According to these slides from [2020.10 Eclipse IoT, Edge and Sparkplug Update - Community Meeting](https://docs.google.com/presentation/d/1iDAN7j8O-X8_lVMz0TGSy9J81C1Y_eqArWV4irI2HhE/edit#slide=id.g7192d53e92_0_145), Eclipse are "planning for the Technology Compatibility Kit (TCK) and compatibility logo program"
    +
    +This is a test suite, which confirms if your device is compatible to the standard, and if so grants you the ability to use the compatibility logo.
    +
    +When this TCK is available, we should test our software against it, and aim to get the logo.
    +
    +The repository for the Sparkplug spec, https://github.com/eclipse/sparkplug will at a minimum point you to the TCK.
    
     

Log in to post a comment.