Connectathon Test Process

Based on registration, participants to the connectathon need to perform a series of test in order to be successful. Some of the tests need to be performed ahead of the connectathon and some need to be performed during the connectathon.

Preparatory (pre-Connectathon) Testing

 All participants are required to run a series of software tests in advance of the Connectathon. These tests, typically done with tools provided by IHE, are designed as auditing tests to make sure you have prepared in advance; these are not full test suites or certification tests for a profile.

Connectathon Testing

Each participant system is assigned a table that typically seats two people comfortably. Participants bring their equipment to the table and use the Gazelle system to identify specific tests as defined by their choice of actor/profile pairs. A participant will select a test to be run, and Gazelle presents a list of test partners. The participant will walk over to a peer’s table and arrange a time for testing. After a specific test is completed, the participant marks the test complete in Gazelle. A volunteer monitor is observing a worklist. He/she will see an item for grading and walk over to grade the test.  Some tests can be graded without visiting the participants who ran the test. The monitor enters the test result in Gazelle, and the cycle continues. During the evenings, Connectathon Managers review the individual test results and enter pass/fail grades in Gazelle for specific profile/actor pairs.

Items to bring to your attention:
- You can run tests in any order. With the exception of a few “Do This First” tests, order is not important.
- After you complete a test, you should start another test. Monitors are limited resources and may not appear immediately.
- The test cases indicate the kind of evidence needed to document the tests, and Gazelle provides a mechanism to store log information and screen captures. This is designed to make it easier for participants and monitors to find test results.
- In many instances, client applications initiate tests because that is a natural workflow. If you have server applications, you should be pro-active and seek out client applications for testing. Do not stand in the corner and wait for someone to ask you to dance.
- If you find a problem while running a test, you are allowed to update your application during the event and re-run the test later.

Testing depth: Supportive/Thorough testing

In order to successfully complete testing at the IHE Connectathon and appear in IHE's public database of successful Connectathon testing results published by IHE, a system must complete either Thorough or Supportive Connectathon testing.  This is call testing depth. This page explains the difference.

 

*** What is Thorough testing? ***

The list of required tests for each actor and profile is developed by the IHE Connectathon Technical Managers and is published for each Connectathon test system in Gazelle Test Management. Thorough testing at a Connectathon normally consists of successful completion of one instance of each Preparatory and Conformance test and three instances of each Interoperability test for the Profile/Actor pair it is testiing.

*** What is Supportive testing? Why would we do Supportive testing? ***

Participants who have successfully tested at a previous Connectathon may qualify to perform Supportive testing.

The benefits of Supportive testing are:

  • Participants receive full Connectathon credit for testing (ie a "pass") if they successfully perform one instance of an Interoperability test for a profile/actor pair.  Of course, you may choose to do more than one.  This frees you up to spend more time testing other profiles.
  • Participants doing Supportive testing are also required to assist other participants testing who need a partner, even if you have already completed your 'one required test'.  

 *** Who qualifies for Supportive testing? ***

In order to qualify for Supportive testing, a Participant:

  • Must previously have passed Thorough testing for that particular Actor and Profile at an IHE Connectathon whose results are reported in the IHE Connectathon Results database at http://connectathon-results.ihe.net/
  • Must also have published an IHE Integration Statement in the IHE Product Registry (http://product-registry.ihe.net/) for that Actor/Profile.

 *** How do we request Supportive testing? ***

Supportive requests are made directly in Gazelle Test Management and must be complete by the close of Connectathon system registration for your testing session.

  • Gazelle Test Management is https://gazelle.ihe.net/gazelle/home.seam in Europe, and https://gazelle.iheusa.org/gazelle-na/home.seam in North America
  • After logging in, select menu Registration .Then select Manage SUTs.
  • Edit your test system.
  • Select the Profile/Actors tab.
  • In the column Requested testing type, select 'Supportive' for the corresponding profile/actor. Note: Only the  actors/profile/options that are eligible for the Supportive testing enable you to select a testing type. For the others, the column shows N/A : Not Applicable.

After registration closes, the Connectathon Technical Project Managers will review and approve your request based on the criteria in the previous section. If you don't meet the criteria, you will do Thorough testing.

 *** FAQ ***

Q: Which profiles are eligible for Supportive testing?
A: In general, profiles that are in Final Text and have had no signficant Change Proposals in the previous year are eligible for supportive testing.

When you register for profile/actors in gazelle, those eligible for supportive testing will contain a dropdown list enabling you to select either Thorough or Supportive testing; otherwise N/A is displayed.

Q: If a profile has been around for multiple years but is not yet Final Text, is it a candidate for Supportive testing?
A: No. Only Final Text profiles are eligible.

Q: Are Connectathon fees the same for Thorough vs. Supportive?
A: There is no difference in cost. Connectathon fees are based on the number of systems registered. Fees are not affected by the number of domains/profiles tested, nor by thorough/supportive.

Q: Is there any difference in the final results for Thorough vs. Supportive?
A: No. If a Participant successfully completes Supportive testing, from a connectathon results point of view, it is equivalent to Thorough testing; ie. the Connectathon sponsors publish Profile/Actor pairs that are successfully tested by a Participant organization in the Connectathon results database.  There is no distinction between those tested as thorough vs supportive.

Q: Can companies update their IHE Integration Statement with these results?
A: In fact, there is no link between Connectathon testing and IHE Integration Statements.  An organization can publish an IHE integration statement for its product without ever testing at an IHE Connectathon.