Test Session Processes
Test Session Management
Objectives
- Prepare Test Session in order to be successful
Processes
Tools
- The Gazelle Test Management tool provides the user with the ability to define a jasper report to generate a contract for the participation to the testing session. The contract is then generated based on the content of the template, and can be customized accordingly (link to sample generated contract)
01 - Preparation
Registration of Participants
Objectives
- Organise participants registration and communication for the Test Session
Pre-requisites
- A new testing session is configured and opened by the Test Session Manager
In case of Accredited Testing, the involved SUT must have passed the targeted tests at an IHE connectathon.
Inputs
- All users who want to participate at the Test Session
- Set of validated test cases to be used for the Test Session
Actions
- SUT Operators :
- Must register hisself and his company
- Register their system(s)
In case of Accredited Testing, the system version is not allowed to change, at any time all along the test session.
- Generate contracts
- Manage their preferences
- Test Session Manager :
- Accepts all users registration (SUT Operators)
- Accepts systems to session
- Creates accounts for monitors
- Monitors
- Log into the application
Outputs
- Participants registered list
- Systems registered list
- Contracts generated
- Monitors list
Exchange of configuration and preparation
Objectives
- Provide configuration information about systems
- Prepare Test Session
Pre-requisites
- A new testing session is configured and opened by the Test Session Manager
- All participants and their systems are registered for the Test Session
Inputs
- N/A
Actions
- SUT Operators :
- Perform pre-Test Session tests
- Review and approve configurations
- Prepare systems (pre-load test input/network configuration/peer configuration)
- Create and consume samples
- Test Session Manager :
- Assign test to monitors
- Generate systems configurations
- Monitors
- Review assigned tests
- Verify samples
Outputs
- Test Session planning
- Approval by Test Session Manager of the readiness to start the session (based on the test session planning dashboard)
Flowchart
02 - Execution
Objectives
- Performed tests for registered systems
Pre-requisites
- Test Session is configured and opened
- Participants are registered
- Systems are registered
- Systems configurations are ready to exchange informations
- Tests are ready, validated and assigned
Inputs
- Key Performance Indicators that allow following the progress of the test execution
- Reporting elements that need to be provided in the test session report
- Satisfaction questionnaire for participants are prepared
Actions
- Test Session Manager
- Sends Kick-off mail (see template)
- Consults KPI
- Turns critical status to on
- Grades systems
- It is possible to add some test case to the test plan, even if they are not planned at the beginning.
- SUT Operators
- Run assigned tests
- Store any proof of test execution (logs, trace...)
- Change tests status
- Complete the satisfaction questionnaire
- Monitors
- Verify assigned tests
- Specific attention must be paid at the moment a test result seems to be false : monitor checks environment first after concluding if the result is false or not
- in case of Test steps are not performed (eg : NIST tools not available), monitor mentions it and adds a comment on the reason why it is not impacting the test result.
- Complete the satisfaction questionnaire
Outputs
- Satisfaction questionnaires, see for example the Istanbul survey
- Set of test instances performed during the Test Session execution (includes Monitors and SUT operators comments)
Tools
Flowchart
Kick-off mail template
CONTEXT (SUT Description)
- List of organizations / SUTs / SUT Operators :
- On Gazelle Test Management : in "Connectathon" -> "Find systems" to view organizations, systems and SUT Operators concerned by the Test Session.
COVERED BY 17025 ACCREDITATION ?
- YES / NO
TEST SESSION IDENTIFICATION
- Test session name :
- If on line session :
- Gotomeeting :
- Risks :
- Begin :
- End :
- Test Session Manager :
- Monitors :
- On Gazelle Test Management : in "Connectathon" -> "Find a monitor" to see all monitors concerned by the Test Session
- Quality controller :
TEST PERIMETER
- Tests to run :
- Tests to run are provided by the Gazelle Test Management Tool
- Test not to run :
- Configurations :
- Number of configuration :
- Configuration identification :
ENVIRONMENT
- Gazelle Platform :
- Version :
- Other Test Tools :
- [TEST TOOL 1] version :
- [TEST TOOL 2] version :
- [TEST TOOL 3] version :
SPECIFIC REQUIREMENTS
- Requirement 1 :
- Requirement 2 :
03 - Reporting
Report Test Session Process
Objectives
- The objective of the test session reporting are double :
- provide the test session participant with a report of the outcome of the participation of the SUT in the test session.
- provide the certification body with a report of the test session.
Pre-requisites
- The test session is closed and neither the participants nor the monitors can update or change any information on the tests performed during the session. All required test were graded by the monitors was performed during the test session and grading of the SUT was performed by the Test Session Manager
Inputs
- Inputs are the elements of testing (logs, reports, annotation, bug reports) that were recorded during the test session
- Survey filled out by testers
Actions
- The Test Session Manager generates and signs the SUT test report using the Gazelle Test Management application
- The Test Session Manager writes a report to the certification body. The reports includes KPI about the test session
Outputs
- Test session report covered under accreditation send to the certification body
- SUT test reports, signed by Lab Manager send to each SUT operators and to the certification body
All outputs are distributed under their electronic version (signed PDF). The electronic version of the document prevails