Difference between revisions of "Running the tests"

Revision as of 03:13, 15 September 2012

This describes how to run the RDM Responder Tests. Before starting you may want to read the Responder Testing FAQ.

Install OLA

Follow one of OLA on OS X or OLA on Linux or OLA on Windows with VMWare to install OLA. If you want the biggest benefit from the tests you should use the version in the git repo as tests are added regularly.

Setup the Test Rig

The following controller devices are supported:

• RDM-TRI
• DMXter4 RDM / MiniDMXter
• Robe Universal Interface (latest firmware required)
• DMX USB Pro (latest firmware required)

DMXter Instructions

You need to put the DMXter into dongle mode. From the RDM menu, hold the left and right buttons and then hit the center button. The display should change to "USB Dongle Mode".

RDM TRI Instructions

In order to use the full test suite you need to use this interface in raw RDM mode. This allows the test suite to send malformed & unusual RDM packets to check the responder does the right thing. [TODO(someone): confirm which version of the firmware you need for this]

To enable raw rdm mode, in ~/.ola/ola-usbserial.conf, change the tri_use_raw_rdm line:

tri_use_raw_rdm = true

Discovering Responders

Connect the device under test to the controller device and start olad. Patch the output port on the controller device to a universe (UNIVERSE_NUMBER). Then run ola_rdm_discover, you should see the responder's UID appear:

 $ ola_rdm_discover -u UNIVERSE_NUMBER
 00a1:00010003
 7a70:ffffff00

Important

RDM Tests Server and Web UI

Since OLA 0.8.23, you can run rdm-tests from a web ui. Start the server with:

$ rdm_test_server.py
Checking olad status
Running RDM Tests Server on 127.0.0.1:9099

Then, open a web browser and go to: http://127.0.0.1:9099/

Running the Tests from the command line

The tests are written in Python and run using rdm_responder_test.py. Below is the output from a typical test run:

 $ ./rdm_responder_test.py --universe 1  00a1:00010003
 Starting tests, universe 3, UID 00a1:00010003
 SetManufacturerLabel: Passed
 SetSoftwareVersionLabel: Passed
 GetManufacturerLabel: Passed
 GetSoftwareVersionLabelWithData: Failed
 ...
 ------------- Warnings --------------
 ------------ By Category ------------
   Product Information:  7 /  7   100%
       RDM Information:  1 /  1   100%
    Core Functionality:  2 /  2   100%
      Error Conditions: 10 / 16   62%
          DMX512 Setup:  3 /  3   100%
 -------------------------------------
 29 / 30 tests run, 23 passed, 6 failed, 0 broken

Useful Options

rdm_responder_test.py has some options which can assist in debugging failures. For a full list of options run with -h

-d, --debug

Show all debugging output, including actual & expected responses.

-l, --log

Log the output of the tests to a file. The UID and timestamp is appended to the filename

-t Test1,Test2 , --tests=Test1,Test2

Only run a subset of the Tests. Only the tests listed (and their dependencies) will be run.

-w BROADCAST_WRITE_DELAY, --broadcast_write_delay=BROADCAST_WRITE_DELAY

Some responders take time to write state changes to memory. When using broadcast SETs, the controller doesn't know how long to wait before sending the next message. This setting adds a delay after (in ms) to wait after sending broadcast set commands.