DICOM Conformance Testing in CI

  • Automated CI tests can assert association negotiation, SOP class filtering, and transfer syntax negotiation using DCMTK tools.
  • Mocking TCP sockets in unit tests misses real wire-level DICOM Upper Layer protocol behavior.
  • DCMTK command-line tools like echoscu and storescu run in milliseconds and produce deterministic, scriptable results.
  • CI conformance testing cannot replace a manual hospital acceptance test covering proprietary private tags and scanner timing quirks.
  • Real hospital scanners vary widely in association batching, from one association per study to one per slice.

You can automate most DICOM network conformance checks in continuous integration (CI) by asserting association negotiation, Service-Object Pair (SOP) classes, and transfer syntax rules on every commit. Running automated tests with headless tools catches broken protocol negotiations before code touches a clinical network. What CI cannot replace is the final manual hospital acceptance test, where proprietary modality software, non-standard private tags, and hospital firewall quirks dictate production behavior.

Testing against live clinical scanners during development is slow and risky. Hospital engineering teams take weeks to schedule test windows. When a transfer fails on site, identifying whether the problem is a closed port, a rejected presentation context, or an unexpected transfer syntax wastes valuable engineering hours.

Automating standard protocol assertions in CI gives you rapid feedback on every pull request. This guide outlines what to test automatically, shows how to script assertions with DCMTK, and defines the exact boundary where automated testing ends and hospital site testing begins.

This guide uses DCMTK 3.7.0 (and works identically on DCMTK 3.6.8) running in standard Linux CI containers.

Why run DICOM conformance testing in continuous integration?

A vendor formal DICOM Conformance Statement (DCS) documents what an application supports on paper. The document lists supported SOP classes, role capabilities, and transfer syntaxes as defined in DICOM PS3.2 Chapter 6. In active software projects, code drifts from documentation quickly. A minor dependency update can change byte decoding order. A refactored network handler might reject valid presentation contexts or drop associations unexpectedly.

Running DICOM conformance testing in continuous integration prevents these regressions from reaching staging environments. Instead of finding out during vendor integration that your listener refuses CT images, your build pipeline flags the failure in seconds.

Unit tests alone cannot catch protocol regressions. Mocking a TCP socket in your programming language tests your internal state machine, but it misses wire-level Upper Layer Protocol quirks. Full end-to-end hospital tests, on the other hand, run too infrequently to protect daily commits.

Running lightweight, protocol-level conformance tests in CI fills this gap. You start your service in a headless runner, point standard command-line tools at its TCP port, and assert exact protocol responses.

What can you assert automatically on every commit?

You do not need to test every clinical workflow in your build pipeline. Focus your CI test suite on 3 protocol layers: association handshakes, presentation context filtering, and transfer syntax negotiation.

First, assert association establishment and Application Entity (AE) title filtering. When a client sends an A-ASSOCIATE-RQ packet, your Service Class Provider (SCP) must validate the Called AE title. If the AE title matches your configuration, the server returns an A-ASSOCIATE-AC acknowledgment. If the AE title is unknown, the server must return an A-ASSOCIATE-RJ packet with result 1 (rejected permanent) and reason 7 (called AE title not recognized) per DICOM PS3.8 Section 9.3. A simple Verification Service (C-ECHO) request confirms that the association completes and returns status 0x0000 (Success).

Second, assert SOP Class filtering across presentation contexts. When a modality connects, it proposes specific abstract syntaxes for the data it intends to transmit. For instance, a scanner might propose CT Image Storage (1.2.840.10008.5.1.4.1.1.2) alongside an unsupported Radiation Therapy format. Your SCP must accept the CT presentation context and return rejection code 1 (abstract syntax not supported) for the unsupported format per DICOM PS3.4 Annex B. Your CI suite should assert that unadvertised SOP classes never get accepted by mistake.

Third, assert transfer syntax negotiation. Senders propose one or more transfer syntaxes for each abstract syntax. A standard client might propose Explicit VR Little Endian (1.2.840.10008.1.2.1), Implicit VR Little Endian (1.2.840.10008.1.2), and JPEG Lossless (1.2.840.10008.1.2.4.70). Your receiver must pick its preferred syntax correctly. If your receiver cannot decompress JPEG 2000, your test suite must prove that the receiver rejects that presentation context during negotiation instead of failing midway through pixel transfer.

Between testing against full Picture Archiving and Communication System (PACS) archives or running lightweight CLI tools in CI, we strongly recommend CLI tools. Full archives require complex storage schemas, heavy background services, and slow startup sequences. Command-line tools like DCMTK run in milliseconds and output exact error codes. You give up testing UI search tables, but you gain fast, deterministic test runs on every pull request.

Automating association negotiation tests with DCMTK

DCMTK provides command-line utilities that test every step of the DICOM handshake. You can run these tools directly in bash scripts inside your CI environment.

To test association negotiation, start your service in the background listening on a test port, such as 11112. First, verify the baseline connection using echoscu. The command connects to the host, negotiates the Verification SOP Class (1.2.840.10008.1.1), and verifies that the receiver returns success:

echoscu -v -aet TEST_SCU -aec TEST_SCP 127.0.0.1 11112

When this command succeeds, echoscu exits with status 0. If your server rejects the association or crashes, echoscu exits with status 1 and prints the rejection details to standard error.

Next, assert that your receiver properly rejects unauthorized or unknown AE titles. Run echoscu with an invalid Called AE title:

# Expect failure: unknown Called AE title
if echoscu -v -aet TEST_SCU -aec WRONG_AET 127.0.0.1 11112 2>&1 | grep -q "Called AE Title Not Recognized"; then
  echo "AE title validation passed"
else
  echo "Receiver accepted unknown AE title or failed unexpectedly"
  exit 1
fi

If the receiver accepts unknown AE titles without restriction, the test fails. To inspect the exact wire bytes returned by a rejected association, see our guide on debugging a failed DICOM association.

To test presentation context negotiation and C-STORE handling, use storescu with the +v (--verbose-pc) flag. The +v flag instructs storescu to display the outcome of every proposed presentation context.

storescu -v +v -aet TEST_SCU -aec TEST_SCP 127.0.0.1 11112 fixtures/ct-slice.dcm

In the command output, inspect the presentation context table.

Presentation Contexts:
  Context ID:        1 (Proposed)
    Abstract Syntax: =CTImageStorage
    Transfer Syntax: =ExplicitVRLittleEndian
  Context ID:        1 (Accepted)
    Abstract Syntax: =CTImageStorage
    Transfer Syntax: =ExplicitVRLittleEndian

If your receiver accepted the context with the expected transfer syntax, the assertion passes. For a complete command syntax reference, consult our guide to DCMTK commands for developers.

Testing SOP classes and transfer syntax negotiation in GitHub Actions

You can integrate these assertions into your CI pipeline using GitHub Actions. The workflow installs DCMTK, builds your service binary, starts the listener, and executes the test script.

Here is a complete workflow example (.github/workflows/dicom-conformance.yml):

name: DICOM Conformance Tests

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  conformance:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Install DCMTK
        run: |
          sudo apt-get update
          sudo apt-get install -y dcmtk

      - name: Build application
        run: make build

      - name: Run conformance test suite
        run: |
          # Start receiver in background on port 11112
          ./bin/dicom-listener --port 11112 --aet TEST_SCP &
          LISTENER_PID=$!

          # Wait up to 10 seconds for the port to open
          timeout 10 bash -c 'until nc -z 127.0.0.1 11112; do sleep 0.2; done'

          # Run automated conformance checks
          bash scripts/ci-dicom-conformance.sh 127.0.0.1 11112

          # Stop background listener
          kill $LISTENER_PID

Inside scripts/ci-dicom-conformance.sh, test specific transfer syntax behaviors. For example, test how your receiver handles compressed transfers versus uncompressed transfers.

Use storescu options to control what transfer syntaxes the client proposes. The -x= flag proposes all standard uncompressed transfer syntaxes using Explicit VR. The -xs flag proposes JPEG Lossless alongside uncompressed syntaxes:

#!/usr/bin/env bash
set -euo pipefail

HOST="$1"
PORT="$2"

echo "1. Testing C-ECHO Verification..."
echoscu -v -aet TEST_SCU -aec TEST_SCP "$HOST" "$PORT"

echo "2. Testing uncompressed transfer syntax acceptance..."
storescu -v +v -x= -aet TEST_SCU -aec TEST_SCP "$HOST" "$PORT" fixtures/ct-slice.dcm

echo "3. Testing unsupported transfer syntax rejection..."
# Propose only JPEG 2000 lossless (+xv) to a receiver that only accepts uncompressed
OUTPUT=$(storescu -v +v +xv -aet TEST_SCU -aec TEST_SCP "$HOST" "$PORT" fixtures/ct-slice.dcm 2>&1 || true)

if echo "$OUTPUT" | grep -q "No presentation context"; then
  echo "Unsupported transfer syntax correctly rejected during negotiation"
else
  echo "Expected transfer syntax rejection, but got: $OUTPUT"
  exit 1
fi

This script verifies that your application enforces protocol boundaries before receiving image data. To learn how transfer syntaxes and SOP classes interact in detail, read our article on DICOM SOP classes and transfer syntaxes explained. You can also run these tests locally by setting up a local DICOM test lab with Docker.

When designing your ingest service, consider how remote hospital senders establish connections. You can review our guide on connecting a sender for configuration patterns, and our documentation on receiving deliveries for handling downstream notifications.

What continuous integration cannot catch: the manual hospital test

Automated CI tests prove that your code conforms to the formal DICOM standard when talking to reference toolkits. They do not guarantee that your software will connect reliably with a 15-year-old hospital scanner.

Clinical environments present 4 real-world challenges that CI runners cannot simulate reliably.

First, proprietary private tags. Modality vendors frequently insert non-standard metadata elements into images. Some vendors use private tags with custom Value Representations or embed proprietary binary data structures into text fields. A parser that strictly adheres to the standard in CI might crash on site when it encounters an undocumented private element.

Second, timing and socket behavior. Real modalities often operate on legacy hardware running embedded operating systems. Some scanners abort associations if an acknowledgment packet takes longer than 500 milliseconds. Other scanners drop the TCP socket without sending an A-RELEASE-RQ packet if their local storage buffer fills up.

Third, association batching variations. In CI, test scripts send 1 study cleanly over a single association. In a hospital, modalities behave unpredictably. Some scanners open 1 association for an entire 1,500-slice CT examination. Other scanners open and close a separate association for every single image slice, creating heavy connection churn.

Fourth, hospital network middleboxes. Hospital networks route traffic through firewalls, network address translation (NAT) gateways, and medical device security filters. These devices can terminate idle TCP connections after 30 seconds or strip custom packet headers.

Automated CI testing protects your codebase against internal regressions. Once your CI suite passes, you still need a scheduled pre-flight integration test with the hospital clinical engineering team. The manual test verifies how the physical scanner handles network latency, private metadata, and association churn under real clinical workloads.

Dicomly builds infrastructure for healthcare software teams that receive DICOM studies from hospitals. Why we built Dicomly →

Sources

  1. DICOM Standard Part 2 (PS3.2, Chapter 6 — Purpose of a Conformance Statement) (dicom.nema.org)
  2. DICOM Standard Part 8 (PS3.8, Section 9.3 — DICOM Upper Layer Protocol Data Units) (dicom.nema.org)
  3. DICOM Standard Part 4 (PS3.4, Annex B — Storage Service Class) (dicom.nema.org)
  4. OFFIS DCMTK — Open Source DICOM Toolkit (github.com)

Start receiving DICOM today.

Early access. No credit card. First endpoint free.