Filters

A destination with a filter gets only the messages the filter matches; a destination without one gets every message. Filters are written in the configuration, in TOML, and checked when the gateway starts: a filter with a mistake stops the gateway with an error that names it. To try one before you deploy it, see Check a filter.

Where a filter goes

On a destination of a route with several destinations (see Several sources and destinations), either inline or as a table of its own:

[[route.destinations]]
name = "lab"
type = "http"
url = "https://lab.example.org/hl7/orders"
filter = { message_type = ["ORM^O01"], fields = { "OBR-4.1" = ["GLU", "K"] } }

# A longer filter reads better as a table of its own, after the destination's other keys
[[route.destinations]]
name = "icu"
type = "mllp"
address = "icu-monitor:6661"

[route.destinations.filter]
message_type = ["ADT"]
fields = { "PV1-3.1" = ["ICU*"] }
not = { processing_id = ["T"] }

When no destination takes a message, the route's on_unmatched decides: "accept" (the default) answers AA and counts it as unmatched in the console and in hl7edge_unmatched_total; "reject" answers AR, so the sender knows. A pass-through route's primary destination gets every message, so it can't have a filter.

What a filter can name

Each key takes a list of values, even for one value. A key the gateway doesn't know, such as a misspelled one, is an error.

Field paths

A path names a segment, a field in it and, optionally, a component and a subcomponent, all counted from 1 as in the HL7 standard. An index in square brackets picks one occurrence of the segment or one repetition of the field; a negative index counts from the end.

SEG[occurrence]-field[repetition].component.subcomponent

PV1-2            PV1, field 2
OBR-4.1          OBR, field 4, component 1
PID-3.4.1        PID, field 3, component 4, subcomponent 1
OBR[2]-4.1       the second OBR only
OBR[-1]-4.1      the last OBR only
PID-3[2].1       the second repetition of PID-3 only
OBR[1]-4[1].1    both: the first OBR, the first repetition
OBR[*]-4.1       every OBR: the same as OBR-4.1

• Without an index, a path looks at every occurrence of the segment and every repetition of the field, and matches if any of them has a matching value.
• With an index, it looks at that occurrence or repetition only. One that isn't in the message (a third OBR when there are two) has no value, so it never matches, not even * or "".
• Segment IDs are three upper-case letters or digits, starting with a letter: PV1, OBX, ZPI. Indexes are whole numbers; [0] is an error.
• MSH counts as in the standard: MSH-1 is the field separator, MSH-2 the encoding characters, MSH-9 the message type.
• Separators come from the message's own MSH, so messages that use other characters than |^~\& work too.

A path, and what it finds

For this order:

MSH|^~\&|MEDITECH|MAIN^1.2.3^ISO|LIS|LAB|20261019120000||ORM^O01^ORM_O01|ORD-1001|P|2.5.1
PID|1||100234^^^HOSP^MR~998877^^^STATE^PI||DOE^JANE
PV1|1|I|4W^401^1
ORC|NW|ORD-1001
OBR|1|ORD-1001||GLU^Glucose
OBR|2|ORD-1001||K^Potassium&Serum

Values

• Case doesn't matter: adt^a08 matches ADT^A08.
• Wildcards: * is any run of characters, ? any one character. NORTH* matches NORTH and NORTHWEST; A0? matches A01 to A09. Values are otherwise compared whole: 4W doesn't match 4WEST.
• Empty: "" matches a field that is empty, or missing from a segment that is there. "*" matches any value, empty too, as long as the segment is there: use it to ask whether a segment is in the message.
• Not allowed in a value: | and line breaks. Values are compared to one component or subcomponent, never across separators.
• An empty list ("PV1-2" = []) is an error: list "" for empty, or "*" for any.

AND, OR and NOT

• OR within a key: its values are alternatives.
• AND across keys: every key the filter names must match.
• OR across keys: any, a list of whole conditions, at least one of which must match.
• NOT: not, a condition; a message that matches it is left out.

The conditions under any and not are filters themselves, with the same keys, so they can have their own any and not. They join the rest of the filter with AND. A filter needs at least one key; one that is just a not takes every message except those.

[route.destinations.filter]
message_type = ["ADT^A01", "ADT^A04"]     # A01 OR A04
fields = { "PV1-2" = ["I", "E"] }         # AND (PV1-2 is I OR E)
any = [                                   # AND at least one of these:
  { sending_facility = ["MAIN"] },        #   from MAIN
  { fields = { "PV1-3.1" = ["ICU*"] } },  #   OR on an ICU ward
]
not = { processing_id = ["T"] }           # AND NOT a test message

Segment groups

Each path is checked on its own, against the whole message. That isn't always enough: in this result, a filter for a potassium test with a high value matches, because there is a K in one OBR and an H in another.

MSH|^~\&|LIS|LAB|EHR|MAIN|20261019120000||ORU^R01^ORU_R01|RES-2001|P|2.5.1
PID|1||100234^^^HOSP^MR||DOE^JANE
ORC|RE|ORD-1001
OBR|1|ORD-1001||GLU^Glucose
OBX|1|NM|GLU^Glucose||250|mg/dL|70-99|H
ORC|RE|ORD-1002
OBR|2|ORD-1002||K^Potassium
OBX|1|NM|K^Potassium||4.1|mmol/L|3.5-5.1|N

hl7-edge-gateway filter -f '{ fields = { "OBR-4.1" = ["K"], "OBX-8" = ["H"] } }' result.hl7

result.hl7  RES-2001 ORU^R01  MATCH
  yes  OBR-4.1 K   found GLU, K
  yes  OBX-8 H     found H, N

A group ties the conditions to one segment group. segment names the segment that starts each group; a group is that segment and the segments after it, up to the next one. The rest of the table is a filter that one of the groups must match, looking only at that group's segments:

hl7-edge-gateway filter -f '{ group = { segment = "OBR", fields = { "OBR-4.1" = ["K"], "OBX-8" = ["H"] } } }' result.hl7

result.hl7  RES-2001 ORU^R01  NO MATCH
  no   group OBR: at least one where
    no   OBR #1
      no   OBR-4.1 K   found GLU
      yes  OBX-8 H     found H
    no   OBR #2
      yes  OBR-4.1 K   found K
      no   OBX-8 H     found N

• Where a group ends: at the next segment, or at a segment named in until. In the sample, the glucose group takes in the next order's ORC unless until = ["ORC"] ends it there. Segments before the first segment, such as PID, are in no group.
• Inside a group, paths look at the group's segments only, and indexes count within it: OBX[1]-8 is the group's first OBX. A path to a segment outside the group, such as PID-3, finds nothing.
• MSH keys (message_type, sending_facility and the rest) apply to the whole message, so they go outside the group; inside one they are an error.
• any, not and groups work inside a group as anywhere else, within the group. not inside a group excludes groups; not around a group excludes messages that have such a group.
• Several groups, written as a list, must each be in the message; they can be different groups. A message without the segment has no groups, so a group never matches it.

# Inline
filter = { message_type = ["ORU^R01"], group = { segment = "OBR", until = ["ORC"], fields = { "OBR-4.1" = ["GLU"], "OBX-8" = ["H", "HH"] } } }

# As tables, after the destination's other keys
[route.destinations.filter]
message_type = ["ORU^R01"]

[route.destinations.filter.group]
segment = "OBR"
until = ["ORC"]
fields = { "OBR-4.1" = ["GLU"], "OBX-8" = ["H", "HH"] }

# Several groups: each must be in the message (not necessarily the same one)
[[route.destinations.filter.group]]
segment = "OBR"
fields = { "OBR-4.1" = ["GLU"] }

[[route.destinations.filter.group]]
segment = "OBR"
fields = { "OBR-4.1" = ["HBA1C"] }

Groups follow the order of the segments in the message, not the message structure in the HL7 standard, so they work for Z-segments and local variants too. Check a group filter on real messages first: the filter command and the console show each group with its conditions. The Results to the EHR, alerts by order example runs two group filters end to end.

Recipes

# Every ADT but discharges
filter = { message_type = ["ADT"], not = { message_type = ["ADT^A03"] } }

# Results for these tests, in any OBR of the message
filter = { message_type = ["ORU^R01"], fields = { "OBR-4.1" = ["GLU", "K", "NA"] } }

# Orders whose first OBR is a glucose test
filter = { message_type = ["ORM"], fields = { "OBR[1]-4.1" = ["GLU"] } }

# Patients with a second identifier from the state registry
filter = { fields = { "PID-3[2].4" = ["STATE"] } }

# Messages that carry a Z-segment at all
filter = { fields = { "ZPI-1" = ["*"] } }

# An empty PV1-3 (no assigned location)
filter = { fields = { "PV1-3" = [""] } }

# OR across fields: an A08, or anything from NORTH
filter = { any = [ { message_type = ["ADT^A08"] }, { sending_facility = ["NORTH"] } ] }

# Everything except test and debug traffic
filter = { not = { processing_id = ["T", "D"] } }

With groups:

# A critical result (HH or LL) in any order
filter = { message_type = ["ORU"], group = { segment = "OBR", until = ["ORC"], fields = { "OBX-8" = ["HH", "LL"] } } }

# A glucose order with no abnormal result: not inside the group
filter = { group = { segment = "OBR", fields = { "OBR-4.1" = ["GLU"] }, not = { fields = { "OBX-8" = ["H", "HH", "L", "LL"] } } } }

# Results with no high value in any order: not around the group
filter = { message_type = ["ORU"], not = { group = { segment = "OBR", fields = { "OBX-8" = ["H", "HH"] } } } }

# The first result of an order is final (OBX[1] counts within the group)
filter = { group = { segment = "OBR", fields = { "OBR-4.1" = ["CBC"], "OBX[1]-11" = ["F"] } } }

# Groups in groups: order ORD-1001 has a glucose test with a high result
[route.destinations.filter.group]
segment = "ORC"
fields = { "ORC-2" = ["ORD-1001"] }
group = { segment = "OBR", fields = { "OBR-4.1" = ["GLU"], "OBX-8" = ["H"] } }

# An insurance plan, with its own IN2 and IN3
filter = { message_type = ["ADT"], group = { segment = "IN1", fields = { "IN1-2.1" = ["MEDICARE"], "IN2-6" = ["*"] } } }

Check a filter

Two ways to check a filter's syntax and see what it does to real messages, without sending anything. Both show each condition with yes or no and the values the message has there, so you can see why a message matched or didn't.

The filter command

hl7-edge-gateway filter takes a filter in any of the forms a configuration uses: the inline table, the whole filter = { … } line, or a [route.destinations.filter] table with its keys and group tables. Add message files to test it on them; a file can hold one message, several, or an HL7 batch.

hl7-edge-gateway filter -f '{ message_type = ["ORM"], fields = { "OBR[1]-4.1" = ["GLU", "K"] }, any = [ { fields = { "PV1-2" = ["E"] } }, { fields = { "PV1-2" = ["I"], "PV1-3.1" = ["ICU*"] } } ] }' order.hl7

filter OK: message_type ORM · OBR[1]-4.1 GLU|K · any(PV1-2 E | PV1-2 I · PV1-3.1 ICU*)

order.hl7  ORD-1001 ORM^O01  NO MATCH
  yes  message_type ORM   found ORM^O01
  yes  OBR[1]-4.1 GLU|K   found GLU
  no   any: at least one of
    no   PV1-2 E          found I
    no   all of
      yes  PV1-2 I        found I
      no   PV1-3.1 ICU*   found 4W

With -c, it checks every filter in a configuration file, and shows which destinations of each route would get each message. It doesn't need the configuration's secrets: ${NAME} variables that aren't set are filled in for the check.

# Every filter in a configuration, and where each message would go
hl7-edge-gateway filter -c gateway.toml messages/*.hl7
hl7-edge-gateway filter -c gateway.toml --route adt-feed messages/adt-a08-outpatient.hl7

# A filter kept in a file (any of the forms above)
hl7-edge-gateway filter -f @lab-filter.toml messages/*.hl7

# With Docker
docker run --rm -v "$PWD:/work" -w /work hl7-edge-gateway:<version> filter -c gateway.toml messages/*.hl7

gateway.toml: filters OK
route adt-feed
  lab              filter: message_type ADT^A01|ADT^A04|ADT^A08 · PV1-2 I|E
  his              every message
  archive          every message

messages/adt-a08-outpatient.hl7  ADT-9003 ADT^A08  route adt-feed: to his, archive
  lab              doesn't get it
    yes  message_type ADT^A01|ADT^A04|ADT^A08   found ADT^A08
    no   PV1-2 I|E                              found O
  his              gets it (no filter)
  archive          gets it (no filter)

It exits with 0 when the filter is valid and every message went to some destination, 1 when a filter isn't valid, and 3 when a message matched no destination, so a build can test a configuration change against sample messages.

The console's filter tester

In the admin console, open Filter tester, paste one or more messages, and either type a filter or choose a route to test its destinations as they run. Nothing is sent or stored. The same check is in the admin API as POST /api/filter-test, with { "filter": "…", "message": "…" } or { "route": "…", "message": "…" }.

What filters don't do

• The HL7 message structure: groups follow the order of the segments, from one segment to the next (or to until); the gateway doesn't read the standard's message structures.
• Comparisons: values are matched as text, with wildcards; there are no numeric ranges, dates or regular expressions.
• Changes: a filter chooses destinations; it doesn't change the message. A transform does.