Suricata not processing traffic - segfault but process still running

[erasedhammer]

Untangle 16.6.2

Turning Intrusion prevention on/off in the GUI results in a segfault in the suricata process. The service restarts successfully and the process is running, but dmesg reports the segfault and the app itself has no traffic present in the reports page under "all events" (when normally theres tons of traffic flowing through it).

Dmesg:

[1007149.120665] Suricata-Main[5613]: segfault at 19 ip 00007fae9feb573c sp 00007ffc8ab974c0 error 4 in libc-2.31.so[7fae9fe50000+15a000]
[1007149.120671] Code: 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 83 ec 18 48 8b 05 cd 77 14 00 48 8b 00 48 85 c0 0f 85 81 00 00 00 48 85 ff 74 74 <48> 8b 47 f8 48 8d 77 f0 a8 02 75 38 48 8b 15 29 76 14 00 64 48 83
[1157528.978766] Suricata-Main[1659]: segfault at 19 ip 00007f1001fa473c sp 00007ffeb58c39e0 error 4 in libc-2.31.so[7f1001f3f000+15a000]
[1157528.978773] Code: 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 83 ec 18 48 8b 05 cd 77 14 00 48 8b 00 48 85 c0 0f 85 81 00 00 00 48 85 ff 74 74 <48> 8b 47 f8 48 8d 77 f0 a8 02 75 38 48 8b 15 29 76 14 00 64 48 83

Journal logs for suricata:

Jan 28 22:17:20 fw.hostname suricata[122479]: [122479] <Notice> -- all 14 packet processing threads, 4 management threads initialized, engine started.
Jan 28 22:17:02 fw.hostname suricata[122479]: [122479] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Proyecto.YopmailLogin' is checked but not set. Checked in 2027735 and 0 other sigs
Jan 28 22:17:02 fw.hostname suricata[122479]: [122479] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.zbot.ua.2106509' is checked but not set. Checked in 2018764 and 0 other sigs
Jan 28 22:17:02 fw.hostname suricata[122479]: [122479] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.NetwireRAT.Client' is checked but not set. Checked in 2028918 and 0 other sigs
Jan 28 22:17:02 fw.hostname suricata[122479]: [122479] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Kuluoz' is checked but not set. Checked in 2019187 and 0 other sigs
Jan 28 22:17:02 fw.hostname suricata[122479]: [122479] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.WinHttpRequest' is checked but not set. Checked in 2019823 and 0 other sigs
Jan 28 22:16:59 fw.hostname suricata[122478]: [122478] <Notice> -- This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 28 22:16:59 fw.hostname suricata[122478]: 28/1/2023 -- 22:16:59 - <Notice> - This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 28 22:16:56 fw.hostname systemd[1]: suricata.service: Failed with result 'core-dump'.
Jan 28 22:16:56 fw.hostname systemd[1]: suricata.service: Main process exited, code=dumped, status=11/SEGV
Jan 28 22:16:48 fw.hostname suricata[1659]: [2128] <Notice> -- (RX-NFQ#2930) Verdict: Accepted 0, Dropped 0, Replaced 0
Jan 28 22:16:48 fw.hostname suricata[1659]: [2128] <Notice> -- (RX-NFQ#2930) Treated: Pkts 0, Bytes 0, Errors 0
Jan 28 22:16:47 fw.hostname suricata[1659]: [1659] <Notice> -- Signal Received.  Stopping engine.
Jan 28 22:16:47 fw.hostname suricatasc[122260]: {"message": "Closing Suricata", "return": "OK"}
Jan 28 01:00:31 fw.hostname suricatasc[123778]: {"message": "done", "return": "OK"}
Jan 28 01:00:31 fw.hostname suricata[1659]: [1659] <Notice> -- rule reload complete
Jan 28 01:00:13 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Proyecto.YopmailLogin' is checked but not set. Checked in 2027735 and 0 other sigs
Jan 28 01:00:13 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.zbot.ua.2106509' is checked but not set. Checked in 2018764 and 0 other sigs
Jan 28 01:00:13 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.NetwireRAT.Client' is checked but not set. Checked in 2028918 and 0 other sigs
Jan 28 01:00:13 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Kuluoz' is checked but not set. Checked in 2019187 and 0 other sigs
Jan 28 01:00:13 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.WinHttpRequest' is checked but not set. Checked in 2019823 and 0 other sigs
Jan 28 01:00:09 fw.hostname suricata[1659]: [1659] <Notice> -- rule reload starting
Jan 27 04:31:49 fw.hostname suricata[1659]: [1659] <Notice> -- all 14 packet processing threads, 4 management threads initialized, engine started.
Jan 27 04:31:31 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Proyecto.YopmailLogin' is checked but not set. Checked in 2027735 and 0 other sigs
Jan 27 04:31:31 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.zbot.ua.2106509' is checked but not set. Checked in 2018764 and 0 other sigs
Jan 27 04:31:31 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.NetwireRAT.Client' is checked but not set. Checked in 2028918 and 0 other sigs
Jan 27 04:31:31 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Kuluoz' is checked but not set. Checked in 2019187 and 0 other sigs
Jan 27 04:31:31 fw.hostname suricata[1659]: [1659] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.WinHttpRequest' is checked but not set. Checked in 2019823 and 0 other sigs
Jan 27 04:31:27 fw.hostname suricata[1583]: [1583] <Notice> -- This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 27 04:31:27 fw.hostname suricata[1583]: 27/1/2023 -- 04:31:27 - <Notice> - This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 27 04:31:27 fw.hostname systemd[1]: Failed to start Suricata IDS/IDP daemon.
Jan 27 04:31:27 fw.hostname systemd[1]: suricata.service: Failed with result 'exit-code'.
Jan 27 04:31:27 fw.hostname suricata[1537]: [1537] <Error> -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - pid file '/var/run/suricata.pid' exists but appears stale. Make sure Suricata is not running and then remove /var/run/suricata.pid. Aborting!
Jan 27 04:31:27 fw.hostname suricata[1537]: 27/1/2023 -- 04:31:27 - <Error> - [ERRCODE: SC_ERR_INITIALIZATION(45)] - pid file '/var/run/suricata.pid' exists but appears stale. Make sure Suricata is not running and then remove /var/run/suricata.pid. Aborting!
Jan 27 04:31:27 fw.hostname suricata[1537]: [1537] <Notice> -- This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 27 04:31:27 fw.hostname suricata[1537]: 27/1/2023 -- 04:31:27 - <Notice> - This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
Jan 27 04:30:37 fw.hostname systemd[1]: suricata.service: Failed with result 'core-dump'.
Jan 27 04:30:37 fw.hostname systemd[1]: suricata.service: Main process exited, code=dumped, status=11/SEGV
Jan 27 04:30:28 fw.hostname suricata[5613]: [7150] <Notice> -- (RX-NFQ#2930) Verdict: Accepted 1159764, Dropped 3293, Replaced 0
Jan 27 04:30:28 fw.hostname suricata[5613]: [7150] <Notice> -- (RX-NFQ#2930) Treated: Pkts 1163058, Bytes 96478834, Errors 0
Jan 27 04:30:28 fw.hostname suricata[5613]: [5613] <Notice> -- Signal Received.  Stopping engine.

Process:

1 S root     122479      1  8  80   0 - 437545 -     22:16 ?        00:01:33   /usr/bin/suricata -D -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid -q 2930

Not sure why the service would report itself as running, and in the process list, but not actually process any traffic and have a segfault?

Can anyone help with this?

[erasedhammer]

Did a NIC swap last night, and a reboot seemed to solve the issue. No segfaults in dmesg and suricata is now logging traffic in reports.