Hello, I have server on Ubuntu 20.04. The system uses HWE kernel 5.15 The server has few mellanox-6 network cards with MOFED 5.6 drivers

When I try to build iscsi module it always failed with infiband errors Could you fix that? Thank you.

:~/scst/scst# make iscsi cd iscsi-scst && make all make[1]: Entering directory '/root/scst/scst/iscsi-scst' OFED_VERS = 5.6-OFED.5.6.2.0.9.1.kver.5.15.0-41-generic make -C usr make[2]: Entering directory '/root/scst/scst/iscsi-scst/usr' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/root/scst/scst/iscsi-scst/usr' make -C /lib/modules/5.15.0-41-generic/build M=/root/scst/scst/iscsi-scst/kernel CONFTEST_CFLAGS="-DBAD_WR_MODIFIER= -DCM_HANDLER_EVENT_MODIFIER= -DRDMA_CREATE_ID_TAKES_NET_ARG=0 -DRDMA_REJECT_HAS_FOUR_ARGS=0 -DSOCK_RECVMSG_HAS_FOUR_ARGS=0"

modules make[2]: Entering directory '/usr/src/linux-headers-5.15.0-41-generic' MODPOST /root/scst/scst/iscsi-scst/kernel/Module.symvers CC [M] /root/scst/scst/iscsi-scst/kernel/iscsi-scst.mod.o LD [M] /root/scst/scst/iscsi-scst/kernel/iscsi-scst.ko BTF [M] /root/scst/scst/iscsi-scst/kernel/iscsi-scst.ko Skipping BTF generation for /root/scst/scst/iscsi-scst/kernel/iscsi-scst.ko due to unavailability of vmlinux make[2]: Leaving directory '/usr/src/linux-headers-5.15.0-41-generic' echo "mods: INFINIBAND_ENABLED = true" mods: INFINIBAND_ENABLED = true if true; then
echo " Building against MOFED RDMA kernel headers.";
make -C /lib/modules/5.15.0-41-generic/build M=/root/scst/scst/iscsi-scst/kernel/isert-scst ISER_CFLAGS="-I/usr/src/ofa_kernel/default/include -I/usr/src/ofa_kernel/default/include/uapi -include /usr/src/ofa_kernel/default/include/linux/compat-2.6.h -DMOFED_MAJOR=5 -DMOFED_MINOR=6-OFED -DOFED_FLAVOR=MOFED -DBAD_WR_MODIFIER= -DCM_HANDLER_EVENT_MODIFIER= -DRDMA_CREATE_ID_TAKES_NET_ARG=0 -DRDMA_REJECT_HAS_FOUR_ARGS=0 -DSOCK_RECVMSG_HAS_FOUR_ARGS=0"

CONFIG_DTRACE= modules;
fi Building against MOFED RDMA kernel headers. make[2]: Entering directory '/usr/src/linux-headers-5.15.0-41-generic' CC [M] /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.o /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_post_recv’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:90:50: error: passing argument 3 of ‘ib_post_recv’ from incompatible pointer type [-Werror=incompatible-pointer-types] 90 | err = ib_post_recv(isert_conn->qp, first_ib_wr, &bad_wr); | ^~~~~~~ | | | struct ib_recv_wr ** In file included from /usr/src/ofa_kernel/default/include/rdma/ib_mad.h:15, from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:13, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/ib_verbs.h:3941:37: note: expected ‘const struct ib_recv_wr **’ but argument is of type ‘struct ib_recv_wr **’ 3941 | const struct ib_recv_wr **bad_recv_wr) | ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_post_send’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:137:50: error: passing argument 3 of ‘ib_post_send’ from incompatible pointer type [-Werror=incompatible-pointer-types] 137 | err = ib_post_send(isert_conn->qp, first_ib_wr, &bad_wr); | ^~~~~~~ | | | struct ib_send_wr ** In file included from /usr/src/ofa_kernel/default/include/rdma/ib_mad.h:15, from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:13, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/ib_verbs.h:3924:37: note: expected ‘const struct ib_send_wr **’ but argument is of type ‘struct ib_send_wr **’ 3924 | const struct ib_send_wr **bad_send_wr) | ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_post_drain_sq’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:167:33: error: passing argument 3 of ‘ib_post_send’ from incompatible pointer type [-Werror=incompatible-pointer-types] 167 | &drain_wr_sq->send_wr.wr, &bad_wr); | ^~~~~~~ | | | struct ib_send_wr ** In file included from /usr/src/ofa_kernel/default/include/rdma/ib_mad.h:15, from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:13, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/ib_verbs.h:3924:37: note: expected ‘const struct ib_send_wr **’ but argument is of type ‘struct ib_send_wr **’ 3924 | const struct ib_send_wr **bad_send_wr) | ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_post_drain_rq’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:192:30: error: passing argument 3 of ‘ib_post_recv’ from incompatible pointer type [-Werror=incompatible-pointer-types] 192 | &drain_wr_rq->recv_wr, &bad_wr); | ^~~~~~~ | | | struct ib_recv_wr ** In file included from /usr/src/ofa_kernel/default/include/rdma/ib_mad.h:15, from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:13, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/ib_verbs.h:3941:37: note: expected ‘const struct ib_recv_wr **’ but argument is of type ‘struct ib_recv_wr **’ 3941 | const struct ib_recv_wr *bad_recv_wr) | ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_device_create’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:998:8: error: macro "ib_create_cq" passed 6 arguments, but takes just 5 998 | i); / completion vector */ | ^ In file included from /usr/src/ofa_kernel/default/include/rdma/ib_mad.h:15, from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:13, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/ib_verbs.h:4009: note: macro "ib_create_cq" defined here 4009 | #define ib_create_cq(device, cmp_hndlr, evt_hndlr, cq_ctxt, cq_attr)
| /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:993:8: error: ‘ib_create_cq’ undeclared (first use in this function); did you mean ‘ib_create_wq’? 993 | cq = ib_create_cq(ib_dev, | ^~~~~~~~~~~~ | ib_create_wq /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:993:8: note: each undeclared identifier is reported only once for each function it appears in /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_conn_create’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:1233:47: error: ‘struct ib_device_attr’ has no member named ‘max_sge’; did you mean ‘max_cqe’? 1233 | isert_conn->max_sge = isert_dev->device_attr.max_sge - 3; | ^~~~~~~ | max_cqe /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_cm_conn_req_handler’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:1482:2: error: too few arguments to function ‘rdma_reject’ 1482 | rdma_reject(cm_id, NULL, 0 | ^~~~~~~~~~~ In file included from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:15, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/rdma_cm.h:272:5: note: declared here 272 | int rdma_reject(struct rdma_cm_id *id, const void *private_data, | ^~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c: In function ‘isert_setup_id’: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:1731:20: error: macro "rdma_create_id" requires 5 arguments, but only 4 given 1731 | IB_QPT_RC); | ^ In file included from /usr/src/ofa_kernel/default/include/rdma/ib_cm.h:15, from /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:44: /usr/src/ofa_kernel/default/include/rdma/rdma_cm.h:139: note: macro "rdma_create_id" defined here 139 | #define rdma_create_id(net, event_handler, context, ps, qp_type)
| /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:1730:7: error: ‘rdma_create_id’ undeclared (first use in this function); did you mean ‘rdma_create_qp’? 1730 | id = rdma_create_id(isert_cm_evt_handler, portal, RDMA_PS_TCP, | ^~~~~~~~~~~~~~ | rdma_create_qp At top level: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:1617:12: warning: ‘isert_cm_evt_handler’ defined but not used [-Wunused-function] 1617 | static int isert_cm_evt_handler(struct rdma_cm_id *cm_id, | ^~~~~~~~~~~~~~~~~~~~ /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.c:770:13: warning: ‘isert_cq_comp_handler’ defined but not used [-Wunused-function] 770 | static void isert_cq_comp_handler(struct ib_cq *cq, void *context) | ^~~~~~~~~~~~~~~~~~~~~ cc1: some warnings being treated as errors make[3]: *** [scripts/Makefile.build:285: /root/scst/scst/iscsi-scst/kernel/isert-scst/iser_rdma.o] Error 1 make[2]: *** [Makefile:1875: /root/scst/scst/iscsi-scst/kernel/isert-scst] Error 2 make[2]: Leaving directory '/usr/src/linux-headers-5.15.0-41-generic' make[1]: *** [Makefile:141: mods] Error 2 make[1]: Leaving directory '/root/scst/scst/iscsi-scst' make: *** [Makefile:226: iscsi] Error 2

Hi Folks,

Recently we tried compiling the scst package(scst3.6 and trunk) on RHEL 8.6, but the compiling failed (kernel: 4.18.0-372.9.1.el8.x86_64). The failure due the parameters is defined blkdev_issue_flush ./include/linux/blkdev.h:

/root/scst/scst-3.6/scst/src/dev_handlers/scst_vdisk.c:489:9: error: too many arguments to function 'blkdev_issue_flush' res = blkdev_issue_flush(bdev, gfp_mask); ^~~~~~~~~~~~~~~~~~ In file included from ./include/linux/blk-cgroup.h:23, from ./include/linux/writeback.h:15, from /root/scst/scst-3.6/scst/src/dev_handlers/scst_vdisk.c:40: ./include/linux/blkdev.h:1288:5: note: declared here int blkdev_issue_flush(struct block_device *bdev); ^~~~~~~~~~~~~~~~~~ make[3]: *** [scripts/Makefile.build:322: /root/scst/scst-3.6/scst/src/dev_handlers/scst_vdisk.o] Error 1 make[2]: *** [Makefile:1577: module/root/scst/scst-3.6/scst/src/dev_handlers] Error 2 make[2]: Leaving directory '/usr/src/kernels/4.18.0-372.9.1.el8.x86_64' make[1]: *** [Makefile:76: all] Error 2 make[1]: Leaving directory '/root/scst/scst-3.6/scst/src' make: *** [Makefile:36: all] Error 2 [root@net156-h105 scst]# uname -r 4.18.0-372.9.1.el8.x86_64 [root@net156-h105 scst]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.6 (Ootpa) [root@net156-h105 scst]# pwd
/root/scst/scst-3.6/scst

After updated the scst_vdisk.c with following changes, the compiling passed successfully. [root@net156-h105 scst]# diff /root/scst/scst-3.6/scst/src/dev_handlers/scst_vdisk.c /root/scst/scst-3.6/scst/src/dev_handlers/scst_vdisk.c.orig 488,490c488 < #elif LINUX_VERSION_CODE < KERNEL_VERSION(5, 12, 0) &&
< (!defined(RHEL_MAJOR) || RHEL_MAJOR -0 < 8 ||
< RHEL_MAJOR -0 == 8 && RHEL_MINOR -0 < 6)

#elif LINUX_VERSION_CODE < KERNEL_VERSION(5, 12, 0) [root@net156-h105 scst]#

Could you help to review if the change is ok?

Thanks, Kyle

I tried building scst-trunk (svn co as of 8/18/2022) on an RHEL 8.6 system and got the following error:

[2022-08-19 13:38:06] make[3]: Entering directory '/usr/src/kernels/4.18.0-372.9.1.el8.x86_64' [2022-08-19 13:38:08] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_cdrom.o [2022-08-19 13:38:09] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_changer.o [2022-08-19 13:38:10] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_disk.o [2022-08-19 13:38:11] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_modisk.o [2022-08-19 13:38:13] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_tape.o [2022-08-19 13:38:14] CC [M] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_vdisk.o [2022-08-19 13:38:15] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_vdisk.c: In function ‘vdisk_blockio_flush’: [2022-08-19 13:38:15] /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_vdisk.c:489:9: error: too many arguments to function ‘blkdev_issue_flush’ [2022-08-19 13:38:15] res = blkdev_issue_flush(bdev, gfp_mask); [2022-08-19 13:38:15] ^~~~~~~~~~~~~~~~~~ [2022-08-19 13:38:15] In file included from ./include/linux/blk-cgroup.h:23, [2022-08-19 13:38:15] from ./include/linux/writeback.h:15, [2022-08-19 13:38:15] from /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_vdisk.c:40: [2022-08-19 13:38:15] ./include/linux/blkdev.h:1288:5: note: declared here [2022-08-19 13:38:15] int blkdev_issue_flush(struct block_device *bdev); [2022-08-19 13:38:15] ^~~~~~~~~~~~~~~~~~ [2022-08-19 13:38:15] make[4]: *** [scripts/Makefile.build:322: /tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers/scst_vdisk.o] Error 1 [2022-08-19 13:38:15] make[3]: *** [Makefile:1577: module/tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src/dev_handlers] Error 2 [2022-08-19 13:38:15] make[3]: Leaving directory '/usr/src/kernels/4.18.0-372.9.1.el8.x86_64' [2022-08-19 13:38:15] make[2]: *** [Makefile:76: all] Error 2 [2022-08-19 13:38:15] make[2]: Leaving directory '/tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst/src' [2022-08-19 13:38:15] make[1]: *** [Makefile:39: install] Error 2 [2022-08-19 13:38:15] make[1]: Leaving directory '/tmp/ontap_mediator.PC0Imv/ontap-mediator-1.5.0/ontap-mediator-1.5.0/dist/scst/scst-3.7.0-pre/scst' [2022-08-19 13:38:15] make: *** [Makefile:158: scst_install] Error 2 [root@scs000088802-1 ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.6 (Ootpa) [root@scs000088802-1 ~]# cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.6 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.6" [root@scs000088802-1 ~]#

Please, let me know if there is fix for this?

My SCST service does not want to come up on an RHEL 8.5 when Secure Boot is turned on; even after I have gone to the UEFI menu and registered the keys using MOK menu per README instructions here: https://github.com/bvanassche/scst/blob/master/README.module-signing

I get the following error:

● mediator-scst.service Loaded: loaded (/etc/systemd/system/mediator-scst.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2021-12-08 10:26:33 EST; 1 day 22h ago Process: 7320 ExecStart=/etc/init.d/scst start (code=exited, status=5)

Dec 08 10:26:33 scs000040681 systemd[1]: mediator-scst.service: Service RestartSec=100ms expired, scheduling restart. Dec 08 10:26:33 scs000040681 systemd[1]: mediator-scst.service: Scheduled restart job, restart counter is at 5. Dec 08 10:26:33 scs000040681 systemd[1]: Stopped mediator-scst.service. Dec 08 10:26:33 scs000040681 systemd[1]: mediator-scst.service: Start request repeated too quickly. Dec 08 10:26:33 scs000040681 systemd[1]: mediator-scst.service: Failed with result 'exit-code'. Dec 08 10:26:33 scs000040681 systemd[1]: Failed to start mediator-scst.service.

My System's Info:

cat /etc/redhat-release: Red Hat Enterprise Linux release 8.5 (Ootpa) mokutil --sb-state: SecureBoot enabled modprobe scst && modprobe -r scst: modprobe: FATAL: Module scst not found in directory /lib/modules/4.18.0-348.2.1.el8_5.x86_64

What am I doing wrong, or need to do more when Secure Boot is turned on? Any help/hint is greatly appreciated.

Hi, first of all I want to thank you for the amazing work you are doing on the SCST project!

We are experiencing kernel crashes shortly after enabling target mode on a FibreChannel NPIV port using the new qla2x00t-32gbit driver. Stacktraces of the crashes mostly indicate a page fault in the function qlt_find_sess_invalidate_other within qla2x00t-32gbit/qla_target.c. However there also were occasions where the page fault occurred in qlt_handle_login or qlt_logo_completion_handler.

We did already test multiple releases of SCST (3.2, 3.5 & 3.6) as well as the current master branch (as of commit f7fabfd200). We also tried multiple levels of debug output (e.g with 2debug and ql2xextended_error_logging) and two types of FC cards (see below). All versions were susceptible to the crashes.

Steps to reproduce:

• Enable target mode on parent hardware port
• Create NPIV port
• Enable target mode on NPIV port

Our System

We are using a custom gentoo-based distribution with linux kernel version 5.4.109 and install that exact OS image to multiple machines with two types of FibreChannel Cards:

1. ISP2722-based 16/32Gb Fibre Channel to PCIe Adapter (QLE2692, ISP2722)
2. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (QLE2562, ISP2532)

Each machine is connected to two initiators using a FibreChannel switch.

Stacktraces:

PID: 0      TASK: ffff88846bd97000  CPU: 7   COMMAND: "swapper/7"
 #0 [ffffc90000234bb0] machine_kexec at ffffffff810865ff
 #1 [ffffc90000234bf8] __crash_kexec at ffffffff8110c8e7
 #2 [ffffc90000234cc0] crash_kexec at ffffffff8110d5f0
 #3 [ffffc90000234cd0] oops_end at ffffffff810651fc
 #4 [ffffc90000234cf0] page_fault at ffffffff81e0102f
    [exception RIP: qlt_find_sess_invalidate_other+433]
    RIP: ffffffffa011902f  RSP: ffffc90000234da8  RFLAGS: 00010007
    RAX: ffffffffffffff90  RBX: ffff88845d680003  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: 0000000000000046  RDI: ffff88845a6e79b0
    RBP: ffff88845a6e7598   R8: 0000000000000001   R9: 0000000000000003
    R10: ffff88845a6e7598  R11: 000000000000f026  R12: 0000000000010100
    R13: ffff88845d182d00  R14: 21000024ff3e2e75  R15: ffff88845a6e79c0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #5 [ffffc90000234df8] qlt_find_sess_invalidate_other at ffffffffa0119cf3 [qla2xxx_scst]
 #6 [ffffc90000234e68] qlt_find_sess_invalidate_other at ffffffffa011a5c9 [qla2xxx_scst]
 #7 [ffffc90000234eb8] qlt_24xx_process_atio_queue at ffffffffa011c10b [qla2xxx_scst]
 #8 [ffffc90000234f10] qla83xx_msix_atio_q at ffffffffa011c818 [qla2xxx_scst]
 #9 [ffffc90000234f30] __handle_irq_event_percpu at ffffffff810e2460
#10 [ffffc90000234f70] handle_irq_event_percpu at ffffffff810e2565
#11 [ffffc90000234f98] handle_irq_event at ffffffff810e25cb
#12 [ffffc90000234fb0] handle_edge_irq at ffffffff810e5eb6
#13 [ffffc90000234fc8] do_IRQ at ffffffff81e01824
--- <IRQ stack> ---
#14 [ffffc900000ffde8] ret_from_intr at ffffffff81e00a0f
    [exception RIP: cpuidle_enter_state+237]
    RIP: ffffffff81a959dc  RSP: ffffc900000ffe90  RFLAGS: 00000202
    RAX: ffff88847fde7600  RBX: ffff88846a6f0800  RCX: 000000000000001f
    RDX: 0000000000000000  RSI: 0000000000000007  RDI: 0000000000000000
    RBP: ffffffff828db1c0   R8: 0000005908d6505f   R9: ffff88847fddc8e0
    R10: 071c71c71c71c71c  R11: 0000000000000020  R12: 0000000000000002
    R13: 00000059079d3dd0  R14: 0000005908d6505f  R15: 0000000000000000
    ORIG_RAX: ffffffffffffffdb  CS: 0010  SS: 0018
#15 [ffffc900000ffec0] cpuidle_enter at ffffffff81a95b0b
#16 [ffffc900000ffee0] do_idle at ffffffff810cc2bf
#17 [ffffc900000fff20] cpu_startup_entry at ffffffff810cc465
#18 [ffffc900000fff30] start_secondary at ffffffff81080654
#19 [ffffc900000fff50] secondary_startup_64 at ffffffff810000d4

---

PID: 0      TASK: ffff88846dd94600  CPU: 4   COMMAND: "swapper/4"
 #0 [ffffc900001b0bc0] machine_kexec at ffffffff810865ff
 #1 [ffffc900001b0c08] __crash_kexec at ffffffff8110c8e7
 #2 [ffffc900001b0cd0] crash_kexec at ffffffff8110d5f0
 #3 [ffffc900001b0ce0] oops_end at ffffffff810651fc
 #4 [ffffc900001b0d00] page_fault at ffffffff81e0102f
    [exception RIP: qlt_logo_completion_handler+305]
    RIP: ffffffffa00a1b24  RSP: ffffc900001b0db0  RFLAGS: 00010013
    RAX: ffffffffffffff90  RBX: ffff88828ea80003  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: 0000000000000046  RDI: ffff888354b7d958
    RBP: ffff888354b7d598   R8: 0000000000000001   R9: 0000000000000003
    R10: 000000000001fb89  R11: 0000000000000090  R12: 0000000000010000
    R13: 21000024ff3e2e75  R14: ffff88845ae80380  R15: ffff888354b7d968
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #5 [ffffc900001b0df8] qlt_logo_completion_handler at ffffffffa00a2662 [qla2xxx_scst]
 #6 [ffffc900001b0e68] qlt_logo_completion_handler at ffffffffa00a2efb [qla2xxx_scst]
 #7 [ffffc900001b0eb8] qlt_24xx_process_atio_queue at ffffffffa00a94b6 [qla2xxx_scst]
 #8 [ffffc900001b0f10] qla83xx_msix_atio_q at ffffffffa00a96d0 [qla2xxx_scst]
 #9 [ffffc900001b0f30] __handle_irq_event_percpu at ffffffff810e2460
#10 [ffffc900001b0f70] handle_irq_event_percpu at ffffffff810e2565
#11 [ffffc900001b0f98] handle_irq_event at ffffffff810e25cb
#12 [ffffc900001b0fb0] handle_edge_irq at ffffffff810e5eb6
#13 [ffffc900001b0fc8] do_IRQ at ffffffff81e01824
--- <IRQ stack> ---
#14 [ffffc900000e7de8] ret_from_intr at ffffffff81e00a0f
    [exception RIP: cpuidle_enter_state+237]
    RIP: ffffffff81a95f3c  RSP: ffffc900000e7e90  RFLAGS: 00000202
    RAX: ffff88846fb27600  RBX: ffff88846c5da000  RCX: 000000000000001f
    RDX: 0000000000000000  RSI: 0000000000000004  RDI: 0000000000000000
    RBP: ffffffff828db1e0   R8: 00020c0bcfa85505   R9: 000000000035a8d0
    R10: 071c71c71c71c71c  R11: 0000000000000020  R12: 0000000000000002
    R13: 00020c0bcf9aeb77  R14: 00020c0bcfa85505  R15: 0000000000000000
    ORIG_RAX: ffffffffffffffdb  CS: 0010  SS: 0018
#15 [ffffc900000e7ec0] cpuidle_enter at ffffffff81a9606b
#16 [ffffc900000e7ee0] do_idle at ffffffff810cc2bf
#17 [ffffc900000e7f20] cpu_startup_entry at ffffffff810cc465
#18 [ffffc900000e7f30] start_secondary at ffffffff81080654
#19 [ffffc900000e7f50] secondary_startup_64 at ffffffff810000d4

---

PID: 0      TASK: ffff88846bd97000  CPU: 7   COMMAND: "swapper/7"
 #0 [ffffc90000234bb0] machine_kexec at ffffffff810865ff
 #1 [ffffc90000234bf8] __crash_kexec at ffffffff8110c8e7
 #2 [ffffc90000234cc0] crash_kexec at ffffffff8110d5f0
 #3 [ffffc90000234cd0] oops_end at ffffffff810651fc
 #4 [ffffc90000234cf0] page_fault at ffffffff81e0102f
    [exception RIP: qlt_handle_login+145]
    RIP: ffffffffa013a0b9  RSP: ffffc90000234da8  RFLAGS: 00010007
    RAX: ffffffffffffff90  RBX: ffff88845b2f0003  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: 0000000000000046  RDI: ffff88835a6f39b0
    RBP: ffff88835a6f3598   R8: 0000000000000001   R9: 0000000000000003
    R10: ffff88835a6f3598  R11: 000000000000f026  R12: 0000000000010000
    R13: ffff88845a982c00  R14: 21000024ff422153  R15: ffff88835a6f39c0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #5 [ffffc90000234df8] qlt_handle_imm_notify at ffffffffa013ad7d [qla2xxx_scst]
 #6 [ffffc90000234e68] qlt_24xx_atio_pkt at ffffffffa013b653 [qla2xxx_scst]
 #7 [ffffc90000234eb8] qlt_24xx_process_atio_queue at ffffffffa013d195 [qla2xxx_scst]
 #8 [ffffc90000234f10] qla83xx_msix_atio_q at ffffffffa013d8a2 [qla2xxx_scst]
 #9 [ffffc90000234f30] __handle_irq_event_percpu at ffffffff810e2460
#10 [ffffc90000234f70] handle_irq_event_percpu at ffffffff810e2565
#11 [ffffc90000234f98] handle_irq_event at ffffffff810e25cb
#12 [ffffc90000234fb0] handle_edge_irq at ffffffff810e5eb6
#13 [ffffc90000234fc8] do_IRQ at ffffffff81e01824
--- <IRQ stack> ---
#14 [ffffc900000ffde8] ret_from_intr at ffffffff81e00a0f
    [exception RIP: cpuidle_enter_state+237]
    RIP: ffffffff81a959dc  RSP: ffffc900000ffe90  RFLAGS: 00000202
    RAX: ffff88847fde7600  RBX: ffff88846b417400  RCX: 000000000000001f
    RDX: 0000000000000000  RSI: 0000000000000007  RDI: 0000000000000000
    RBP: ffffffff828db1c0   R8: 000003b075d219c0   R9: ffff88847fddc8e0
    R10: 071c71c71c71c71c  R11: 0000000000000020  R12: 0000000000000002
    R13: 000003b075a89bf8  R14: 000003b075d219c0  R15: 0000000000000000
    ORIG_RAX: ffffffffffffffdb  CS: 0010  SS: 0018
#15 [ffffc900000ffec0] cpuidle_enter at ffffffff81a95b0b
#16 [ffffc900000ffee0] do_idle at ffffffff810cc2bf
#17 [ffffc900000fff20] cpu_startup_entry at ffffffff810cc465
#18 [ffffc900000fff30] start_secondary at ffffffff81080654
#19 [ffffc900000fff50] secondary_startup_64 at ffffffff810000d4

Hi I have just tested SCST 3.6 release and find out one weird problem when using lvm in blockio device. I created a lv and used it as block device. then I created a device group and added all targets and devices in it. after that changed the device group into transitioning, but device not closed or inactived in scst. but when device existed in any target it closed. long story short, in version 3.5, after going into transitioning all devices at that device group is closed. but in version 3.6 if a device not in any target, device is not closed. Is this behavior not a problem?

Details of this crash can be found after the VM is rebooted. Here is the kernel BUG (nthred.c) that is hit:

NOTE: I used Codenomicon ISCSI testing to hit this issue!

[75999.447928] [33903]: iscsi-scst: Creating connection 00000000b60b426d for sid 0x3f20000000000, cid 0 (initiator iqn.1994-09.org.freebsd:iscsi) [75999.471707] [33887]: iscsi-scst: Logout received from initiator iqn.1994-09.org.freebsd:iscsi [75999.471718] [33889]: iscsi-scst: Closing connection at initiator's iqn.1994-09.org.freebsd:iscsi request [75999.471815] [34498]: scst: TM fn NEXUS_LOSS_SESS/6 (mcmd 0000000087b23588, initiator iqn.1994-09.org.freebsd:iscsi, target iqn.2012-05.local:mailbox.target.ryan00001) [75999.471831] [33884]: scst: TM fn 6 (mcmd 0000000087b23588) finished, status 0 [75999.471838] [34498]: iscsi-scst: Freeing conn 00000000b60b426d (sess=00000000f4d68531, 0x3f20000000000 0, initiator iqn.1994-09.org.freebsd:iscsi) [75999.471848] [34498]: iscsi-scst: Freeing session 00000000f4d68531 (SID 3f20000000000) [76035.304588] [33903]: scst: Using security group "access_group" for initiator "iqn.1994-09.org.freebsd:iscsi" (target iqn.2012-05.local:mailbox.target.ryan00001) [76035.305532] [33903]: iscsi-scst: Session 00000000f4d68531 created: target 0000000000f93e99, tid 1, sid 0x4f30000000000, initiator iqn.1994-09.org.freebsd:iscsi [76035.305536] [33903]: iscsi-scst: Negotiated parameters: InitialR2T Yes, ImmediateData Yes, MaxConnections 1, MaxRecvDataSegmentLength 1048576, MaxXmitDataSegmentLength 8192, [76035.305538] [33903]: iscsi-scst: MaxBurstLength 262144, FirstBurstLength 65536, DefaultTime2Wait 0, DefaultTime2Retain 0, [76035.305539] [33903]: iscsi-scst: MaxOutstandingR2T 1, DataPDUInOrder Yes, DataSequenceInOrder Yes, ErrorRecoveryLevel 0, [76035.305540] [33903]: iscsi-scst: HeaderDigest None, DataDigest None, OFMarker No, IFMarker No, OFMarkInt 2048, IFMarkInt 2048, RDMAExtensions No [76035.305542] [33903]: iscsi-scst: Target parameters set for session 4f30000000000: QueuedCommands 32, Response timeout 90, Nop-In interval 30, Nop-In timeout 30 [76035.305582] [33903]: iscsi-scst: Creating connection 00000000b60b426d for sid 0x4f30000000000, cid 0 (initiator iqn.1994-09.org.freebsd:iscsi) [76035.419640] [34506]: iscsi-scst: ERROR: error 0 at sid:cid 0x4f30000000000:0, cmnd 0000000031af3b7e [76035.419646] [34506]: iscsi-scst: ERROR: Sending data failed: initiator iqn.1994-09.org.freebsd:iscsi (conn 00000000b60b426d), write_size 8240, write_state 1, res 0 [76035.419696] [33889]: iscsi-scst: CRITICAL ERROR: 0 6 25 [76035.419717] ------------[ cut here ]------------ [76035.419718] kernel BUG at /root/ontap-mediator/dist/scst/scst-3.5.0/iscsi-scst/kernel/nthread.c:1506! [76035.419754] invalid opcode: 0000 [#1] SMP PTI [76035.419757] CPU: 0 PID: 33889 Comm: iscsiwr0_1 Kdump: loaded Tainted: G OE --------- - - 4.18.0-240.el8.x86_64 #1 [76035.419758] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/03/2018 [76035.419765] RIP: 0010:iscsi_send+0x820/0x850 [iscsi_scst] [76035.419767] Code: 02 6e c0 41 b8 e1 05 00 00 48 c7 c1 c8 e4 6d c0 48 c7 c6 e8 14 6e c0 83 e2 3f 52 48 c7 c2 c4 14 6e c0 50 6a 00 e8 00 21 17 00 <0f> 0b e8 e9 f1 bd d3 8b 85 08 01 00 00 bb f5 ff ff ff 89 44 24 14 [76035.419768] RSP: 0018:ffffb87081823de8 EFLAGS: 00010292 [76035.419770] RAX: 0000000000000036 RBX: ffff8ee397ff3440 RCX: 0000000000000000 [76035.419771] RDX: 0000000000000007 RSI: 0000000000000246 RDI: 0000000000000246 [76035.419772] RBP: ffff8ee3edec1400 R08: 0000000000000585 R09: 0000000000000040 [76035.419773] R10: 2a2a2a524f525245 R11: 353220362030203a R12: ffff8ee397ff3488 [76035.419774] R13: ffff8ee39536e180 R14: ffff8ee3edec14b8 R15: ffff8ee397ff34a0 [76035.419775] FS: 0000000000000000(0000) GS:ffff8ee3ffc00000(0000) knlGS:0000000000000000 [76035.419776] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [76035.419777] CR2: 00007fbaba5a3000 CR3: 0000000027e0a001 CR4: 00000000003606f0 [76035.419813] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [76035.419814] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [76035.419815] Call Trace: [76035.419829] ? __schedule+0x2ae/0x700 [76035.419833] istwr+0x120/0x3b0 [iscsi_scst] [76035.419840] ? finish_wait+0x80/0x80 [76035.419844] ? iscsi_send+0x850/0x850 [iscsi_scst] [76035.419846] kthread+0x112/0x130 [76035.419848] ? kthread_flush_work_fn+0x10/0x10 [76035.419850] ret_from_fork+0x35/0x40 [76035.419852] Modules linked in: scst_vdisk(OE) isert_scst(OE) iscsi_scst(OE) scst(OE) dlm xt_CHECKSUM ipt_MASQUERADE xt_conntrack ipt_REJECT nf_nat_tftp nft_objref nf_conntrack_tftp nft_counter tun bridge stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi target_core_mod ib_srp scsi_transport_srp ib_ipoib nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set rdma_ucm ib_uverbs ib_umad nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 rdma_cm iw_cm ib_cm ib_core ip6_tables nft_compat ip_set nf_tables nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsock sunrpc intel_rapl_msr intel_rapl_common sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel vmw_balloon intel_rapl_perf joydev pcspkr vmw_vmci i2c_piix4 ip_tables xfs libcrc32c vmwgfx drm_kms_helper sr_mod cdrom ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod sg drm [76035.419968] ata_piix crc32c_intel libata serio_raw vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: dlm]

Any clues/hints what might be going on here?

I use Truenas Scale which ships scst 3.6.0.8557-~truenas+2. When my kubernetes cluster connects iscsi targets to pods, from time to time i get the following kernel panic. I can trigger it semi-reliably by randomly killing and spawning pods which require the iscsi targets. This happens both on kernels 5.10 and 5.15

[20421.307062] [20891]: scst: Attached to virtual device pvc-227dff34-8712-4b08-a0cd-2e9ee26fe99b (id 19)
[20421.308079] [1120566]: dev_vdisk: T10 device id for device pvc-227dff34-8712-4b08-a0cd-2e9ee26fe99b changed to 974a63b6449af8f
[20421.308116] [1120566]: dev_vdisk: USN for device pvc-227dff34-8712-4b08-a0cd-2e9ee26fe99b changed to 974a63b6449af8f
[20421.308121] list_del corruption. next->prev should be ffff955cb1ea2540, but was ffff955c54a32440
[20421.308128] ------------[ cut here ]------------
[20421.308130] kernel BUG at lib/list_debug.c:54!
[20421.308134] invalid opcode: 0000 [#1] SMP PTI
[20421.308137] CPU: 4 PID: 93251 Comm: kworker/4:1 Tainted: P           OE     5.15.62+truenas #1
[20421.308140] Hardware name: Default string Default string/SKYBAY, BIOS QZ01AR12 09/17/2017
[20421.308143] Workqueue: events vdev_inq_changed_fn [scst_vdisk]
[20421.308151] RIP: 0010:__list_del_entry_valid.cold+0x1d/0x47
[20421.308168] Code: c7 c7 e0 fe 15 bb e8 b4 f3 fe ff 0f 0b 48 89 fe 48 c7 c7 70 ff 15 bb e8 a3 f3 fe ff 0f 0b 48 c7 c7 20 00 16 bb e8 95 f3 fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 e0 ff 15 bb e8 81 f3 fe ff 0f 0b
[20421.308172] RSP: 0018:ffffb889443afe38 EFLAGS: 00010246
[20421.308175] RAX: 0000000000000054 RBX: ffffffffc1d15220 RCX: ffff9563ddd20448
[20421.308177] RDX: 0000000000000000 RSI: 0000000000000027 RDI: ffff9563ddd20440
[20421.308179] RBP: ffff955cb1ea2540 R08: 0000000000000000 R09: ffffb889443afc68
[20421.308181] R10: ffffb889443afc60 R11: ffffffffbb6d3268 R12: 0000000000000000
[20421.308183] R13: dead000000000122 R14: dead000000000100 R15: ffff955e1c6ab100
[20421.308186] FS:  0000000000000000(0000) GS:ffff9563ddd00000(0000) knlGS:0000000000000000
[20421.308188] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[20421.308190] CR2: 000055e2327427f8 CR3: 000000037fcc4005 CR4: 00000000003706e0
[20421.308193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[20421.308195] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[20421.308197] Call Trace:
[20421.308199]  <TASK>
[20421.308201]  scst_cm_dev_unregister+0x66/0xd0 [scst]
[20421.308217]  scst_cm_update_dev+0x41/0xc0 [scst]
[20421.308231]  process_one_work+0x1ee/0x390
[20421.308234]  worker_thread+0x53/0x3e0
[20421.308237]  ? process_one_work+0x390/0x390
[20421.308239]  kthread+0x124/0x150
[20421.308241]  ? set_kthread_struct+0x50/0x50
[20421.308244]  ret_from_fork+0x1f/0x30
[20421.308248]  </TASK>
[20421.308249] Modules linked in: scst_vdisk(OE) isert_scst(OE) iscsi_scst(OE) scst(OE) rdma_cm(E) iw_cm(E) ib_cm(E) ib_core(E) dlm(E) rpcsec_gss_krb5(E) wireguard(E) libchacha20poly1305(E) chacha_x86_64(E) poly1305_x86_64(E) curve25519_x86_64(E) libcurve25519_generic(E) libchacha(E) ip6_udp_tunnel(E) udp_tunnel(E) xt_nat(E) xt_tcpudp(E) veth(E) xt_conntrack(E) nft_chain_nat(E) xt_MASQUERADE(E) nf_nat(E) nf_conntrack_netlink(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) xfrm_user(E) xfrm_algo(E) nft_counter(E) xt_addrtype(E) nft_compat(E) nf_tables(E) nfnetlink(E) br_netfilter(E) bridge(E) msr(E) binfmt_misc(E) essiv(E) authenc(E) dm_crypt(E) dm_mod(E) 8021q(E) garp(E) stp(E) mrp(E) llc(E) bonding(E) ntb_netdev(E) ntb_transport(E) ntb_split(E) ntb(E) ioatdma(E) intel_rapl_msr(E) intel_rapl_common(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) snd_hda_codec_hdmi(E) kvm_intel(E) snd_hda_codec_realtek(E) snd_hda_codec_generic(E) mei_wdt(E) mei_hdcp(E) ledtrig_audio(E)
[20421.308285]  kvm(E) irqbypass(E) rapl(E) intel_cstate(E) evdev(E) intel_uncore(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_intel_sdw_acpi(E) mei_me(E) snd_hda_codec(E) wdat_wdt(E) i915(E) ir_rc6_decoder(E) pcspkr(E) watchdog(E) snd_hda_core(E) intel_wmi_thunderbolt(E) snd_hwdep(E) ttm(E) snd_pcm(E) snd_timer(E) drm_kms_helper(E) snd(E) soundcore(E) ee1004(E) rc_rc6_mce(E) mei(E) cec(E) sg(E) intel_pch_thermal(E) ite_cir(E) rc_core(E) intel_pmc_core(E) button(E) acpi_pad(E) nfsd(E) auth_rpcgss(E) fuse(E) nfs_acl(E) configfs(E) lockd(E) drm(E) grace(E) sunrpc(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) zfs(POE) zunicode(POE) zzstd(OE) zlua(OE) zcommon(POE) znvpair(POE) zavl(POE) icp(POE) spl(OE) raid10(E) raid456(E) async_raid6_recov(E) async_memcpy(E) async_pq(E) async_xor(E) async_tx(E) xor(E) raid6_pq(E) libcrc32c(E) crc32c_generic(E) raid1(E) raid0(E) multipath(E) linear(E) md_mod(E) ses(E) enclosure(E) scsi_transport_sas(E) sd_mod(E) crc32_pclmul(E) crc32c_intel(E)
[20421.308337]  ghash_clmulni_intel(E) nvme(E) igb(E) ahci(E) ahciem(E) i2c_algo_bit(E) xhci_pci(E) dca(E) nvme_core(E) t10_pi(E) crc_t10dif(E) e1000e(E) libahci(E) aesni_intel(E) crypto_simd(E) xhci_hcd(E) ptp(E) i2c_i801(E) intel_lpss_pci(E) cryptd(E) crct10dif_generic(E) libata(E) i2c_smbus(E) crct10dif_pclmul(E) crct10dif_common(E) pps_core(E) scsi_mod(E) intel_lpss(E) scsi_common(E) idma64(E) usbcore(E) usb_common(E) fan(E) wmi(E) video(E)
[20421.308373] ---[ end trace e89eb550d12b0ed7 ]---

Hi experts, I just got a kernel panic with SCST events program, without loading target driver(qlogic/emulex). Here are my steps for this:

Test env: RHEL8.4 with kernel version 4.18.0-305.19.1.el8_4.x86_64 SCST 3.6

Steps:

1. Compile SCST driver and install them.
2. Load driver scst.ko.

cat /sys/module/scst/version

3.6.0 3. Compile usr/events and run it.

./usr/events/events

11:3:27 [116378]: ./events:main:trace_flag 6f0a 11:3:27 [116378]: ./events:main:Setting allowed event code 0, issuer_name * 4. Kernel will panic, the 'vmcore-dmesg.txt' as below. [34984.671869] [116173]: scst: scst_print_config:2319:Enabled features: EXTRACHECKS, TRACING, DEBUG [35055.411331] usercopy: Kernel memory overwrite attempt detected to SLUB object 'kmalloc-512' (offset 232, size 296)! [35055.411385] ------------[ cut here ]------------ [35055.411386] kernel BUG at mm/usercopy.c:102! [35055.411408] invalid opcode: 0000 [#1] SMP PTI [35055.411428] CPU: 14 PID: 116378 Comm: events Kdump: loaded Tainted: G OE --------- - - 4.18.0-305.19.1.el8_4.x86_64 #1 [35055.411469] Hardware name: Intel Corporation S2600GZ/S2600GZ, BIOS SE5C600.86B.02.06.0007.082420181029 08/24/2018 [35055.411520] RIP: 0010:usercopy_abort+0x74/0x76 [35055.411538] Code: 0f 45 c6 51 48 89 f9 48 c7 c2 0d 2e 6e b7 41 52 48 c7 c6 c0 e2 6c b7 48 c7 c7 d8 2e 6e b7 48 0f 45 f2 48 89 c2 e8 0f c4 e3 ff <0f> 0b 49 89 f0 48 89 f9 44 89 e2 31 f6 48 c7 c7 34 2e 6e b7 e8 73 [35055.411624] RSP: 0018:ffffa03b4b77bd90 EFLAGS: 00010246 [35055.411645] RAX: 0000000000000067 RBX: ffff916dad62a6e8 RCX: 0000000000000000 [35055.411691] RDX: 0000000000000000 RSI: ffff916e9ee167c8 RDI: ffff916e9ee167c8 [35055.411715] RBP: 0000000000000128 R08: 0000000000000515 R09: 0000000000aaaaaa [35055.411739] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [35055.411764] R13: ffff916dad62a810 R14: 0000000000000128 R15: ffff916dad62a6e8 [35055.411789] FS: 00007f4bf298d740(0000) GS:ffff916e9ee00000(0000) knlGS:0000000000000000 [35055.411816] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [35055.411837] CR2: 00007f4bf21dfec0 CR3: 000000075d3d0001 CR4: 00000000000606e0 [35055.411861] Call Trace: [35055.411879] __check_heap_object+0xd3/0x100 [35055.411896] __check_object_size+0xff/0x16b [35055.411937] scst_event_get_event_from_user+0xbb/0x2e0 [scst] [35055.411971] scst_event_ioctl+0x5f1/0xde0 [scst] [35055.411992] do_vfs_ioctl+0xa4/0x680 [35055.412010] ? syscall_trace_enter+0x1d3/0x2c0 [35055.412028] ksys_ioctl+0x60/0x90 [35055.412043] __x64_sys_ioctl+0x16/0x20 [35055.412059] do_syscall_64+0x5b/0x1a0 [35055.412076] entry_SYSCALL_64_after_hwframe+0x65/0xca [35055.412096] RIP: 0033:0x7f4bf227a62b [35055.412111] Code: 0f 1e fa 48 8b 05 5d b8 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 2d b8 2c 00 f7 d8 64 89 01 48 [35055.412170] RSP: 002b:00007fff554132e8 EFLAGS: 00000206 ORIG_RAX: 0000000000000010 [35055.412196] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4bf227a62b [35055.412221] RDX: 00007fff55413320 RSI: 0000000040407501 RDI: 0000000000000003 [35055.412257] RBP: 0000000000603180 R08: 000000000000000e R09: 0000000000000000 [35055.412282] R10: 0000000000000004 R11: 0000000000000206 R12: 00007fff55413320 [35055.412307] R13: 0000000000000003 R14: 0000000000000000 R15: 000000000060314c [35055.412344] Modules linked in: scst(OE) dlm xfs libcrc32c sd_mod t10_pi sg mgag200 drm_kms_helper syscopyarea sysfillrect ahci sysimgblt fb_sys_fops libahci drm ixgbe libata crc32c_intel igb megaraid_sas mdio dca i2c_algo_bit dm_mirror dm_region_hash dm_log dm_mod

vmcore-dmesg.txt.gz

Compilation is broken for OS RHEL 9.1 kernel 5.14.0-162.6.1.el9_1.x86_64:

[root@scspa2671671002 ontap-mediator-1.6.0]# uname -a Linux scspa2671671002.rtp.openenglab.netapp.com 5.14.0-162.6.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Sep 30 07:36:03 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux [root@scspa2671671002 ontap-mediator-1.6.0]#

[root@scspa2671671002 ontap-mediator-1.6.0]# cat /etc/redhat-release Red Hat Enterprise Linux release 9.1 (Plow) [root@scspa2671671002 ontap-mediator-1.6.0]# cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="9.1 (Plow)" ID="rhel" ID_LIKE="fedora" VERSION_ID="9.1" PLATFORM_ID="platform:el9" PRETTY_NAME="Red Hat Enterprise Linux 9.1 (Plow)" ANSI_COLOR="0;31" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/9/" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9" REDHAT_BUGZILLA_PRODUCT_VERSION=9.1 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.1" [root@scspa2671671002 ontap-mediator-1.6.0]#

Compilation error below:

[2022-11-28 18:42:05] In file included from /tmp/ontap_mediator.UFnM45/ontap-mediator-1.6.0/ontap-mediator-1.6.0/dist/scst/scst-3.7.0-pre/scst/src/../include/scst.h:54, [2022-11-28 18:42:05] from /tmp/ontap_mediator.UFnM45/ontap-mediator-1.6.0/ontap-mediator-1.6.0/dist/scst/scst-3.7.0-pre/scst/src/scst_copy_mgr.c:17: [2022-11-28 18:42:05] ./include/scsi/scsi_cmnd.h:179:6: note: previous declaration of 'scsi_done' with type 'void(struct scsi_cmnd *)' [2022-11-28 18:42:05] 179 | void scsi_done(struct scsi_cmnd *cmd); [2022-11-28 18:42:05] | ^~~~~~~~~

I'm trying to build scst on a clean install of Centos 8 Stream and getting the following errors:

]# BUILD_2X_MODULE=y CONFIG_SCSI_QLA_FC=y CONFIG_SCSI_QLA2XXX_TARGET=y make all install if [ all = extraclean ]; then rm -f TAGS tags cscope.out; fi for d in scst iscsi-scst qla2x00t-32gbit/qla2x00-target srpt
scst_local fcst usr scstadmin; do
make -j$(nproc) -C "$d" all || break;
done make[1]: Entering directory '/root/scst/scst' cd src && make all make[2]: Entering directory '/root/scst/scst/src' make -C certs KDIR=/lib/modules/4.18.0-408.el8.x86_64/build \

make[3]: Entering directory '/root/scst/scst/src/certs' make[3]: Nothing to be done for 'all'. make[3]: Leaving directory '/root/scst/scst/src/certs' make -C /lib/modules/4.18.0-408.el8.x86_64/build M=/root/scst/scst/src \

make[3]: Entering directory '/usr/src/kernels/4.18.0-408.el8.x86_64' CC [M] /root/scst/scst/src/scst_copy_mgr.o CC [M] /root/scst/scst/src/scst_debug.o CC [M] /root/scst/scst/src/scst_dlm.o CC [M] /root/scst/scst/src/scst_event.o CC [M] /root/scst/scst/src/scst_lib.o CC [M] /root/scst/scst/src/scst_local_cmd.o CC [M] /root/scst/scst/src/scst_main.o CC [M] /root/scst/scst/src/scst_mem.o CC [M] /root/scst/scst/src/scst_no_dlm.o CC [M] /root/scst/scst/src/scst_pres.o CC [M] /root/scst/scst/src/scst_sysfs.o CC [M] /root/scst/scst/src/scst_targ.o In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_copy_mgr.c:17: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_copy_mgr.c:17: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_copy_mgr.c:17: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_copy_mgr.c:17: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_debug.c:31: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_debug.c:31: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_debug.c:31: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_debug.c:31: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_event.c:26: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_event.c:26: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_event.c:26: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_event.c:26: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ make[4]: *** [scripts/Makefile.build:315: /root/scst/scst/src/scst_debug.o] Error 1 make[4]: *** Waiting for unfinished jobs.... In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_dlm.c:26: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_dlm.c:26: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_dlm.c:26: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_dlm.c:26: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_lib.c:54: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_lib.c:54: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_lib.c:54: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_lib.c:54: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_local_cmd.c:22: /root/scst/scst/src/../include/backport.h:1473:31: error: redefinition of ‘scsi_cmd_to_rq’ static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_local_cmd.c:22: ./include/scsi/scsi_cmnd.h:171:31: note: previous definition of ‘scsi_cmd_to_rq’ was here static inline struct request *scsi_cmd_to_rq(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:64, from /root/scst/scst/src/scst_local_cmd.c:22: /root/scst/scst/src/../include/backport.h:1497:19: error: redefinition of ‘scsi_prot_ref_tag’ static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd) ^~~~~~~~~~~~~~~~~ In file included from /root/scst/scst/src/../include/scst.h:54, from /root/scst/scst/src/scst_local_cmd.c:22: ./include/scsi/scsi_cmnd.h:327:19: note: previous definition of ‘scsi_prot_ref_tag’ was here static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd)

Any help would be appreciated. Kernel is: 4.18.0-408.el8.x86_64. I've tried 3.6.x stable as well as the master branch with the same error. Compiles fine on Centos 7.9.

In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_debug.c:33: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_copy_mgr.c:17: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_copy_mgr.c:17: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_debug.c:33: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_copy_mgr.c:17: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_debug.c:33: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_copy_mgr.c:17: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_debug.c:33: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_event.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_event.c:26: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_mem.c:32: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_lib.c:63: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from ./include/linux/t10-pi.h:6, from /root/scst-3.6.x/scst/src/scst_lib.c:57: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/mm.h:21, from /root/scst-3.6.x/scst/src/scst_mem.c:25: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_event.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_event.c:26: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_lib.c:63: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_mem.c:32: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_lib.c:63: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_mem.c:32: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_dlm.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_dlm.c:26: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_dlm.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_dlm.c:26: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_no_dlm.c:22: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_no_dlm.c:22: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_no_dlm.c:22: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_no_dlm.c:22: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_tg.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_tg.c:26: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_tg.c:26: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_tg.c:26: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_main.c:37: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_local_cmd.c:22: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_main.c:37: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_local_cmd.c:22: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_main.c:37: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_local_cmd.c:22: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_main.c:37: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_local_cmd.c:22: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_sysfs.c:31: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_sysfs.c:31: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_sysfs.c:31: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_sysfs.c:31: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_debug.o] 错误 1 make[4]: *** 正在等待未完成的任务.... In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_targ.c:40: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/genhd.h:17, from ./include/linux/blkdev.h:11, from /root/scst-3.6.x/scst/src/../include/scst.h:37, from /root/scst-3.6.x/scst/src/scst_targ.c:40: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_targ.c:40: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_targ.c:40: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_pres.c:62: /root/scst-3.6.x/scst/src/../include/backport.h:1322:1: 错误：对‘percpu_ref_resurrect’的静态声明出现在非静态声明之后 { ^ In file included from ./include/linux/mm.h:21, from ./include/linux/ring_buffer.h:5, from ./include/linux/trace_events.h:6, from ./include/trace/syscall.h:7, from ./include/linux/syscalls.h:87, from /root/scst-3.6.x/scst/src/scst_pres.c:33: ./include/linux/percpu-refcount.h:119:6: 附注：‘percpu_ref_resurrect’的上一个声明在此 void percpu_ref_resurrect(struct percpu_ref *ref); ^~~~~~~~~~~~~~~~~~~~ In file included from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_pres.c:62: /root/scst-3.6.x/scst/src/../include/backport.h:2375:1: 错误：对‘fc_host_fpin_rcv’的静态声明出现在非静态声明之后 { ^ In file included from /root/scst-3.6.x/scst/src/../include/backport.h:56, from /root/scst-3.6.x/scst/src/../include/scst.h:64, from /root/scst-3.6.x/scst/src/scst_pres.c:62: ./include/scsi/scsi_transport_fc.h:810:6: 附注：‘fc_host_fpin_rcv’的上一个声明在此 void fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf); ^~~~~~~~~~~~~~~~ make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_no_dlm.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_dlm.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_mem.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_local_cmd.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_copy_mgr.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_tg.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_event.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_main.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_pres.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_targ.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_sysfs.o] 错误 1 make[4]: *** [scripts/Makefile.build:304：/root/scst-3.6.x/scst/src/scst_lib.o] 错误 1 make[3]: *** [Makefile:1522：module/root/scst-3.6.x/scst/src] 错误 2 make[3]: 离开目录“/usr/src/kernels/4.19.0-91.82.92.uelc20.x86_64” make[2]: *** [Makefile:75：all] 错误 2 make[2]: 离开目录“/root/scst-3.6.x/scst/src” make[1]: *** [Makefile:36：all] 错误 2 make[1]: 离开目录“/root/scst-3.6.x/scst”

Hi

I got this error on our storage server that caused kernel panic:

[10292670.543240] sqatgt(18/0): Registering initiator: pwwn=50:01:43:80:74:ab:7a:aa
[10292670.543256] [20014]: scst: Using security group "51:40:2e:c0:01:c2:c9:02" for initiator "50:01:43:80:74:ab:7a:aa" (target 51:40:2e:c0:01:c2:c9:02)
[10292670.543897] qla2xxx [0000:04:00.1]-d034:18: qla24xx_do_nack_work create sess success 000000000284bc05
[10292693.209939] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX
[10292693.210011] igb 0000:08:00.0 eno2: Link Speed was downgraded by SmartSpeed
[10292694.238548] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Down
[10292719.191912] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX
[10292719.191985] igb 0000:08:00.0 eno2: Link Speed was downgraded by SmartSpeed
[10292720.221548] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Down
[10292745.142919] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX
[10292745.142993] igb 0000:08:00.0 eno2: Link Speed was downgraded by SmartSpeed
[10292746.174590] igb 0000:08:00.0 eno2: igb: eno2 NIC Link is Down
[10292749.710836] ------------[ cut here ]------------
[10292749.710840] kernel BUG at /usr/src/packages/BUILD/scst-3.6.0.4/qla2x00t-32gbit/qla_target.c:2478!
[10292749.719996] invalid opcode: 0000 [#1] SMP PTI
[10292749.724612] CPU: 13 PID: 14160 Comm: kworker/13:2 Kdump: loaded Tainted: G        W  OE     5.4.52-HPDS #1
[10292749.734519] Hardware name: Supermicro Super Server/X10DRL-i, BIOS 2.0a 08/25/2016
[10292749.742270] Workqueue: qla_tgt_wq qlt_do_work [qla2xxx_scst]
[10292749.748189] RIP: 0010:qlt_pci_map_calc_cnt+0xbb/0xf0 [qla2xxx_scst]
[10292749.754710] Code: 02 00 00 02 31 c9 8b 43 28 83 f8 01 7e 16 8d 70 03 ba 67 66 66 66 89 f0 c1 fe 1f f7 ea d1 fa 29 f2 01 53 2c 89 c8 5b c3 0f 0b <0f> 0b 0f 0b 48 8b b0 f8 01 00 00 44 8b 88 a0 02 00 00 48 c7 c1 58
[10292749.773707] RSP: 0018:ffffa513c5ccfc68 EFLAGS: 00010246
[10292749.779183] RAX: ffff8ad10de0b220 RBX: ffffa513c5ccfc90 RCX: 0000000000000000
[10292749.786565] RDX: 0000000000000000 RSI: ffff8acd66af11f4 RDI: ffffa513c5ccfc90
[10292749.793950] RBP: ffff8acd5b139800 R08: 0000000000000009 R09: 0000000000000200
[10292749.801335] R10: 000000000000008b R11: 00000000007e7581 R12: ffffa513c5ccfc90
[10292749.808719] R13: ffff8ad1664e67b8 R14: ffff8ad10de0b428 R15: 0000000000000000
[10292749.816103] FS:  0000000000000000(0000) GS:ffff8ad16fb40000(0000) knlGS:0000000000000000
[10292749.824440] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10292749.830437] CR2: 00007fd8dc00b348 CR3: 000000086800a005 CR4: 00000000003606e0
[10292749.837820] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10292749.845204] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[10292749.852587] Call Trace:
[10292749.855308]  qlt_rdy_to_xfer+0x65/0x280 [qla2xxx_scst]
[10292749.860711]  ? sgv_pool_alloc+0xb8/0x8e0 [scst]
[10292749.865498]  sqa_rdy_to_xfer+0xb2/0x290 [qla2x00tgt]
[10292749.870726]  ? scst_adjust_sg+0x42/0xf0 [scst]
[10292749.875431]  scst_process_active_cmd+0x237/0x1630 [scst]
[10292749.881005]  ? scst_cmd_init_done+0xcd/0x5a0 [scst]
[10292749.886146]  ? scst_alloc_cmd+0x43/0xb0 [scst]
[10292749.890849]  ? __switch_to_asm+0x40/0x70
[10292749.895036]  ? __scst_rx_cmd.isra.35+0x40/0x80 [scst]
[10292749.900344]  sqa_qla2xxx_handle_cmd+0x20b/0x290 [qla2x00tgt]
[10292749.906261]  qlt_do_work+0x16b/0x320 [qla2xxx_scst]
[10292749.911398]  process_one_work+0x165/0x370
[10292749.915667]  worker_thread+0x49/0x3e0
[10292749.919583]  kthread+0xf8/0x130
[10292749.922980]  ? rescuer_thread+0x330/0x330
[10292749.927243]  ? kthread_bind+0x10/0x10
[10292749.931163]  ret_from_fork+0x35/0x40
[10292749.934991] Modules linked in: dm_mirror dm_region_hash dm_log ib_iser libiscsi scsi_transport_iscsi ib_srpt(OE) qla2x00tgt(OE) scst_vdisk(OE) isert_scst(OE) iscsi_scst(OE) scst(OE) dlm rdma_cm iw_cm ib_cm ib_core nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ebtable_filter ebtables ip6table_nat ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_mangle iptable_security iptable_raw iptable_filter rapidstor_rand(OE) rapidstor_lru(OE) rapidstor_fifo(OE) rapidstor(OE) dm_mod intel_rapl_msr intel_rapl_common sb_edac iTCO_wdt iTCO_vendor_support x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel cas_cache(OE) cas_disk(OE) ast drm_vram_helper ttm aesni_intel drm_kms_helper syscopyarea glue_helper sysfillrect crypto_simd
[10292749.935031]  sysimgblt cryptd fb_sys_fops pcspkr drm i2c_i801 lpc_ich mfd_core ses mei_me enclosure scsi_transport_sas mei sg ioatdma wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc32c_intel igb ahci qla2xxx_scst(OE) ptp libahci pps_core nvme_fc dca nvme_fabrics i2c_algo_bit libata i2c_core nvme_core megaraid_sas scsi_transport_fc

have any idea what caused this bug? may this be a hardware problem?

Bart good afternoon, you could help me, 5 years ago you helped solve the problem of freezes of cycles of recreations of the number of goals, a drop in reading speed during mass creation of goals, retention of block devices, I didn't want to bother with such a problem, but maybe I found an error, I encountered such an error if I give the target attribute the address of the machine for which if the target is intended, then the attribute for the target does not work correctly, it closes the target not only from all addresses, but also for this address of the machine, too, it turns out that everything closes access, there is nothing special in the logs, it turns out that the goal disappears from the list of available connections, I hope you will help with this problem, or I did not correctly understand the functionality of this parameter. The goal setting takes place with this parameter. echo "add_target_attribute RAM01 allowed_portal 192.168.0.50" > /sys/kernel/scst_tgt/targets/iscsi/mgmt Another oddity is if you add the server address to the rule. Then such a rule stops working and the goals remain in the list for everyone, the goals disappear only if it is any address other than the server, even if there is no such machine on the network, and the address number is taken from the ceiling and it does not physically exist on the network. If you need something from my side, write to me, I will provide, thank you. I apologize for the translation, the translator distorts the thoughts and words of the message..

Request for Enhancement to add hashing to IncomingUser target property data in scst.conf

As security is becoming more important, this is considered as a leak! Any plans to have a fix for this?

I recently upgraded my server running scst (latest release) to ubuntu 22.04. I am having issues. Things start out fine but once I start trying to vmotion vms to the datastore it either errors out on each VM, or eventually, the host loses connectivity and needs to be rebooted. This was fine prior to the Ubuntu upgrade. The logs are a bit over my head, I don’t see a smoking gun other than cmd aborts. I am assuming maybe something changed with the newer kernel. I am happy to enable whatever logging might be required and work with someone to try and nail down the issue. It is a bit hard to fully describe. Connectivity is generally present but not stable. I am fairly sure the equipment is fine as i have been running this for years and it crept up immediately after the ubuntu upgrade. There is no switch present. There are 2 esxi hosts, each have 2x 100gb connections (mellanox) directly into the ubuntu box which as 2x dual port adapters. each physical link is on a 255.255.255.248 subnet. I create 4x vmkernel interfaces per physical link, which i know sounds a little weird but i found that is the sweet spot to force esxi to really push the storage. The immediate log below has net.ipv4.conf.all.arp_ignore = 2 set. I wasn't aware of this setting prior and had not encountered any problems with my setup previously without it. I can see how not having that might have created a problem since the two interfaces are isolated, an arp for the other address on the wrong interface would definitely be odd. Its too early for me to call this resolved, but that setting does seem to have helped the storage at least be functional. I also changed the scst queue depths from 32 to 128. between these two sets of logs (the immediate one below being 128). It seems slightly more functional now I am not really sure why that would have helped though,generally i would associate a higher queue depth to pushing the storage, but maybe its letting things get further behind and then they catch up. I haven't pushed it much yet though. I wanted to have someone who had a better understanding of what these aborts mean to determine if they are somewhat normal or indicative of a problem.

Prior to those two changes above it was unusable/unstable after the ubuntu upgrade.

SCST:

[ 1897.244291] [21658]: scst: scst_rx_mgmt_fn:7000:TM fn ABORT_TASK/0 (mcmd 00000000bc68ffe8, initiator iqn.1998-01.com.vmware:esxi-2.psc.net:1134957192:65, target iqn.PSC.tgt) [ 1897.244295] [21658]: scst_rx_mgmt_fn:7008:sess=00000000ead6b191, tag_set 1, tag 1073741865, lun_set 1, lun=0, cmd_sn_set 1, cmd_sn 5544771, priv 000000008f780b48 [ 1897.244298] [21658]: scst_post_rx_mgmt_cmd:6929:Adding mgmt cmd 00000000bc68ffe8 to active mgmt cmd list [ 1897.244308] [2703]: scst_tm_thread:6815:Deleting mgmt cmd 00000000bc68ffe8 from active cmd list [ 1897.244320] [2703]: scst_abort_task:6302:Aborting task (cmd 00000000a5da2161, sn -20, set 1, tag 1073741865, queue_type 1) [ 1897.244324] [2703]: scst: scst_abort_cmd:5331:Aborting cmd 00000000a5da2161 (tag 1073741865, op READ(10)) [ 1897.244328] [2703]: scst_abort_cmd:5416:cmd 00000000a5da2161 (tag 1073741865) is being executed [ 1897.244330] [2703]: scst: scst_abort_cmd:5439:cmd 00000000a5da2161 (tag 1073741865, sn 4294967276) being executed/xmitted (state EXEC_WAIT, op READ(10), proc time 40 sec., timeout 60 sec.), deferring ABORT (cmd_done_wait_count 1, cmd_finish_wait_count 1, internal 0, mcmd fn 0 (mcmd 00000000bc68ffe8), initiator iqn.1998-01.com.vmware:esxi-2.psc.net:1134957192:65, target iqn.PSC.tgt) [ 1897.244339] [2703]: iscsi_on_abort_cmd:2540:Scheduling abort check for scst_cmd 00000000a5da2161 [ 1897.244344] [2703]: scst: scst_set_mcmd_next_state:5492:cmd_done_wait_count(1) not 0, preparing to wait [ 1897.244349] [302]: iscsi_cmnd_abort_fn:2484:Checking aborted scst_cmd 00000000a5da2161 (cmnd 0000000008fd665b) [ 1897.244353] [302]: __cmnd_abort:2323:Aborting cmd 0000000008fd665b, scst_cmd 00000000a5da2161 (scst state 4, ref_cnt 1, on_write_timeout_list 0, write_start 0, ITT 40000029, sn 5517670, op 1, r2t_len_to_receive 0, r2t_len_to_send 0, CDB op 28, size to write 0, outstanding_r2t 0, sess->exp_cmd_sn 5544771, conn 0000000049710af5, rd_task 00000000d69ad377, read_cmnd 0000000000000000, read_state 0) [ 1897.244360] [302]: __cmnd_abort:2348:Setting conn_tm_active for conn 0000000049710af5 [ 1897.244361] [302]: __cmnd_abort:2360:Mod timer on 4295369090 (conn 0000000049710af5)

Esxi

2022-04-24T02:00:34.747Z cpu31:2097914)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "eui.495345522d505343" state in doubt; requested fast path state update... 2022-04-24T02:00:35.058Z cpu11:2098194)WARNING: World: vm 2108243: 3817: vm not found 2022-04-24T02:00:55.223Z cpu31:2097914)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "eui.495345522d505343" state in doubt; requested fast path state update... 2022-04-24T02:00:55.532Z cpu13:2098194)WARNING: World: vm 2108623: 3817: vm not found 2022-04-24T02:00:56.223Z cpu18:2097913)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "eui.495345522d505343" state in doubt; requested fast path state update... 2022-04-24T02:00:56.535Z cpu8:2098194)WARNING: World: vm 2108242: 3817: vm not found

I also see these in dmesg quite a bit, from my understanding it shouldn't cause a problem but from past mailing list convos it looks like these should have been suppressed:

1452.609301] [2209078]: scst: scst_report_luns_local:46:Unsupported SELECT REPORT value 0x11 in REPORT LUNS command [61452.609345] [2209078]: scst: scst_report_luns_local:46:Unsupported SELECT REPORT value 0x11 in REPORT LUNS command [61452.676383] [2209078]: scst: scst_report_luns_local:46:Unsupported SELECT REPORT value 0x11 in REPORT LUNS command [61452.676448] [2209078]: scst: scst_report_luns_local:46:Unsupported SELECT REPORT value 0x11 in REPORT LUNS command [61452.676487] [2209078]: scst: scst_report_luns_local:46:Unsupported SELECT REPORT value 0x11 in REPORT LUNS command

earlier attempts before config changes:

scst dmesg

[59959.506899] [0]: conn_rsp_timer_fn:562:TM active: making conn 00000000dc6f1fcf RD active [59959.506917] [0]: conn_rsp_timer_fn:562:TM active: making conn 000000004128dbaa RD active [59961.035311] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59961.035316] [2311262]: execute_task_management:2627:TM req 00000000b178f2e7, ITT 80000062, RTT 80000077, sn 4355068, con 00000000c7135b92 [59961.035322] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59961.035325] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000b178f2e7 finished [59961.035329] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59961.035334] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 00000000b178f2e7) [59963.036057] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59963.036061] [2311262]: execute_task_management:2627:TM req 0000000069c6f3f8, ITT 80000059, RTT 80000077, sn 4355068, con 00000000c7135b92 [59963.036066] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59963.036070] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 0000000069c6f3f8 finished [59963.036073] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59963.036077] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 0000000069c6f3f8) [59963.090622] [0]: conn_rsp_timer_fn:562:TM active: making conn 00000000fdb858f7 RD active [59965.036091] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59965.036095] [2311262]: execute_task_management:2627:TM req 000000002e6e7792, ITT 8000000c, RTT 80000077, sn 4355068, con 00000000c7135b92 [59965.036101] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59965.036103] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 000000002e6e7792 finished [59965.036106] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59965.036111] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 000000002e6e7792) [59965.036190] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59965.036192] [2311262]: execute_task_management:2627:TM req 0000000091e85b21, ITT 8000000b, RTT 80000053, sn 4355068, con 00000000c7135b92 [59965.036196] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000053 not found [59965.036199] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 0000000091e85b21 finished [59965.036202] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59965.036205] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 0000000091e85b21) [59965.906472] [0]: conn_rsp_timer_fn:562:TM active: making conn 00000000f58edacd RD active [59966.875307] [1725755]: req_add_to_write_timeout_list:1087:Setting conn_tm_active for conn 00000000d4decc12 [59967.036496] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59967.036500] [2311262]: execute_task_management:2627:TM req 00000000b83d99bd, ITT 80000035, RTT 80000077, sn 4355068, con 00000000c7135b92 [59967.036505] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59967.036507] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000b83d99bd finished [59967.036510] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59967.036514] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 00000000b83d99bd) [59968.978145] [0]: conn_rsp_timer_fn:562:TM active: making conn 0000000094a01306 RD active [59969.036884] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59969.036888] [2311262]: execute_task_management:2627:TM req 000000002d62bbc5, ITT 80000018, RTT 80000077, sn 4355068, con 00000000c7135b92 [59969.036894] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59969.036897] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 000000002d62bbc5 finished [59969.036900] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59969.036904] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 000000002d62bbc5) [59971.038307] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59971.038314] [2311262]: execute_task_management:2627:TM req 00000000bd791c5c, ITT 8000000a, RTT 80000077, sn 4355068, con 00000000c7135b92 [59971.038319] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59971.038322] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000bd791c5c finished [59971.038326] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59971.038331] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 00000000bd791c5c) [59973.040328] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59973.040332] [2311262]: execute_task_management:2627:TM req 00000000087f9614, ITT 80000009, RTT 80000077, sn 4355068, con 00000000c7135b92 [59973.040337] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59973.040340] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000087f9614 finished [59973.040344] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59973.040348] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 00000000087f9614) [59975.042354] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59975.042359] [2311262]: execute_task_management:2627:TM req 000000006e4ada2b, ITT 80000037, RTT 80000077, sn 4355068, con 00000000c7135b92 [59975.042364] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59975.042367] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 000000006e4ada2b finished [59975.042371] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59975.042375] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 000000006e4ada2b) [59975.876686] [2507423]: req_add_to_write_timeout_list:1087:Setting conn_tm_active for conn 0000000019ddb073 [59977.044351] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59977.044354] [2311262]: execute_task_management:2627:TM req 000000001b998dc7, ITT 8000005f, RTT 80000053, sn 4355068, con 00000000c7135b92 [59977.044359] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000053 not found [59977.044362] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 000000001b998dc7 finished [59977.044365] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59977.044370] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 000000001b998dc7) [59977.937543] [0]: conn_rsp_timer_fn:562:TM active: making conn 00000000d4decc12 RD active [59979.045093] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59979.045098] [2311262]: execute_task_management:2627:TM req 00000000f5769a79, ITT 8000005b, RTT 80000077, sn 4355068, con 00000000c7135b92 [59979.045106] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59979.045109] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000f5769a79 finished [59979.045113] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59979.045118] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 00000000f5769a79) [59979.045212] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59979.045215] [2311262]: execute_task_management:2627:TM req 0000000036485a65, ITT 80000025, RTT 80000053, sn 4355068, con 00000000c7135b92 [59979.045220] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000053 not found [59979.045223] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 0000000036485a65 finished [59979.045226] [2311262]: iscsi-scst: iscsi_send_task_mgmt_resp:3635:iSCSI TM fn 1 finished, status 1, dropped 0 [59979.045230] [2311262]: iscsi_get_send_cmnd:517:Going to send TM response 000000000d5ed8e9 (status 1, fn 1, parent_req 0000000036485a65) [59981.047145] [2311262]: iscsi-scst: execute_task_management:2625:iSCSI TM fn 1 [59981.047151] [2311262]: execute_task_management:2627:TM req 00000000f41b1b69, ITT 80000031, RTT 80000077, sn 4355068, con 00000000c7135b92 [59981.047157] [2311262]: cmnd_abort_pre_checks:2430:cmd RTT 80000077 not found [59981.047161] [2311262]: iscsi_send_task_mgmt_resp:3634:TM req 00000000f41b1b69 finished

Esxi vmkernel output

2022-04-23T17:01:39.711Z cpu24:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d9001810c0 failed: No connection 2022-04-23T17:01:39.712Z cpu24:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d900056740 failed: No connection 2022-04-23T17:01:39.962Z cpu23:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d90006a740 failed: No connection 2022-04-23T17:01:40.213Z cpu22:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d900062540 failed: No connection 2022-04-23T17:01:40.464Z cpu20:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d92f60afc0 failed: No connection 2022-04-23T17:01:40.714Z cpu18:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d900062540 failed: No connection 2022-04-23T17:01:40.965Z cpu30:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d90014fac0 failed: No connection 2022-04-23T17:01:41.216Z cpu30:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d9000d2c00 failed: No connection 2022-04-23T17:01:41.216Z cpu30:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d9000a8680 failed: No connection 2022-04-23T17:01:41.466Z cpu21:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d9001c7840 failed: No connection 2022-04-23T17:01:41.482Z cpu1:2097445)iser: iser_ScsiTaskMgmt: path vmhba65:3:0:0 TaskMgmt 0x4538c929bf00 invoked abort on CmdSN 0x0 2022-04-23T17:01:41.482Z cpu1:2097445)iser: iser_AbortCommands: session 0x4316de037270 taskMgmt 0x4538c929bf00 2022-04-23T17:01:41.482Z cpu1:2097445)iser: iser_AbortCommands: session 0x4316de037270 failing sc 0x45d900d01a08 itt 0x77 state 3 status 1 2022-04-23T17:01:41.482Z cpu1:2097445)iser: iser_SendMgmtTask: session: 0x4316de037270 tmf set timeout 2022-04-23T17:01:41.482Z cpu11:2097898)iser: iser_TmfResponse: TMF task response: 1, task state: 5 2022-04-23T17:01:41.482Z cpu1:2097445)iser: iser_AbortCommands: task 0x4316de139040 tmf state 5 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_ScsiTaskMgmt: path vmhba65:13:0:0 TaskMgmt 0x4538e771bef0 invoked abort on CmdSN 0x1d87536 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_AbortCommands: session 0x4316de4ce800 taskMgmt 0x4538e771bef0 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_ScsiTaskMgmt: path vmhba65:3:0:0 TaskMgmt 0x4538e771bef0 invoked abort on CmdSN 0x1d87536 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_AbortCommands: session 0x4316de037270 taskMgmt 0x4538e771bef0 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_AbortCommands: session 0x4316de037270 failing sc 0x45b9323e83c8 itt 0x53 state 3 status 1 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_SendMgmtTask: session: 0x4316de037270 tmf set timeout 2022-04-23T17:01:41.553Z cpu11:2097898)iser: iser_TmfResponse: TMF task response: 1, task state: 5 2022-04-23T17:01:41.553Z cpu27:2115657)iser: iser_AbortCommands: task 0x4316de1407e0 tmf state 5 2022-04-23T17:01:41.717Z cpu22:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d90001f4c0 failed: No connection 2022-04-23T17:01:41.794Z cpu17:2097935)HBX: 747: Reading HB at 4030464 on vol 'SAN.PSC.Net' failed: No connection 2022-04-23T17:01:41.972Z cpu29:2102106)WARNING: SVM: 1861: scsi0:0 IO failed on handle 2497915, childToken 0x45d90002bf40 failed: No connection