Commits
Kui Liu authored and Konstantin Khorenko committed 60bf2c64fdd
fs/fuse kio: always ack RIO_MSG_RDMA_READ_REQ received from csd In our userspace RDMA implementation, it is required that every RIO_MSG_RDMA_READ_REQ msg must be acked strictly in order. However this rule can be broken due to a bug in kio, which though is triggered by very abnormal hardware behaviour that it can take very long time (>10s) for a WR to complete. This happens in the read workload with large block size that the the client needs to issue RDMA READ wr to pull the data portion of a response msg returned by csd. When this operation takes very long time to complete for a msg, it will block responses to requests after it from being sent out by csd for as long as it can take. As a result, these requests will be killed due to calendar timeout. However when these responses arrives later in form of RIO_MSG_RDMA_READ_REQ msg, they will be ignored silently due to missing request msg without returning corresponding RIO_MSG_RDMA_RAD_ACK back, therefore breaks the expectation of ordered ack on the side of csd. Since the rio connection is still in working state, a later valid msg exchange will trigger the BUGON check of rb->xid in csd, causing it to crash. This patch makes sure client will always ack every RIO_MSG_RDMA_READ_REQ received and in order to avoid crashing csd. However it can't address any performance impact due to the strange hardware behaviour that it takes abnormal long time for a WR to complete. https://pmc.acronis.work/browse/VSTOR-76834 https://pmc.acronis.work/browse/VSTOR-70758 https://pmc.acronis.work/browse/VSTOR-60807 https://pmc.acronis.work/browse/VSTOR-57903 Signed-off-by: Liu Kui <Kui.Liu@acronis.com> Acked-by: Alexey Kuznetsov <kuznet@acronis.com>