warren/libiscsi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

iser: enhance connection procedure

Browse Source 
This patch is used to fix the following problems in the current connection
method:
1. iscsi_iser_connect() waits until the connection is established or failed,
and may block the caller for a long time.
2. Although there's a cm_thread handles communication events, but in fact it
has no effects after the connection is established.
3. Resources are not released properly after reconnection failed. And once we
try to reconnect again, the resources will leak permanently.
(see iscsi_reconnect()).

This patch eliminate cm_thread and handle communication events in the caller
thread.
Connection procedure:
1. Create a mock fd by eventfd() (or just use old_iscsi->fd while reconnecting),
and assign it to iscsi->fd.
2. Create communication event channel, make it non-blocking and dup the
notifier fd to iscsi->fd.
3. Handle communication events by iscsi_which_events()/iscsi_service() loop
until connection established or falied.
4. If connection is established successfully, dup the notifier fd of completion
queue (CQ) events to iscsi->fd.
5. Handle completion queue (CQ) events by iscsi_which_events()/iscsi_service()
loop.
The entire procedure is non-blocking.

After established, whenever iscsi_service() is called with revents=0 or
queue_pdu() is called with a NOP pdu, communication events will be checked.

When connection failed, iser transport cleanup itself before callbacks.

Signed-off-by: wanghonghao <wanghonghao@bytedance.com>
This commit is contained in:
wanghonghao
2020-04-06 21:07:54 +08:00
parent cdcb35e6c6
commit 2212021747
3 changed files with 141 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
include/iser-private.h
View File

@@ -29,8 +29,6 @@
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <sys/ioctl.h>
#include <pthread.h>
#include <semaphore.h>
#ifdef __linux
@@ -72,11 +70,6 @@ enum desc_type {
ISCSI_CONTROL = 0,
ISCSI_COMMAND};
enum conn_state{
CONN_ERROR = 0,
CONN_DISCONNECTED,
CONN_ESTABLISHED};
enum data_dir{
DATA_WRITE = 0,
DATA_READ};
@@ -179,22 +172,19 @@ struct iser_buf_chunk {
struct iser_conn {
struct rdma_cm_id *cma_id;
struct rdma_event_channel *cma_channel;
struct rdma_cm_event *cma_event;
struct ibv_pd *pd;
struct ibv_cq *cq;
struct ibv_qp *qp;
struct ibv_comp_channel *comp_channel;
struct ibv_recv_wr rx_wr[ISER_MIN_POSTED_RX];
int rdma_connect_sent;
sem_t sem_connect;
struct ibv_recv_wr rx_wr[ISER_MIN_POSTED_RX];
struct ibv_mr *login_resp_mr;
unsigned char *login_resp_buf;
pthread_t cmthread;
struct iser_rx_desc *rx_descs;
uint32_t num_rx_descs;
unsigned int rx_desc_head;
@@ -204,8 +194,6 @@ struct iser_conn {
int min_posted_rx;
uint16_t max_cmds;
enum conn_state conn_state;
struct iser_tx_desc *tx_desc;
struct iser_buf_chunk *buf_chunk;
};