etcd网络通讯层(一)——rafthttp中重要的接口

       前面我们提到etcd-raft模块，etcd-raft模块并未提供网络层的相关实现，而是将待发送的消息封装进Ready实例返回给上层模块，然后由上层模块决定如何将这些消息发送到集群中的其他节点。etcd将网络层的相关实现单独封装成一个模块，也就是etcd-rafthttp模块。从而降低了etcd-raft和etcd-rafthttp之间的耦合，提高程序的可扩展性。

      etcd-rafthttp的模块主要在https://github.com/etcd-io/etcd/tree/master/etcdserver/api/rafthttp 目录下。

      在etcd集群中，每个节点启动时都会与集群中的其他节点建立连接(每个节点即是服务端,也是客户端)，这里以三个节点为例，最终形成如图的网络结构。

      在etcd-rafthttp模块中，主要使用两个消息传输通道，Stream和Pipeline消息通道。

      Stream消息通道维护的HTTP长连接，主要负责传输数据量较小、发送比较频繁的消息，例如，MsgApp消息、MsgHeartbeat消息、MsgVote消息等。

     而Pipeline消息通道在传输数据完成后立即关闭连接，主要负责传输数据量较大、发送频率较低的消息，例如，MsgSnap消息等。

 

一、rafthttp.Transporter接口

Transporter接口是rafthttp包的核心接口，它定义了etcd网络层的核心功能，其具体定义如下：

 type Transporter interface {
	// Start starts the given Transporter.
	// Start MUST be called before calling other functions in the interface.
	Start() error  //初始化操作
	// Handler returns the HTTP handler of the transporter.
	// A transporter HTTP handler handles the HTTP requests
	// from remote peers.
	// The handler MUST be used to handle RaftPrefix(/raft)
	// endpoint.
	Handler() http.Handler		//创建Handler实例，并关联到指定URL上
	// Send sends out the given messages to the remote peers.
	// Each message has a To field, which is an id that maps
	// to an existing peer in the transport.
	// If the id cannot be found in the transport, the message
	// will be ignored.
	Send(m []raftpb.Message)  //发送消息
	// SendSnapshot sends out the given snapshot message to a remote peer.
	// The behavior of SendSnapshot is similar to Send.
	SendSnapshot(m snap.Message)   //发送快照数据
	// AddRemote adds a remote with given peer urls into the transport.
	// A remote helps newly joined member to catch up the progress of cluster,
	// and will not be used after that.
	// It is the caller's responsibility to ensure the urls are all valid,
	// or it panics.
	AddRemote(id types.ID, urls []string)   //在集群中添加一个节点时，其他节点会通过该方法添加该新加入节点的信息
	// AddPeer adds a peer with given peer urls into the transport.
	// It is the caller's responsibility to ensure the urls are all valid,
	// or it panics.
	// Peer urls are used to connect to the remote peer.
	//Peer接口是当前节点对集群中其他节点的抽象表示，而结构体Peer则是Peer接口的一个具体实现
	//下面几个方法是对Peer的操作
	AddPeer(id types.ID, urls []string)
	// RemovePeer removes the peer with given id.
	RemovePeer(id types.ID)
	// RemoveAllPeers removes all the existing peers in the transport.
	RemoveAllPeers()
	// UpdatePeer updates the peer urls of the peer with the given id.
	// It is the caller's responsibility to ensure the urls are all valid,
	// or it panics.
	UpdatePeer(id types.ID, urls []string)
	// ActiveSince returns the time that the connection with the peer
	// of the given id becomes active.
	// If the connection is active since peer was added, it returns the adding time.
	// If the connection is currently inactive, it returns zero time.
	ActiveSince(id types.ID) time.Time
	// ActivePeers returns the number of active peers.
	ActivePeers() int
	// Stop closes the connections and stops the transporter.
	Stop()  //关闭操作
}

 Start()方法完成初始化操作

 Handler() 创建Handler实例，并关联到指定的URL上，创建HTTP服务时使用

 Send()方法用于给对端发送消息

 SendSnapshot()用于向对端发送快照数据

 Peer接口是当前节点对集群中其他节点(对端)的抽象表示,AddPeer、RemovePeer、RemoveAllPeer、UpdatePeer分别用于添加节点、删除节点、删除所有节点和更新节点的操作。

 

二、Raft接口 

     Raft接口也是rafthttp包中非常重要的接口，用于集群中当前节点的网络层(etcd-rafthttp)向etcd-raft层发送消息。

type Raft interface {
	Process(ctx context.Context, m raftpb.Message) error  //将指定消息传递到etcd-raft模块进行处理
	IsIDRemoved(id uint64) bool     //检测当前节点是否从当前集群中被移除
	ReportUnreachable(id uint64)  //通知底层的etcd-raft模块，当前节点与指定的节点无法连通
	ReportSnapshot(id uint64, status raft.SnapshotStatus)  //通知底层的etcd-raft模块，快照数据是否发送成功
}

Raft接口提供了当前节点的网络层向底层的etcd-raft模块发送消息的功能

 

三、Peer接口

type Peer interface {
	// send sends the message to the remote peer. The function is non-blocking
	// and has no promise that the message will be received by the remote.
	// When it fails to send message out, it will report the status to underlying
	// raft.
	//发送单个消息给对端，该方法时非阻塞的，如果出现发送失败，则会将失败信息报告给底层的Raft接口
	send(m raftpb.Message)

	// sendSnap sends the merged snapshot message to the remote peer. Its behavior
	// is similar to send.
	//发送snap.Message，其他行为与上面的send()方法类似
	sendSnap(m snap.Message)

	// update updates the urls of remote peer.
	//更新对应节点暴露的URL地址
	update(urls types.URLs)

	// attachOutgoingConn attaches the outgoing connection to the peer for
	// stream usage. After the call, the ownership of the outgoing
	// connection hands over to the peer. The peer will close the connection
	// when it is no longer used.
	/*
	将指定连接与当前的Peer绑定，Peer会将该连接作为Stream消息通道使用
	当Peer不再使用该连接时，会将该连接关闭
	*/
	attachOutgoingConn(conn *outgoingConn)
	// activeSince returns the time that the connection with the
	// peer becomes active.
	/*
	activeSince返回与其他端(peer)的连接变为active状态的时间。
	*/
	activeSince() time.Time
	// stop performs any necessary finalization and terminates the peer
	// elegantly.
	/*
	关闭当前Peer实例，会关闭底层的网络连接
	*/
	stop()
}

 Peer接口是当前节点对集群中其他节点(对端)的抽象表示，该接口主要作用是向对端(集群中的其他节点)发送消息的功能。

后面会对这些接口的实现进行详细分析。