[feat] add http deployment method by liuminjian · Pull Request #365 · opencurve/curveadm

liuminjian · 2023-11-24T01:45:08Z

Issue Number: #328

1.增加http daemon的部署方式，host增加protocol和http port字段

2.在每个host节点上部署http daemon

3.部署http daemon后能正常部署curve集群

遗留问题：目前enter指令不支持http方式

caoxianfei1 · 2023-11-24T08:22:05Z

@liuminjian BTW, we can provide command curveadm http start/stop is better? And the http service commit has already done.
So you can refer the commit.

liuminjian · 2023-11-24T10:03:51Z

@liuminjian BTW, we can provide command curveadm http start/stop is better? And the http service commit has already done. So you can refer the commit.

ok

Wine93 · 2023-11-29T02:15:03Z


 	subname := fmt.Sprintf("host=%s volume=%s", options.Host, volume)
-	t := task.NewTask("Add Target", subname, hc.GetSSHConfig())
+	t := task.NewTask("Add Target", subname, hc.GetSSHConfig(), hc.GetHttpConfig())


这里的 GetSSHConfig 和 GetHttpConfig 可以合并成一个函数，比如 GetConnectConfig，然后在 NewTask 里面再做处理，这样后续有其他的连接方式控制，扩展性也会好一些。

BTW：尽可能保持变量命名方式的一致性，比如用大写，那么可以用 SSH 和 HTTP，或者都用 ssh 和 http。

Wine93 · 2023-11-29T02:15:10Z

 	subname := fmt.Sprintf("host=%s role=%s containerId=%s",
 		dc.GetHost(), dc.GetRole(), tui.TrimContainerId(containerId))
-	t := task.NewTask("Balance Leader", subname, hc.GetSSHConfig())
+	t := task.NewTask("Balance Leader", subname, hc.GetSSHConfig(), hc.GetHttpConfig())


Wine93 · 2023-11-29T02:25:41Z

@@ -0,0 +1,44 @@
+/*


这个命令我认为叫 deamon 更好一些。
另外目前的做法是安装 pigeon 二进制，然后在 http start 时启动这个二进制，其实这是没必要的，我们可以把 pigeon 的 start 命令逻辑复制到 http start 这个命令中，详见：pigeon start。

目前：

curveadm http start -> pigeon start

期待：

curveadm deamon start

Wine93 · 2023-11-29T02:28:14Z

@@ -0,0 +1,71 @@
+/*


这个 HTTP 的核心逻辑建议移动到 internal 目录下，建议叫 daemon。

Wine93 · 2023-12-13T08:03:04Z

+
+
+    # generate http service config file
+    local httpConfpath="${g_curveadm_home}/conf/pigeon.yaml"


建议在 curveadm 的主目录下新建一个 deamon 目录，配置文件 pigeon.yaml 以及一些 daemon 的日志都可以放到这个目录下

Wine93 · 2023-12-13T08:04:14Z

+/*
+* Project: Curveadm
+* Created Date: 2023-03-31
+* Author: wanghai (SeanHai)


这里的作者建议改一下~

Wine93 · 2023-12-13T08:04:39Z

+func NewDaemonCommand(curveadm *cli.CurveAdm) *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "daemon",
+		Short: "Manage http service",


Manage deamon service

Wine93 · 2023-12-13T08:05:04Z

 		target.NewTargetCommand(curveadm),         // curveadm target ...
 		pfs.NewPFSCommand(curveadm),               // curveadm pfs ...
 		monitor.NewMonitorCommand(curveadm),       // curveadm monitor ...
+		daemon.NewDaemonCommand(curveadm),         // curveadm http


curveadm deamon ...

Wine93 · 2023-12-13T08:05:51Z


 # output
 OUTPUT := bin/curveadm
+SERVER_OUTPUT := bin/pigeon


这个是不是不需要了？

Wine93 · 2023-12-13T08:37:19Z

+/*
+* Project: Curveadm
+* Created Date: 2023-03-31
+* Author: wanghai (SeanHai)


Wine93 · 2023-12-13T08:37:24Z

+/*
+* Project: Curveadm
+* Created Date: 2023-03-31
+* Author: wanghai (SeanHai)


Wine93 · 2023-12-13T08:39:03Z

+
+	cmd := &cobra.Command{
+		Use:     "stop",
+		Short:   "Stop http service",


Stop deamon service

Wine93 · 2023-12-13T08:39:14Z

+
+	cmd := &cobra.Command{
+		Use:     "start [OPTIONS]",
+		Short:   "Start http service",


Start deamon service

Wine93 · 2023-12-19T08:26:32Z

+ */
+
+/*
+* Project: Curveadm


Wine93 · 2023-12-19T08:26:56Z

+ */
+
+/*
+* Project: Curveadm


Wine93 · 2023-12-19T08:27:34Z

+ */
+
+/*
+* Project: Curveadm


Wine93 · 2023-12-19T08:27:53Z

+
+const (
+	STOP_EXAMPLR = `Examples:
+  $ curveadm daemon stop -c ~/.curveadm/daemon/config/pigeon.yaml  # Stop an daemon http service`


$ curveadm daemon stop # Stop daemon service

Wine93 · 2023-12-19T08:28:04Z

+
+const (
+	START_EXAMPLR = `Examples:
+  $ curveadm daemon start -c ~/.curveadm/daemon/config/pigeon.yaml  # Start an daemon http service to receive requests`


$ curveadm daemon start # Start daemon service

Wine93 · 2023-12-19T08:31:22Z


+func (hc *HostConfig) GetProtocol() string { return hc.getString(CONFIG_PROTOCOL) }
 func (hc *HostConfig) GetSSHConfig() *module.SSHConfig {
+


多余的空格.

Wine93 · 2023-12-19T08:31:53Z

 }
+
+func (hc *HostConfig) GetHTTPConfig() *module.HTTPConfig {
+


Wine93 · 2023-12-19T08:34:34Z

+	}
+}
+
+func (hc *HostConfig) GetConnectConfig() *module.ConnectConfig {


既然有了 GetConnectConfig 函数，GetSSHConfig 和 GetHTTPConfig 这 2 个函数应该可以移除了.

Wine93 · 2023-12-19T08:35:12Z

+*  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+*  See the License for the specific language governing permissions and
+*  limitations under the License.
+ */


建议加上 Author 等相关信息。

Wine93 · 2023-12-19T08:35:45Z

+*  See the License for the specific language governing permissions and
+*  limitations under the License.
+ */
+


Wine93 · 2023-12-19T08:35:58Z

+*  See the License for the specific language governing permissions and
+*  limitations under the License.
+ */
+


Wine93 · 2023-12-19T08:36:15Z

+*  See the License for the specific language governing permissions and
+*  limitations under the License.
+ */
+


Wine93 · 2023-12-19T08:37:17Z

+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ */
+


Wine93 · 2023-12-19T08:37:28Z

@@ -0,0 +1,166 @@
+/*
+ *  Copyright (c) 2021 NetEase Inc.


2021 -> 2023.

Wine93 · 2023-12-19T08:45:36Z

+		return
+	}
+
+	if cfg.Protocol == SSH_PROTOCOL {


这里的逻辑可以简化一下：

if cfg.Protocol == HTTP_PROTOCOL { client, err = NewHTTPClient(*cfg.GetHTTPConfig()) } else { client, err = NewSSHClient(*cfg.GetSSHConfig()) } if (err != nil) { ... } return client, err

Wine93 · 2023-12-19T08:46:07Z

 url = "${g_db_path}"
+__EOF__
+    fi
+


多余的空格

Wine93 · 2023-12-19T08:49:57Z

+}
+
+func (client *HttpClient) Upload(localPath string, remotePath string) (err error) {
+	bodyBuf := &bytes.Buffer{}


这里的 upload 和 download 都太麻烦了，可以看下 resty 库，这个模块只要准备要命令就可以了，客户端这些逻辑可以交给 resty 去处理。

Wine93 · 2023-12-19T08:50:21Z

+}
+
+func (client *HttpClient) RunCommand(ctx context.Context, command string) (out []byte, err error) {
+	data := make(map[string]interface{})


Wine93 · 2023-12-19T08:52:32Z

+
+
+    # generate http service config file
+    local httpConfpath="${g_curveadm_home}/daemon/conf/pigeon.yaml"


这个逻辑可以单独拆成一个函数，包括上面的 curveadm.cfg 都可以单独拆一下。

caoxianfei1 · 2023-12-26T01:57:42Z

@liuminjian please fix the conflict.

Signed-off-by: liuminjian <liuminjian@chinatelecom.cn>

liuminjian · 2023-12-26T02:23:15Z

@caoxianfei1 fixed

liuminjian force-pushed the feat/http branch 2 times, most recently from 6c5c13f to 9624e34 Compare November 24, 2023 02:04

aspirer requested review from Wine93 and caoxianfei1 November 24, 2023 02:18

liuminjian force-pushed the feat/http branch from 9624e34 to 6db1c35 Compare November 24, 2023 09:58

caoxianfei1 added the developer activities label Nov 27, 2023

Wine93 reviewed Nov 29, 2023

View reviewed changes

liuminjian force-pushed the feat/http branch from 6db1c35 to 281ba47 Compare November 29, 2023 04:13

liuminjian force-pushed the feat/http branch 5 times, most recently from a6bebd0 to 36a36a4 Compare December 13, 2023 03:35

Wine93 reviewed Dec 13, 2023

View reviewed changes

liuminjian force-pushed the feat/http branch 2 times, most recently from d51ef71 to a7b3b84 Compare December 13, 2023 08:35

Wine93 reviewed Dec 13, 2023

View reviewed changes

liuminjian force-pushed the feat/http branch from a7b3b84 to 5afef56 Compare December 13, 2023 08:40

liuminjian force-pushed the feat/http branch from 68f2c6f to 1afda6d Compare December 15, 2023 06:16