Lifelong learning supporting non-structure by jaypume · Pull Request #352 · kubeedge/sedna

jaypume · 2022-08-22T01:18:50Z

No description provided.

Signed-off-by: luosiqi <1587295470@qq.com>

Sedna lifelong learning supports unstructured data based on semantic segmentation example

Signed-off-by: luosiqi <1587295470@qq.com>

Code check and base model improvement of unstructured lifelong learning framework

JimmyYang20 · 2022-08-22T01:22:03Z

@luosiqi Delete code files irrelevant to the scenarios in example folder.

JimmyYang20 · 2022-08-22T01:25:00Z

+    return CPA
+
+
+if __name__ == '__main__':


@luosiqi put the test code elsewhere, e.g. ./test/test_basemodel.py

JimmyYang20 · 2022-08-22T01:31:39Z

+
+
+def train_args():
+    parser = argparse.ArgumentParser(description="PyTorch RFNet Training")


@luosiqi
The command-line parsing module argparse should not be used, because it dose not use in this scense. It's easy to misunderstand.

@luosiqi suggest that:

Class TrainArgs: def __init__(self, **kwargs): self.depth = kwargs.get('depth', False) self.dateaset = Context.get_parameters('dataset', 'cityscapes') ``

JimmyYang20 · 2022-08-22T01:54:33Z

+                    'best_pred': self.trainer.best_pred,
+                }, is_best)
+
+            # if not self.trainer.args.no_val and \


@luosiqi delete comment code

JimmyYang20 · 2022-08-22T01:56:55Z

+    return args
+
+
+def accuracy(y_true, y_pred, **kwargs):


@luosiqi the ./accuracy.py has this func accuracy in the project, so you can import it.

JimmyYang20 · 2022-08-22T01:58:18Z

+from dataloaders import make_data_loader
+from dataloaders import custom_transforms as tr
+
+def preprocess(image_urls):


this func may be the class(Model)‘s private func

JoeyHwong-gk · 2022-08-22T02:04:57Z

+from utils.metrics import Evaluator
+from tqdm import tqdm
+from dataloaders import make_data_loader
+from sedna.common.class_factory import ClassType, ClassFactory


note the order of import

JoeyHwong-gk · 2022-08-22T02:05:35Z

@@ -0,0 +1,38 @@
+from basemodel import val_args


The import of the relative path should be adjusted.

JoeyHwong-gk · 2022-08-22T02:06:25Z

+__all__ = ('accuracy')
+
+@ClassFactory.register(ClassType.GENERAL)
+def accuracy(y_true, y_pred, **kwargs):


Common keyword. Use alias while register.

JoeyHwong-gk · 2022-08-22T02:06:38Z

+    _, _, test_loader, num_class = make_data_loader(args, test_data=y_true)
+    evaluator = Evaluator(num_class)
+
+    tbar = tqdm(test_loader, desc='\r')


JoeyHwong-gk · 2022-08-22T02:07:18Z

+        if args.cuda:
+            image, target = image.cuda(), target.cuda()
+            if args.depth:
+                depth = depth.cuda()


Check whether the device supports GPU.

JoeyHwong-gk · 2022-08-22T02:10:54Z

+            'cityrand',
+            'target',
+            'xrlab',
+            'e1',


what's the meanning of xrlab and e1

JoeyHwong-gk · 2022-08-22T02:12:06Z

+
+    if args.checkname is None:
+        args.checkname = 'RFNet'
+    print(args)


relace print by using logger

JoeyHwong-gk · 2022-08-22T02:12:40Z

+        choices=[
+            'citylostfound',
+            'cityscapes',
+            'xrlab',


Seems to be inconsistent with the training

JoeyHwong-gk · 2022-08-22T02:14:45Z

+from dataloaders import custom_transforms as tr
+
+class CityscapesSegmentation(data.Dataset):
+    NUM_CLASSES = 19


magic number

JoeyHwong-gk · 2022-08-22T02:15:27Z

+    def __init__(self, args, root=Path.db_root_dir('cityscapes'), data=None, split="train"):
+
+        # self.root = root
+        self.root = "/home/lsq/Dataset/"


JoeyHwong-gk · 2022-08-22T02:18:36Z

@@ -0,0 +1,27 @@
+import torch.nn as nn
+from itertools import chain # 串联多个迭代对象


replace with english will be more general

JoeyHwong-gk · 2022-08-22T02:19:51Z

+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(BasicBlock, [2, 2, 2, 2], **kwargs)


[2, 2, 2, 2] Hyperparameters are restricted.

JoeyHwong-gk · 2022-08-22T02:21:21Z

@@ -0,0 +1,88 @@
+# -*- coding: utf-8 -*-
+# File   : replicate.py
+# Author : Jiayuan Mao


Be aware of the use of other people's code under community constraints

JoeyHwong-gk · 2022-08-22T02:22:49Z

+        label_colours = get_cityscapes_labels()
+    elif dataset == 'target':
+        n_classes = 24
+        label_colours = get_cityscapes_labels()


switch Statements

kubeedge-bot · 2022-08-22T02:22:59Z

@JoeyHwong-gk: changing LGTM is restricted to collaborators

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

JoeyHwong-gk · 2022-08-22T02:23:53Z

Please add kubeedge copyright at the top

JoeyHwong-gk · 2022-08-22T12:12:00Z

+        self.extractor_key = KBResourceConstant.EXTRACTOR.value
+
+        ModelLoadingThread(self, self.task_index).start()
+


Lots of repetitive code, please move up.

JoeyHwong-gk · 2022-08-22T12:12:40Z

+            try:
+                task_index = FileOps.load(task_index_url)
+            except Exception as err:
+                self.log.error(f"{err}")


proposed merge

JoeyHwong-gk · 2022-08-22T12:18:45Z

    DATASET = 'data_process'
    CALLBACK = 'post_process_callback'

+    # TODO


what the todo tags for?

JoeyHwong-gk · 2022-08-22T12:20:57Z

+
+    def __init__(self, task_extractor, **kwargs):
+        self.task_extractor = task_extractor
+        self.log = LOGGER


what's the reasons to define self.log ?

JoeyHwong-gk · 2022-08-22T12:22:22Z

+        for i in range(self.n_class):
+            # sample = BaseDataSource()
+            # sample.x = samples.x[i * partition_length: (i + 1) * partition_length]
+            # sample.y = samples.y[i * partition_length: (i + 1) * partition_length]


JimmyYang20 · 2022-08-23T11:54:09Z

@luosiqi
Note that each line code cannot contain more than 80 characters. Otherwise, the CI check fails.

JimmyYang20 · 2022-08-23T11:58:03Z

+        self.val_args.label_save_path = os.path.join(label_save_dir, "label")
+        self.val_args.save_predicted_image = kwargs.get(
+            "save_predicted_image", "true").lower()
+        self.validator = Validator(self.val_args)


It is not recommended that self.validator = Validator(self.val_args) be placed in the initialization phase.

JimmyYang20 · 2022-08-23T12:00:39Z

+from dataloaders import custom_transforms as tr
+
+class CityscapesSegmentation(data.Dataset):
+    NUM_CLASSES = 24


magic number

JimmyYang20 · 2022-08-23T12:03:23Z

+    txt file which contain image list parser
+    """
+
+    def __init__(self, data_type, func=None):


Use func to handle it!

# func may use this # func = _data_feature_process def _data_feature_process(line: str): res = line.strip().split() return res[:-1], res[-1]

JimmyYang20 · 2022-08-23T12:05:27Z

+                KBResourceConstant.EDGE_KB_DIR.value),
+            task_index=KBResourceConstant.KB_INDEX_NAME.value)
+
+        self.cloud_knowledge_management = CloudKnowledgeManagement(


Don't put it in the this func.
you can put it in "train func" and "eval func"

JimmyYang20 · 2022-08-23T12:05:45Z

+        self.cloud_knowledge_management = CloudKnowledgeManagement(
+            config, estimator=e)
+
+        self.edge_knowledge_management = EdgeKnowledgeManagement(


Don't put it in the this func.
you can put it in "infer func"

JimmyYang20 · 2022-08-23T12:07:13Z

+            self.edge_knowledge_management,
+            unseen_task_allocation)
+
        task_index = FileOps.join_path(config['output_url'],


CloudKnowledgeManagement also has this command, delete it?

JimmyYang20 · 2022-08-23T12:07:50Z

+
+        seen_samples, unseen_samples = unseen_sample_re_recognition(train_data)
+
+        # TODO: retrain temporarily


Delete the omment

JimmyYang20 · 2022-08-23T12:12:05Z

+                      f"KB index save in {task_index}")
        return callback_func(self.estimator, res) if callback_func else res

    def update(self, train_data, valid_data=None, post_process=None, **kwargs):


combine this funcupdate and train into an external func, e,g,:

def train(self): if not has_completed_initial_training: return self._initial_train() return self._update(self)

Combine A and B into an external function.

JimmyYang20 · 2022-08-23T12:16:03Z

+    train_data = IndexDataParse(data_type="train", func=_load_txt_dataset)
+    train_data.parse(train_dataset_url, use_raw=False)
+
+    is_completed_initilization = str(Context.get_parameters("HAS_COMPLETED_INITIAL_TRAINING", "false")).lower()


Put this judgment in the sedna lib

Only one train interface is exposed to users.

JimmyYang20 · 2022-08-23T12:32:30Z

+        self.estimator = set_backend(estimator=estimator, config=config)
+        self.cloud_knowledge_management = cloud_knowledge_management
+        self.edge_knowledge_management = edge_knowledge_management
+


put parameters（cloud_knowledge_management and edge_knowledge_management）to other funs instead of initial func.

JimmyYang20 · 2022-08-23T16:36:26Z

+
+        feedback = {}
+        for i, task in enumerate(task_groups):
+            LOGGER.info(f"MTL Train start {i} : {task.entry}")


task.samples may be [ ]

xiaochanwang · 2022-09-01T07:02:24Z

+        self.task_update_decision = task_update_decision or {
+            "method": "UpdateStrategyDefault"
+        }
+        self.task_update_decision_param = e._parse_param(


if task_update_decision is a callable module instance, then there's no need to set its param by _parse_param.

xiaochanwang · 2022-09-01T07:04:56Z

+        seen_samples.y = np.concatenate(
+            (seen_samples.y, unseen_samples.y), axis=0)
+
+        task_update_decision = ClassFactory.get_cls(


if task_update_decision is callable, then skip ClassFactory.get_cls method and set task index instead.

xiaochanwang · 2022-09-01T07:10:39Z

+
+
+@ClassFactory.register(ClassType.KM)
+class UpdateStrategyDefault:


Add set method to set task index.

Signed-off-by: JimmyYang20 <yangjin39@huawei.com>

Add pylint in ci

kubeedge-bot · 2022-09-08T04:11:32Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign jaypume after the PR has been reviewed.
You can assign the PR to them by writing /assign @jaypume in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kubeedge-bot · 2022-11-02T06:49:14Z

@jaypume: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

luosiqi and others added 5 commits August 12, 2022 22:15

Add lifelong learning example: semantic segmentation

98a4047

Signed-off-by: luosiqi <1587295470@qq.com>

Sedna integrates unstructured data in lib

e5d2291

Signed-off-by: luosiqi <1587295470@qq.com>

Merge pull request #349 from luosiqi/dev-lifelong-n

53e5ae4

Sedna lifelong learning supports unstructured data based on semantic segmentation example

Code check and base model improvement

88827ef

Signed-off-by: luosiqi <1587295470@qq.com>

Merge pull request #350 from luosiqi/dev-lifelong-n

3ed85a0

Code check and base model improvement of unstructured lifelong learning framework

kubeedge-bot requested review from JimmyYang20 and TymonXie August 22, 2022 01:18

kubeedge-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 22, 2022

JimmyYang20 reviewed Aug 22, 2022

View reviewed changes

JoeyHwong-gk reviewed Aug 22, 2022

View reviewed changes

JoeyHwong-gk suggested changes Aug 22, 2022

View reviewed changes

JoeyHwong-gk reviewed Aug 22, 2022

View reviewed changes

JimmyYang20 reviewed Aug 23, 2022

View reviewed changes

xiaochanwang reviewed Sep 1, 2022

View reviewed changes

JimmyYang20 and others added 2 commits September 8, 2022 10:59

Add pylint in ci

572f4e3

Signed-off-by: JimmyYang20 <yangjin39@huawei.com>

Merge pull request #361 from JimmyYang20/lifelong-ci

e12ccac

Add pylint in ci

kubeedge-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 2, 2022



		def train_args():
		parser = argparse.ArgumentParser(description="PyTorch RFNet Training")

		@@ -0,0 +1,27 @@
		import torch.nn as nn
		from itertools import chain # 串联多个迭代对象

		self.extractor_key = KBResourceConstant.EXTRACTOR.value

		ModelLoadingThread(self, self.task_index).start()


		seen_samples, unseen_samples = unseen_sample_re_recognition(train_data)

		# TODO: retrain temporarily



		@ClassFactory.register(ClassType.KM)
		class UpdateStrategyDefault:

Uh oh!

Conversation

jaypume commented Aug 22, 2022

Uh oh!

JimmyYang20 commented Aug 22, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kubeedge-bot commented Aug 22, 2022

Uh oh!

JoeyHwong-gk commented Aug 22, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JimmyYang20 commented Aug 23, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!