lacking training data when train on the second stage.

[wuliwuliy]

This is a very awesome work, and I am very interested in this project. But i have met some problem when train on the second stage, to build the traindataset, the code in LLMGA/llmga/diffusers/my_datasets/dataset_text2img.py need

1. os.path.join(indir,"LLMGA-dataset","LAION","lmdb_train-00000-of-00002.json")
2. os.path.join(indir,"LLMGA-dataset","LAION","lmdb_train-00001-of-00002.json"
   3.os.path.join(indir,"LLMGA-dataset","LAION","laion_3m_prompt.json"

these kinds of json files, which i have tried lots of ways fail to obtain. How can i get these files?
Thank you for your time and answers!

[Zj-BinXia]

Thanks for your interest. I have updated the code in LLMGA/llmga/diffusers/my_datasets/dataset_text2img.py ：

class Text2ImgTrainDataset(Dataset):
def init(self, indir, args=None):
json_path = "./data/jsons/llmga-data/T2I/train.json"
self.prompt_dict = json.load(open(json_path,"r"))

    self.image_folder = "./data/imgs"
   
    self.train_transforms = transforms.Compose(
    [
        transforms.Resize(args.resolution, interpolation=transforms.InterpolationMode.BILINEAR),
        transforms.CenterCrop(args.resolution), #if args.center_crop else transforms.RandomCrop(args.resolution),
        transforms.RandomHorizontalFlip() if args.random_flip else transforms.Lambda(lambda x: x),
        transforms.ToTensor(),
        transforms.Normalize([0.5], [0.5]),
    ]
)

    self.len=len(self.prompt_dict)

def __len__(self):
    return self.len

def __getitem__(self, index):
    prompt = self.prompt_dict[index]["caption"]
    image_path=os.path.join(self.image_folder,self.prompt_dict[index]["image"])
    img=Image.open(image_path)
    img=self.train_transforms(img)

    res = {
            "pixel_values": img,
            "caption": prompt,
            }
    return res

[wuliwuliy]

Thanks for your interest. I have updated the code in LLMGA/llmga/diffusers/my_datasets/dataset_text2img.py ：

class Text2ImgTrainDataset(Dataset): def init(self, indir, args=None): json_path = "./data/jsons/llmga-data/T2I/train.json" self.prompt_dict = json.load(open(json_path,"r"))

    self.image_folder = "./data/imgs"
   
    self.train_transforms = transforms.Compose(
    [
        transforms.Resize(args.resolution, interpolation=transforms.InterpolationMode.BILINEAR),
        transforms.CenterCrop(args.resolution), #if args.center_crop else transforms.RandomCrop(args.resolution),
        transforms.RandomHorizontalFlip() if args.random_flip else transforms.Lambda(lambda x: x),
        transforms.ToTensor(),
        transforms.Normalize([0.5], [0.5]),
    ]
)

    self.len=len(self.prompt_dict)

def __len__(self):
    return self.len

def __getitem__(self, index):
    prompt = self.prompt_dict[index]["caption"]
    image_path=os.path.join(self.image_folder,self.prompt_dict[index]["image"])
    img=Image.open(image_path)
    img=self.train_transforms(img)

    res = {
            "pixel_values": img,
            "caption": prompt,
            }
    return res

Thank you for your quick reply, it works!!
The other three datasets
llmga/diffusers/my_datasets/dataset_inpainting_sdxl.py,
llmga/diffusers/my_datasets/dataset_inpainting.py
llmga/diffusers/my_datasets/dataset_inpainting.py
feel like can also be transformed based on this on the same way.

And by the way, may I ask if there any plan to update detailed datasets or code for validate? about FID, IS, CLIP or LPIPS etc metrics?