Dial-in LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues

This repository contains sample code and datasets for the paper "Dial-in LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues". The repository is structured into two main directories: ./data for datasets and ./code for implementation.

---

Dataset Description

Sample dataset is stored at ./data.

• data_labels.csv contains the human-annotated intent labels for all 1507 clusters. The "relative" column indicates domain-specific (1) or out-of-domain (0).
• sample_dialogue_intent_goodness.json contains 110 clusters with human-annotated goodness evaluation labels: 50 good clusters and 60 bad clusters.
• sample_dialogue_intent_label.json contains 110 clusters with human-annotated intent labels.
• data_snapshot.png depicts the data input for intent clustering.

Code Description

Sample code is stored at ./code.

• cluster.py contains code implementation for base (machine learning) models, including AgglomerativeClustering, KMeans, GaussianMixture, and DBSCAN.
• config.py and config.json contain configurations for the proposed method, including model path, instruction prompts, and hyperparameters.
• embedding.py contains code implementation for sentence embedding with SentenceTransformer.
• multi_clustering.py depicts the code implementation of the proposed iterative clustering algorithm.

The fine-tuned LLMs will soon be uploaded to cloud drives for convenient access.