Using this library for biometric recognition

[wl2776]

Hi!
Is it possible to apply losses and optimizers, implemented in this library in the task of biometric identification?

Example:
We have a train set of photos of certain persons, all photos are geometrically transformed to the same size, and each photo has a label, denoting the person.
Here is the example of such dataset: https://www.kaggle.com/datasets/vasukipatel/face-recognition-dataset

Typical approach:
Train a classifier, based, say, on Resnet, then remove last linear layer and use other layers as feature detectors, like explained in this link: https://timm.fast.ai/create_model#Turn-any-model-into-a-feature-extractor

So, the question: is it possible to immediately remove last fully-connected layer and train feature extractor using losses and optimizers, implemented in this library?

I have tried to adopt the first example, Optimizing_AUROC_with_ResNet20_on_Imbalanced_CIFAR10, but have met errors, caused by mismatching shapes of tensors. After studying the papers, I have a doubt that this library would suit my needs.

[optmai]

Hi,

Sorry for the late reply!

Thank you for your interest. You can definitely use our library for this purpose.

Basically, you would like to train a feature extractor such that the same person has similar embeddings and different persons have different embeddings. AUC score is a standard measure for face recognition tasks.

But you need to formulate the problem into a binary classification. You can consider the following:

For each person with ID i, it has several face images $X_i={(x_i^1, \ldots, x_i^m)}$. Then for each image $x_i^j$, other images from $X_i$ are positive data and images of other persons are negative data. If the embedding network is denoted by $E$, then you can compute a prediction score of a data $x$ given an anchor data $x_i^j$ by $sim(E(x_i^j), E(x))$, where sim denotes cosine similarity. It means that for each anchor data $x_i^j$ you can compute a AUC score, then average all data for each person and average all persons, gives you the final objective.

This problem is indeed like contrastive learning. You may want to consider the SogCLR, iSogCLR algorithms in LibAUC library for your task.

Please let me know if you have any additional question. You can also send email to tianbao-yang@tamu.edu.