[BUG-python/deployment] [CI] Fix `argilla-server` Unit Tests

[JonnyTran]

What happened?

Unit tests are failing in the argilla-server/ component due to two main issues:

The first issue is related to a schema change in text fields and text questions settings, which now include a new use_table property set to False by default. This is causing multiple assertion failures in the tests.

The second issue is with workspace-related endpoints that are returning 500 Internal Server Error instead of the expected status codes.

These issues appear to be causing 21 test failures in the CI pipeline.

Stacktrace and Code to reproduce the bug

# Run the failing tests locally
cd argilla-server
pdm install
export ARGILLA_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/argilla
export ARGILLA_ELASTICSEARCH=http://localhost:9200
export ARGILLA_SEARCH_ENGINE=elasticsearch
pdm test tests/unit/api/handlers/v1/datasets/test_questions.py::TestDatasetQuestions::test_create_dataset_question
pdm test tests/unit/api/handlers/v1/test_datasets.py::TestSuiteDatasets::test_create_dataset_field
pdm test tests/unit/api/handlers/v1/test_workspaces.py::TestSuiteWorkspaces::test_delete_workspace

Extralit/Argilla Version

0.4.1

Extralit Server Version (optional)

0.4.0

ElasticSearch Version

8.15.0

Docker Image (optional)

No response

Python Version

Other (specify in additional context)

Operating System

Linux (Ubuntu/Debian)

Additional context

Additional context

Setup test environment steps:

1. Setup local services:

   # Start PostgreSQL
   docker run -d --name postgres -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=argilla -p 5432:5432 postgres:14
   
   # Start Elasticsearch
   docker run -d --name elasticsearch -p 9200:9200 -e discovery.type=single-node -e xpack.security.enabled=false docker.elastic.co/elasticsearch/elasticsearch:8.15.0
   
   # Start Redis
   docker run -d --name redis -p 6379:6379 redis
   
   # Start MinIO
   docker run -d --name minio -p 9000:9000 -e MINIO_ACCESS_KEY=minioadmin -e MINIO_SECRET_KEY=minioadmin lazybit/minio

2. Setup Python environment:

   cd argilla-server
   pdm install

3. Set environment variables:

   export ARGILLA_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/argilla
   export ARGILLA_ELASTICSEARCH=http://localhost:9200
   export ARGILLA_SEARCH_ENGINE=elasticsearch
   export S3_ENDPOINT=http://localhost:9000
   export S3_ACCESS_KEY=minioadmin
   export S3_SECRET_KEY=minioadmin

4. Run specific failing tests:

   # Run tests for text field schema change
   pdm test tests/unit/api/handlers/v1/datasets/test_questions.py::TestDatasetQuestions::test_create_dataset_question -v
   
   # Run tests for workspace 500 errors
   pdm test tests/unit/api/handlers/v1/test_workspaces.py::TestSuiteWorkspaces::test_delete_workspace -v

Suggested fixes:

1. For the schema changes, update test expectations to include the new use_table: False property or adjust the code to maintain backward compatibility.

2. For the workspace endpoint 500 errors, investigate the internal server error - likely a regression in the workspace deletion implementation.