`run_pipeline`'s `votes` param should have more explicit types

[nicobao]

Issue 1

I noticed that the input to run_pipeline(votes) is a list[dict]
See https://github.com/polis-community/red-dwarf/blob/main/reddwarf/implementations/base.py#L56

The actual data when loaded from the polis API looks like list[VoteRecord] with the object described as follows:


@dataclass
class VoteRecord:
    conversation_id: Optional[Union[str, int]]
    datetime: str
    modified: float
    participant_id: Union[str, int]
    statement_id: Union[str, int]
    vote: int
    weight_x_32767: Optional[int]


Is that correct?

Which fields are absolutely mandatory? Can I ignore conversation_id, modified, weight_x_32767, datetime? Put them to None, or simply not putting the field at all? Is modified supposed to be a timestamp? Are dates processed in any way, other than for exports, potentially?

EDIT: Issue 1 still needs to be resolved.

Issue 2

When trying to type this manually, ignoring the existing type, I get an error.

The following code:

@dataclass
class VoteRecord:
    conversation_id: Optional[Union[str, int]]
    datetime: Optional[str]
    modified: Optional[float]
    participant_id: Union[str, int]
    statement_id: Union[str, int]
    vote: int
    weight_x_32767: Optional[int]


@app.route("/math", methods=["POST"])
def getMathResults():
    data = request.get_json()
    if not data or not isinstance(data, list):
        abort(400, description="Expected a JSON array of vote records.")
    try:
        votes: List[VoteRecord] = [VoteRecord(**item) for item in data]
    except (TypeError, ValueError) as e:
        abort(400, description=f"Invalid vote record format: {e}")
    run_pipeline(votes)

Throws the following error:

1. Argument of type "List[VoteRecord]" cannot be assigned to parameter "votes" of type "list[dict[Unknown, Unknown]]" in function "run_pipeline"
     "List[VoteRecord]" is not assignable to "list[dict[Unknown, Unknown]]"
       Type parameter "_T@list" is invariant, but "VoteRecord" is not the same as "dict[Unknown, Unknown]"
       Consider switching from "list" to "Sequence" which is covariant [reportArgumentType]

EDIT: This issue 2 is solved (see next comment)

[nicobao]

My bad, the solution was to use

    vote_dicts = [asdict(vote) for vote in votes]

like so:

from typing import Optional, Union, List
from dataclasses import dataclass, asdict
from flask import Flask, jsonify, request, abort
from reddwarf.data_loader import Loader
from reddwarf.implementations.base import run_pipeline


@dataclass
class VoteRecord:
    conversation_id: Optional[Union[str, int]]
    datetime: Optional[str]
    modified: Optional[float]
    participant_id: Union[str, int]
    statement_id: Union[str, int]
    vote: int
    weight_x_32767: Optional[int]


@app.route("/math", methods=["POST"])
def getMathResults():
    data = request.get_json()
    if not data or not isinstance(data, list):
        abort(400, description="Expected a JSON array of vote records.")
    try:
        votes: List[VoteRecord] = [VoteRecord(**item) for item in data]
    except (TypeError, ValueError) as e:
        abort(400, description=f"Invalid vote record format: {e}")
    vote_dicts = [asdict(vote) for vote in votes]
    run_pipeline(votes=vote_dicts)

[nicobao]

Reopening, as the dataclasses-specific issue is not the main issue here.

The main issue is the lack of typing. Would be nice to provide a pydantic type or something for votes.