Skip to content

README.md

Latest commit

 

History

History
284 lines (209 loc) · 8.02 KB

README.md

File metadata and controls

284 lines (209 loc) · 8.02 KB

Porsche Sales API (Medallion Serving + Ingestion)

Spring Boot REST API for a Medallion Architecture data platform.

This service has two responsibilities:

  • Read Gold analytics from Azure PostgreSQL (top_porsche_models)
  • Accept new raw sales events and upload them as JSON files to Azure Blob Storage Bronze container (1-bronze)

Architecture Role

  • Bronze (raw): POST /api/porsche/new-sale validates and uploads sale events as JSON blobs
  • Gold (serving): GET /api/porsche/top-models returns aggregated model metrics already computed by your PySpark pipeline

This keeps ingestion and serving concerns separate while preserving a simple API layer for UI/integrations.

Tech Stack

  • Java 21
  • Spring Boot 4.0.3 (Web MVC, Validation, Data JPA, Actuator)
  • Azure Spring Cloud Storage starter + Azure Blob SDK
  • PostgreSQL JDBC
  • Lombok
  • springdoc OpenAPI (springdoc-openapi-starter-webmvc-ui:3.0.2)

Project Structure

src/main/java/com/porsche/sales_api
  |- config/
  |   |- CorrelationIdFilter.java
  |- controller/
  |   |- PorscheController.java
  |   |- ApiExceptionHandler.java
  |- service/
  |   |- PorscheDataService.java
  |- dto/
  |   |- BronzeUploadResult.java
  |   |- SaleRequest.java
  |   |- UploadSaleResponse.java
  |- entity/
  |   |- TopPorscheModel.java
  |- repository/
  |   |- TopPorscheModelRepository.java

API Endpoints

1) Get Gold Analytics

  • GET /api/porsche/top-models
  • Returns all rows from top_porsche_models

Example response:

[
  {
    "modelName": "718 Cayman",
    "totalRevenue": 10452231.0,
    "carsSold": 92
  }
]

2) Ingest New Bronze Event

  • POST /api/porsche/new-sale
  • Validates request using Bean Validation (fail fast)
  • Uploads JSON payload to Azure Blob with generated name:
    • sale_<sale_id>_<timestamp>.json
  • Returns 201 Created with Location header set to the uploaded blob URL
  • Uses X-Correlation-Id for end-to-end traceability (request, logs, blob metadata)
  • Reads DB/blob credentials from a local gitignored secrets file

Example request:

{
  "sale_id": "015ddcf1-1b32-4a17-b6bf-ae1ebcb91cc9",
  "vin_number": "64321",
  "model_name": "718 Cayman",
  "price": 124986,
  "currency": "EUR",
  "country": "China",
  "sale_timestamp": "2026-03-02T00:44:15.163198",
  "is_electric": false
}

3) Get Latest Sale Details By VIN

  • GET /api/porsche/sales/{vinNumber}
  • Reads Bronze JSON files and returns the latest sale matching the VIN

Example:

GET /api/porsche/sales/64321
X-Correlation-Id: test-corr-002

Example success response:

{
  "message": "Sale uploaded successfully to bronze container",
  "fileName": "sale_015ddcf1-1b32-4a17-b6bf-ae1ebcb91cc9_1763156792000.json"
}

Example response header:

Location: https://<storage-account>.blob.core.windows.net/1-bronze/sale_015ddcf1-1b32-4a17-b6bf-ae1ebcb91cc9_1763156792000.json
X-Correlation-Id: corr-123

Example validation error (400):

{
  "timestamp": "2026-03-15T19:12:42.123Z",
  "status": 400,
  "error": "Validation failed",
  "field": "vinNumber",
  "message": "vin_number must be a numeric value between 10000 and 99999"
}

Validation Rules (Fail Fast)

spring.validation.fail-fast=true is enabled.

SaleRequest constraints:

  • sale_id: required, not blank
  • vin_number: required, numeric string in range 10000..99999
  • model_name: required and must be one of:
    • 911 Carrera
    • Taycan
    • Cayenne
    • Panamera
    • 718 Cayman
    • Macan
    • 911 GT3
  • price: required, > 0
  • currency: required, 3 uppercase letters (ex: EUR)
  • country: required, length 2..64
  • sale_timestamp: required, ISO local datetime
  • is_electric: required, boolean

Swagger / OpenAPI

After startup, open:

  • Swagger UI: http://localhost:8080/swagger-ui/index.html
  • OpenAPI JSON: http://localhost:8080/v3/api-docs

Note: this project uses springdoc-openapi-starter-webmvc-ui:3.0.2 to stay compatible with Spring Boot 4 / Spring Framework 7.

Configuration

Current properties used by the app:

  • spring.datasource.url
  • spring.datasource.username
  • spring.datasource.password
  • spring.datasource.driver-class-name
  • spring.jpa.hibernate.ddl-auto=none
  • azure.storage.connection-string
  • azure.storage.container-name
  • logging.pattern.level (includes MDC correlationId)

Local file based setup (recommended)

This project reads secrets from src/main/resources/application-local-secrets.properties. That file is ignored by git, so your credentials stay local.

  1. Copy template file:
    • src/main/resources/application-local-secrets.properties.example
    • to src/main/resources/application-local-secrets.properties
  2. Fill in your local credentials.

Expected keys in local file:

  • app.db.url
  • app.db.username
  • app.db.password
  • app.azure.storage.connection-string
  • app.azure.storage.container-name

Run Locally

Set-Location "C:\Horatiu\Proiect personale\sales-api\sales-api"
Copy-Item "src\main\resources\application-local-secrets.properties.example" "src\main\resources\application-local-secrets.properties" -ErrorAction SilentlyContinue
.\mvnw.cmd clean spring-boot:run

Run Tests

Set-Location "C:\Horatiu\Proiect personale\sales-api\sales-api"
.\mvnw.cmd test

Tests include controller validation scenarios so invalid payloads do not reach upload service logic.

Operational Notes

  • PorscheDataService logs ingestion lifecycle (received/uploaded/serialization errors)
  • ApiExceptionHandler centralizes validation errors into clean 400 payloads
  • TopPorscheModel is read-only from API perspective and maps to existing pipeline-generated Gold table
  • POST /api/porsche/new-sale sets Location header to the Azure blob URL for downstream traceability
  • CorrelationIdFilter enforces/propagates X-Correlation-Id and stores it in MDC as correlationId
  • Bronze blobs include metadata keys correlation-id and sale-id for downstream Spark tracing

Request Correlation ID (What It Is)

A request correlation ID is a unique identifier attached to one API request and propagated through all logs/events produced by that request.

In this pipeline, the same correlation ID can be:

  • Logged by this API when receiving /new-sale
  • Added to blob metadata or filename-adjacent metadata
  • Emitted by PySpark jobs when reading/processing that blob

Result: you can trace one business event end-to-end (API -> Bronze blob -> Spark processing -> Gold outputs) quickly during debugging and incident response.

Current implementation in this project

  • Inbound header: X-Correlation-Id (optional)
  • If missing, the API generates a UUID
  • Outbound response always includes X-Correlation-Id
  • The value is injected into MDC (correlationId) and appears in logs
  • The same value is attached to uploaded blob metadata (correlation-id)

Example call with custom correlation ID:

Invoke-RestMethod -Method Post `
  -Uri "http://localhost:8080/api/porsche/new-sale" `
  -Headers @{ "X-Correlation-Id" = "spark-trace-2026-03-15-001" } `
  -ContentType "application/json" `
  -Body '{"sale_id":"015ddcf1-1b32-4a17-b6bf-ae1ebcb91cc9","vin_number":"64321","model_name":"718 Cayman","price":124986,"currency":"EUR","country":"China","sale_timestamp":"2026-03-02T00:44:15.163198","is_electric":false}'

Troubleshooting

  • Swagger returns 500 with NoSuchMethodError ControllerAdviceBean
    • Ensure springdoc-openapi-starter-webmvc-ui is 3.0.2 (or newer Boot 4 compatible)
  • Validation returns 400 unexpectedly
    • Confirm payload keys are snake_case (sale_id, vin_number, model_name, sale_timestamp, is_electric)
  • PostgreSQL connection errors
    • Verify Azure PostgreSQL server state, firewall rules, SSL mode, and local secrets values
  • Blob upload failures
    • Validate app.azure.storage.connection-string and that container 1-bronze exists
  • Netty version mismatch warnings from Azure SDK
    • Usually non-blocking; align versions only if runtime issues appear