Skip to content

Commit 42e7854

Browse files
committed
data: Olist + GitHub Archive como datasets del módulo
Datasets: - data/raw/olist/: 3000 órdenes Olist (ene-feb 2018), 8 tablas relacionales - data/raw/events/github_events.jsonl: 2000 eventos GitHub Archive (2024-01-15) Schema: - sql/001_schema.sql: 8 tablas con FK (customers, sellers, products, orders, order_items, order_payments, order_reviews, category_translations) Scripts: - scripts/load_postgres.py: carga las 8 tablas desde CSVs con psycopg2 - scripts/download_data.py: descarga datasets completos (Kaggle + gharchive.org) Atribución: CREDITS.md
1 parent f277170 commit 42e7854

13 files changed

Lines changed: 20921 additions & 12 deletions

CREDITS.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Credits & Data Sources
2+
3+
## Olist Brazilian E-Commerce Dataset
4+
5+
- **Source:** https://www.kaggle.com/datasets/olistbr/brazilian-ecommerce
6+
- **Provider:** Olist (https://olist.com)
7+
- **License:** Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)
8+
https://creativecommons.org/licenses/by-nc-sa/4.0/
9+
- **Use in this repo:** Sample of 3,000 orders (Jan–Feb 2018) used for educational labs.
10+
Full dataset not included. Run `scripts/download_data.py` to download it.
11+
12+
## GitHub Archive
13+
14+
- **Source:** https://www.gharchive.org
15+
- **Provider:** Ilya Grigorik / GitHub Archive project
16+
- **License:** Creative Commons Attribution 4.0 International (CC BY 4.0)
17+
https://creativecommons.org/licenses/by/4.0/
18+
- **GitHub Event data license:** https://docs.github.com/en/site-policy/github-terms/github-terms-of-service
19+
- **Use in this repo:** Sample of 2,000 public events (Jan 15 2024, 14:00 UTC) used for
20+
streaming and queue labs. Run `scripts/download_data.py` to download more data.
21+
22+
## Notes
23+
24+
All data in `data/raw/` is used strictly for educational, non-commercial purposes.
25+
No personally identifiable information (PII) is included beyond what is already
26+
anonymized in the original public datasets.

data/raw/events/github_events.jsonl

Lines changed: 2000 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
product_category_name,product_category_name_english
2+
beleza_saude,health_beauty
3+
informatica_acessorios,computers_accessories
4+
automotivo,auto
5+
cama_mesa_banho,bed_bath_table
6+
moveis_decoracao,furniture_decor
7+
esporte_lazer,sports_leisure
8+
perfumaria,perfumery
9+
utilidades_domesticas,housewares
10+
telefonia,telephony
11+
relogios_presentes,watches_gifts
12+
alimentos_bebidas,food_drink
13+
bebes,baby
14+
papelaria,stationery
15+
tablets_impressao_imagem,tablets_printing_image
16+
brinquedos,toys
17+
telefonia_fixa,fixed_telephony
18+
ferramentas_jardim,garden_tools
19+
fashion_bolsas_e_acessorios,fashion_bags_accessories
20+
eletroportateis,small_appliances
21+
consoles_games,consoles_games
22+
audio,audio
23+
fashion_calcados,fashion_shoes
24+
cool_stuff,cool_stuff
25+
malas_acessorios,luggage_accessories
26+
climatizacao,air_conditioning
27+
construcao_ferramentas_construcao,construction_tools_construction
28+
moveis_cozinha_area_de_servico_jantar_e_jardim,kitchen_dining_laundry_garden_furniture
29+
construcao_ferramentas_jardim,costruction_tools_garden
30+
fashion_roupa_masculina,fashion_male_clothing
31+
pet_shop,pet_shop
32+
moveis_escritorio,office_furniture
33+
market_place,market_place
34+
eletronicos,electronics
35+
eletrodomesticos,home_appliances
36+
artigos_de_festas,party_supplies
37+
casa_conforto,home_confort
38+
construcao_ferramentas_ferramentas,costruction_tools_tools
39+
agro_industria_e_comercio,agro_industry_and_commerce
40+
moveis_colchao_e_estofado,furniture_mattress_and_upholstery
41+
livros_tecnicos,books_technical
42+
casa_construcao,home_construction
43+
instrumentos_musicais,musical_instruments
44+
moveis_sala,furniture_living_room
45+
construcao_ferramentas_iluminacao,construction_tools_lights
46+
industria_comercio_e_negocios,industry_commerce_and_business
47+
alimentos,food
48+
artes,art
49+
moveis_quarto,furniture_bedroom
50+
livros_interesse_geral,books_general_interest
51+
construcao_ferramentas_seguranca,construction_tools_safety
52+
fashion_underwear_e_moda_praia,fashion_underwear_beach
53+
fashion_esporte,fashion_sport
54+
sinalizacao_e_seguranca,signaling_and_security
55+
pcs,computers
56+
artigos_de_natal,christmas_supplies
57+
fashion_roupa_feminina,fashio_female_clothing
58+
eletrodomesticos_2,home_appliances_2
59+
livros_importados,books_imported
60+
bebidas,drinks
61+
cine_foto,cine_photo
62+
la_cuisine,la_cuisine
63+
musica,music
64+
casa_conforto_2,home_comfort_2
65+
portateis_casa_forno_e_cafe,small_appliances_home_oven_and_coffee
66+
cds_dvds_musicais,cds_dvds_musicals
67+
dvds_blu_ray,dvds_blu_ray
68+
flores,flowers
69+
artes_e_artesanato,arts_and_craftmanship
70+
fraldas_higiene,diapers_and_hygiene
71+
fashion_roupa_infanto_juvenil,fashion_childrens_clothes
72+
seguros_e_servicos,security_and_services

data/raw/olist/customers.csv

Lines changed: 3001 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/order_items.csv

Lines changed: 3399 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/order_payments.csv

Lines changed: 3143 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/order_reviews.csv

Lines changed: 3183 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/orders.csv

Lines changed: 3001 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/products.csv

Lines changed: 2121 additions & 0 deletions
Large diffs are not rendered by default.

data/raw/olist/sellers.csv

Lines changed: 718 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)